KR102415754B1 - Electronic device, method, computer-readable storage medium and computer program for processing image in virtual space - Google Patents

Abstract

An embodiment provides an electronic device for processing an image in a virtual space, a communication module for receiving at least two image data having different resolutions, a display unit, and at least a portion of the display unit corresponding to a virtual region to be provided in the virtual space The resolution of a region is determined, and while the image corresponding to the virtual space is displayed, the at least partial region of the display unit corresponding to the virtual region is synthesized into image data corresponding to the determined resolution among the at least two image data It may include a control unit for displaying.

Description

ELECTRONIC DEVICE, METHOD, COMPUTER-READABLE STORAGE MEDIUM AND COMPUTER PROGRAM FOR PROCESSING IMAGE IN VIRTUAL SPACE

The present invention relates to an electronic device, a method, a computer-readable recording medium, and a computer program for processing an image in a virtual space.

Recently, services for providing various contents using virtual reality (VR) technology have been commercialized. Virtual reality refers to a technology that enables a person to have a realistic experience in a virtual world created using a computer or the like.

When content is provided using virtual reality technology, a two-dimensional space image may be provided on a virtual space, and accordingly, the user can experience viewing the two-dimensional image in the virtual space. can In the case of such a service, in order to provide a two-dimensional image in an actual virtual space, when the resolution of an image input to the electronic device is a high resolution such as high definition (HD) or full high definition (FHD), the electronic device transmits the When a high-resolution image is displayed on a partial area of a display, there has been a problem in that a texture is linearly interpolated and aliasing occurs.

To solve this problem, conventionally, a technique of sampling an image using a filter and displaying the sampled image on a partial area of a display by an electronic device has been applied. In this conventional technique, the sampling has to be performed for every frame of an image, and due to many processes, the electronic device has a problem of heat generation.

In addition, in order to solve the aliasing problem, it may be considered to apply a deep learning-based image processing technology, but applying the deep learning technology to an electronic device that provides a conventional virtual space is the capacity of the electronic device. , there are difficulties due to the performance of the processor, etc.

Accordingly, an embodiment of the present invention may provide an electronic device, a method, a computer-readable recording medium, and a computer program for processing an image in a virtual space for providing a two-dimensional image with a suitable resolution on the virtual space.

For example, the electronic device may determine (confirm) a resolution corresponding to a partial region of a display to display the 2D image, and receive an image corresponding to the determined resolution from a server providing the 2D image Thus, the received image may be provided through the display.

An embodiment of the present invention provides an electronic device, method, and computer reading method for displaying a two-dimensional image in a region where an image corresponding to a virtual space is displayed so that a two-dimensional image is provided on a virtual space that is a three-dimensional image Possible recording media and computer programs can be provided.

For example, the electronic device may receive a 2D image to be displayed on a separate display area on a virtual space that is a 3D image (eg, a 360-degree image), and display the 3D image and the 2D image. By synthesizing, the 2D image may be overlaid and displayed on the virtual space.

An embodiment provides an electronic device for processing an image in a virtual space, a communication module for receiving at least two image data having different resolutions, a display unit, and at least a portion of the display unit corresponding to a virtual region to be provided in the virtual space The resolution of a region is determined, and while the image corresponding to the virtual space is displayed, the at least partial region of the display unit corresponding to the virtual region is synthesized into image data corresponding to the determined resolution among the at least two image data It may include a control unit for displaying.

The controller may change the at least partial area of the display unit to the determined resolution.

The electronic device may further include a memory in which a distortion map is previously stored, and the controller may correct edge distortion of the synthesized image data using the distortion map.

The control unit is configured to control at least one of a distance between a location where the virtual area is to be provided in the virtual space and a location where an avatar corresponding to a user is to be provided in the real space, a size of the virtual area in the virtual space, and a designated viewing angle corresponding to the avatar. Based on one, the resolution of at least a partial area of the display unit corresponding to the virtual area may be determined.

The controller may identify a location in the virtual space where the avatar corresponding to the user is to be provided, based on the location information of the user in the real space.

The control unit is configured to display the virtual area in the virtual area based on at least one of a distance between a location where the virtual area is to be provided and a location of the user, a size of the virtual area in the virtual space, and a designated viewing angle corresponding to the user in the virtual space. A resolution of at least a portion of the corresponding display unit may be determined.

The controller determines the location of the user in the virtual space based on the location information of the user in the real space, and the virtual area in the virtual space is determined based on the location of the user in the virtual space. A distance between the location to be provided and the location of the user may be checked.

In addition, an embodiment provides a method for processing an image in a virtual space of an electronic device, comprising the steps of: receiving at least two image data having different resolutions; and a display unit of the electronic device corresponding to a virtual region to be provided on the virtual space determining the resolution of at least a partial region of the display unit corresponding to the determined resolution among the at least two image data while displaying the image corresponding to the virtual space; It may include the step of synthesizing the image data to be displayed.

In addition, an embodiment is a computer-readable recording medium storing a computer program, the computer program comprising the steps of: receiving at least two image data having different resolutions; determining the resolution of at least a partial area of the display unit of the device; and while displaying the image corresponding to the virtual space, the at least partial area of the display unit corresponding to the virtual area is selected from among the at least two image data. It may include instructions for causing the processor to perform an operation including the step of synthesizing and displaying image data corresponding to the resolution.

Also, an embodiment is a computer program stored in a computer-readable recording medium, wherein the computer program includes the steps of: receiving at least two image data having different resolutions; determining the resolution of at least a partial area of the display unit of the device; and while displaying the image corresponding to the virtual space, the at least partial area of the display unit corresponding to the virtual area is selected from among the at least two image data. It may include instructions for causing the processor to perform an operation including the step of synthesizing and displaying image data corresponding to the resolution.

An electronic device, a method, a computer-readable recording medium, and a computer program for processing an image in a virtual space according to an embodiment of the present invention may provide a two-dimensional image having an appropriate resolution in the virtual space.

For example, the electronic device may determine (confirm) a resolution corresponding to a partial region of a display to display the 2D image, and receive an image corresponding to the determined resolution from a server providing the 2D image Thus, the received image may be provided through the display.

An electronic device, a method, a computer-readable recording medium, and a computer program for processing an image in a virtual space according to an embodiment of the present invention provide a two-dimensional image corresponding to a virtual space so that a two-dimensional image is provided on a virtual space that is a three-dimensional image. The two-dimensional image may be displayed in a region where the image is displayed.

For example, the electronic device may receive a 2D image to be displayed on a separate display area on a virtual space that is a 3D image (eg, a 360-degree image), and display the 3D image and the 2D image. By synthesizing, the 2D image may be overlaid and displayed on the virtual space.

1 is a block diagram of a system for processing an image in a virtual space according to an embodiment of the present invention.
2 is a flowchart of an operation of processing an image in a virtual space according to an embodiment of the present invention.
3 is a flowchart illustrating an operation of an electronic device that processes an image in a virtual space according to an embodiment of the present invention.
4 is an operation flowchart of an electronic device that processes an image in a virtual space according to an embodiment of the present invention.
5 is a diagram for explaining an operation of selecting an image resolution based on image processing in a virtual space according to an embodiment of the present invention.
6 is a flowchart of an operation of processing an image in a virtual space of an electronic device according to an embodiment of the present invention.

First, the advantages and features of the present invention, and a method for achieving them will become clear with reference to the embodiments described below in detail in conjunction with the accompanying drawings. Here, the present invention is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms, and only these embodiments allow the disclosure of the present invention to be complete, and are common in the technical field to which the present invention pertains. The technical scope of the present invention should be defined by the claims since it is provided by way of example so that those with knowledge can clearly understand the scope of the invention.

In addition, in the following description of the present invention, if it is determined that a detailed description of a well-known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. And, the terms to be described later are terms defined in consideration of functions in the present invention, which may vary depending on the intentions or customs of users, operators, etc., of course. Therefore, the definition should be made based on the technical idea described throughout this specification.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a system for processing an image in a virtual space according to an embodiment of the present invention.

Referring to FIG. 1 , a system may include an electronic device 100 , a server 10 , and/or a server 1 .

The electronic device 100 may include a control unit 101 , a display unit 103 , a memory 105 , and a communication module 107 .

The control unit 101 (also referred to as a processor, a control device, or a control circuit) includes at least one other component (eg, a hardware component (eg, a display unit 103 , a memory 105 , and a communication module) of the connected electronic device 100 . (107)) or software components), and can perform various data processing and operations.

The display unit 103 is a device for visually providing information to a user of the electronic device 100, and includes, for example, a display (or touch screen), a hologram device, or a projector and a control circuit for controlling the projector and the device. can do.

The memory 105 includes various data, for example, software (eg, a program) used by at least one component (the control unit 101 , the memory 105 and the communication module 107 ) of the electronic device 100 . And, it is possible to store input data or output data for a command related thereto. Memory 105 may include volatile memory or non-volatile memory.

The communication module 107 supports establishment of a wired or wireless communication channel between the electronic device 100 and an external device, for example, the server 1 and/or the server 10, and performing communication through the established communication channel. can For example, the communication module 107 may include a wireless communication module (eg, a cellular communication module, a Wi-Fi communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) and/or a wired communication module (eg, a LAN (local area network) including a communication module, or a power line communication module), and using a corresponding communication module among them, a first network (not shown) (eg, Bluetooth, WiFi direct, or short-range communication such as IrDA (infrared data association)) network) or through a second network (not shown) (eg, a cellular network, the Internet, or a telecommunication network such as a computer network (eg, LAN or WAN)) via an external device, eg, the server 1 and/or It can communicate with the server 10 .

According to an embodiment, the electronic device 100 may further include a camera module 109 . The camera module 109 may capture still images and moving images. For example, the camera module 109 may include one or more image sensors and one or more lenses. For example, the camera module 109 may include a depth camera. For example, the camera module 109 may include a 360 degree camera.

According to another embodiment, the camera module 109 is configured as a separate device from the electronic device 100 , so that the electronic device 100 receives an image and/or information obtained from the camera module 109 through the communication module 107 . Alternatively, information corresponding to the image may be received.

According to another embodiment, although not shown, the electronic device 100 scans a real space (real space) with a laser scanner to obtain a point cloud (eg, lidar) ( not shown) may be further included. Alternatively, since the laser scanner is configured as a separate device from the electronic device 100 , the electronic device 100 may receive the point cloud (data, information) from the laser scanner through the communication module 107 .

According to an embodiment, the control unit 101 controls the camera module 109 and/or the laser scanner, or based on the information about the real space (real space) obtained by controlling the communication module 107, A virtual space that is a three-dimensional space model corresponding to the real real space may be generated.

For example, the information on the real space may include image related data corresponding to the real space. For example, the image-related data may include image information of the real space (two-dimensional image data), depth information corresponding to the image information (depth information of a frame corresponding to image information), and/or the image information It may include a point cloud obtained by scanning real space.

For example, the controller 101 may generate the virtual space by generating a three-dimensional space model corresponding to the real space based on the image-related data. For example, the controller 101 may generate a virtual space that is a three-dimensional space model by generating a mesh model (data) based on an image obtained by matching the image data and the point cloud. For example, the controller 101 may generate a virtual space that is a three-dimensional space model similar to the real space by generating a 3D mesh model of the real space based on the image data and the depth information. In addition, the controller 101 may generate the virtual space by using at least one of various conventional virtual space creation techniques.

According to another embodiment, the memory 105 may store three-dimensional space models (virtual space models) corresponding to one or more previously generated virtual spaces, and the controller 101 may store the one or more three-dimensional space models stored in the memory 105 . One of the spatial models may be selected (confirmed, determined) based on a user's selection or a specified condition.

According to another embodiment, the controller 101 receives a three-dimensional space model (three-dimensional space model data) corresponding to one or more virtual spaces from an external device, for example, the server 1 through the communication module 107 . You may.

According to an embodiment, the controller 101 may control the camera module 109 and/or the laser scanner to generate an avatar corresponding to the user based on the acquired information about the user in the real space. have. At least one of various conventional avatar generation techniques may be applied to the operation of generating the avatar.

According to another embodiment, the memory 105 may store one or more previously generated avatars, and the controller 101 selects one of the one or more avatars stored in the memory 105 based on a user's selection or a specified condition. You can choose (confirm, decide).

According to another embodiment, the controller 101 may receive one or more avatars (avatar data) from an external device, for example, the server 1 through the communication module 107 .

According to an embodiment, the control unit 101 of the electronic device 100 determines a resolution of at least a partial area of the display unit corresponding to a virtual area to be provided in a virtual space, and configures the communication module 107 to information corresponding to the determined resolution is transmitted using the While displaying an image corresponding to , the display unit 103 may be controlled to display an image corresponding to the received image data on the at least partial region of the display unit 103 corresponding to the virtual region.

For example, the controller 101 may receive one or more image data, for example, first image data and second image data, from an external device through the communication module 107 . For example, the controller 101 may receive first image data from the server 1 and receive second image data from the server 10 through the communication module 107 . For example, the first image data is image data of an image corresponding to the virtual space, and may be 360 degree image data (virtual space image data). For example, the second image data may be image data corresponding to an image to be displayed on the display unit. For example, the first image data and the second image data may have different resolutions. For example, the image corresponding to the first image data may be displayed on the entire area of the screen of the display unit 103 , and the image corresponding to the second image data may be displayed on at least a partial area of the display unit 103 . It may be an image displayed on the . The at least partial area is an area overlaid on a part of the entire area, and may be a pre-designated area (an area of a pre-designated size on a pre-specified coordinate). Accordingly, the controller 101 is configured so that the image corresponding to the second image data is overlaid and displayed on the designated area of the screen on which the image corresponding to the first image data, which is the image corresponding to the virtual space, is displayed. , an image corresponding to the first image data and an image corresponding to the second image data may be synthesized. For example, when the image corresponding to the first image data and the image corresponding to the second image data have different resolutions, the designated image of the screen in which the image corresponding to the first image data is displayed The resolution of the region may be changed to a resolution corresponding to the second image data, that is, the second resolution according to the synthesis of the two images. For example, when the image corresponding to the first image data and the image corresponding to the second image data are synthesized, the controller 101 is stored in the memory 105 or received through the communication module 107 . The specified distortion effect related information may be used. For example, the designated distortion effect related information may be used to correct an image corresponding to the second image data, and may include a pre-generated and stored distortion map. For example, when the image corresponding to the second image data is overlaid on a partial region of the image corresponding to the first image data, distortion may occur from the center to the edge of the image of the second image data, To solve this problem, the distortion map may be used for distortion correction.

According to an embodiment, the controller 101 determines a location in the virtual space where the avatar corresponding to the user is to be provided, based on the location information of the user in the real space, and the virtual area in the virtual space the display unit 103 corresponding to the virtual area based on at least one of a distance between a location to be provided and a location where the avatar is to be provided, a size of the virtual area in the virtual space, and a designated viewing angle corresponding to the avatar. The resolution of at least a partial area may be determined.

For example, the controller 101 may determine the resolution of the display unit 103 corresponding to the virtual area further based on the resolution of the display unit 103 .

For example, the control unit 101 is configured to determine at least one of a distance between a location where the virtual area is to be provided and a location where the avatar is to be provided, a size of the virtual area in the virtual space, and a designated viewing angle corresponding to the avatar. The size of the at least partial region may be determined based on the determined size of the at least partial region and the resolution of the at least partial region of the display unit 103 corresponding to the virtual region may be determined based on the determined size of the at least partial region and the resolution of the display unit. .

According to an embodiment, the memory 105 may store, in the virtual space, a reference distance between a location where a virtual area is to be provided (a location of the virtual area) and a location where an avatar is to be provided (a location of the avatar), and the virtual area in the virtual space. Reference size information of an area in which the virtual area is to be displayed on the display unit 103, corresponding to the reference size of the area and/or the reference viewing angle corresponding to the avatar, and/or information on the area in which the virtual area is to be displayed on the display unit 103 Reference resolution information may be stored. For example, the electronic device may provide, for each of the reference viewing angles corresponding to the avatar, a corresponding reference distance, a corresponding reference size, reference size information and/or a reference resolution of an area to display the virtual area on the corresponding display unit 103 . information can be stored.

According to an embodiment, the controller 101 is configured to control a reference distance between a location where the virtual area is to be provided and a location where the avatar is to be provided in the virtual space stored in the memory 105, and the virtual space in the virtual space. The resolution of the at least partial region of the display unit 103 corresponding to the virtual region may be determined further based on the resolution information of the region stored corresponding to at least one of the reference size of the region and the reference viewing angle corresponding to the avatar. .

According to an embodiment, the controller 101 determines the location of the user in the virtual space based on location information of the user in the real space, and based on the location of the user in the virtual space , check a distance between a location where the virtual area is to be provided and a location of the user on the virtual space, a distance between a location where the virtual area is provided and a location of the user, a size of the virtual area in the virtual space, and the The resolution of at least a partial area of the display unit 103 corresponding to the virtual area may be determined based on at least one of the designated viewing angles corresponding to the user.

According to an embodiment, the memory 105 may include a reference distance between a location where a virtual area is to be provided (a location of a virtual area) and a user's location in the virtual space, a reference size of the virtual area in the virtual space, and/ Alternatively, reference size information of an area in which the virtual area is to be displayed on the display unit 103 and/or reference resolution information of an area in which the virtual area is to be displayed on the display unit 103 corresponding to the reference viewing angle corresponding to the user may be stored. . For example, the electronic device may provide, for each of the reference viewing angles corresponding to the user, a corresponding reference distance, a corresponding reference size, corresponding reference size information of an area to display the virtual area on the display unit 103 and/or Reference resolution information may be stored.

According to an embodiment, the controller 101 is configured to control a reference distance between a location to which the virtual area is to be provided and a location of the user in the virtual space stored in the memory 105, The resolution of the at least partial area of the display unit 103 corresponding to the virtual area may be further based on resolution information of an area stored corresponding to at least one of a reference size and a reference viewing angle corresponding to the user.

The server 10 may include a control unit 11 , a communication module 13 , and a memory 15 .

The control unit 11 (also referred to as a processor, control device or control circuit) is at least one other component (eg, hardware component (eg, communication module 13 and memory 15) of the connected server 10) or software component) and can perform various data processing and operations.

The communication module 13 may support establishment of a wired or wireless communication channel between the server 10 and an external device, for example, the electronic device 100 and performing communication through the established communication channel.

The memory 15 includes various data used by at least one component of the server 10 (control unit 11, communication module 13, and memory 15), for example, software (eg, a program) and , can store input data or output data for commands related thereto. The memory 15 may include a volatile memory or a non-volatile memory.

According to an embodiment, the server 10 may be a media streaming server. For example, the server 10 may store one or more image (video) data in the memory 15 . For example, the one or more image (moving image) data for each specified resolution may be stored in the memory 15 . The specified resolution is SD (standard definition) (720 x 480), HD (high definition) (1366 x 768), FHD (full high definition) (1920 x 1080), UHD (ultra high definition) (3840 x 2160) and/or other one or more user-specified resolutions.

Meanwhile, although not shown, the server 1 may include at least one configuration corresponding to the configurations of the server 10 .

2 is a flowchart of an operation of processing an image in a virtual space according to an embodiment of the present invention.

Referring to FIG. 2 , when the electronic device 100 receives an image from the server 10 and separately provides the image in a virtual space, a display unit (eg, a display unit) of the electronic device 100 on which the image is to be displayed. Estimating (calculating) the size of a partial region of (103)), which is included in the virtual space and providing the image (also called a virtual display device or a virtual display panel), from the user's real gaze Accordingly, an image corresponding thereto may be received, and the received image may be displayed on the partial area of the display unit.

In operation 210 , the electronic device 100 (eg, the controller 101 of the electronic device 100 ) may determine a resolution of a partial region of the display unit corresponding to the virtual region to be provided in the virtual space.

According to an embodiment, the virtual space includes a three-dimensional space model generated by the electronic device 100 to correspond to a real space, a three-dimensional space model stored in the electronic device 100 and selected by a user or the electronic device 100 . , or a three-dimensional space model received from an external device.

According to an embodiment, the electronic device 100 may identify a position of a display unit that displays a virtual region to be provided on the virtual space, and is configured to estimate a size of the virtual region to be viewed from a user's real gaze. Based on the specified condition, the size of the area in which the virtual area is to be displayed on the display unit may be determined (estimated). The electronic device 100 may determine a resolution corresponding to a partial area of the display unit in which the virtual area is to be displayed, based on the determined size of the display area and the resolution of the display unit of the electronic device 100 .

For example, the specified condition may include a distance between a location where the virtual area is to be provided and a location where an avatar is to be provided on the virtual space, a size of the virtual area on the virtual space, and/or a designated designated condition corresponding to the avatar. It may include an angle of view.

As another example, the specified condition includes a distance between positions of users where the virtual region is to be provided in the virtual space, a size of the virtual region in the virtual space, and/or a specified viewing angle corresponding to the user. can do.

A detailed operation of determining the size of a region to display the virtual region on the display unit based on the specified condition will be described later.

In operation 220, the electronic device 100 may transmit information corresponding to the determined resolution.

According to an embodiment, the electronic device 100 may transmit information corresponding to the determined resolution to the server 10 that provides image data using a communication module (eg, the communication module 107 ).

The server 10 (controller 11 of the server 10 ) that has received the information corresponding to the determined resolution in operation 230 may check image data corresponding to the determined resolution.

According to an embodiment, the server 10 may store one or more image (video) data corresponding to each of the specified resolutions (for each specified resolution) in a memory (eg, the memory 105), and From the above image data, image data having a resolution corresponding to the determined resolution may be checked.

In operation 240 , the server 10 may transmit image data corresponding to the determined resolution.

According to an embodiment, the server 10 may transmit the image data to the electronic device 100 using the communication module 13 . For example, the server 10 may transmit the image data to the electronic device 100 using a streaming technology.

In operation 250 , the electronic device 100 that has received the image data displays the image corresponding to the virtual space, while displaying the image corresponding to the received image data in the at least partial region of the display unit corresponding to the virtual region. Images can be displayed.

According to an embodiment, when receiving the image data, the electronic device 100 provides (includes) an image corresponding to the image data in the virtual space in real time while displaying the virtual space. The display may be displayed on a partial area of the display unit so as to be displayed on the area. For example, the partial region may include the determined resolution of a position of the display unit corresponding to a designated position of the virtual region on the virtual space (a position corresponding to the virtual space region on the virtual space). It may be an area of size.

According to an embodiment, the image data may be streaming data.

According to the embodiment of FIG. 2 described above, the electronic device 100 transmits information corresponding to the determined resolution to the server 10, and the server 10 confirms the image data corresponding to the determined resolution. explained. However, according to another embodiment, the electronic device 100 may select image data stored by the server 10 based on the determined resolution and request transmission of the selected image data. For example, the electronic device 100 may check one or more image (video) data corresponding to each of the specified resolutions stored by the server 10 (for each specified resolution), and an image corresponding to the determined resolution. It may also request the server 10 to transmit data.

2, the virtual space has been described as providing one virtual area (also referred to as a virtual display device or a virtual display panel). However, according to another embodiment, the virtual space includes a plurality of virtual areas. may provide. For example, the electronic device may display one received image in a region corresponding to each of the plurality of virtual regions of the display unit so that the same image is provided to all of the plurality of virtual regions. As another example, the electronic device may receive a plurality of images, and the electronic device may display each of the plurality of received images in each of the plurality of virtual regions of the display unit so that different images are provided to the plurality of virtual regions. It may be displayed in an area corresponding to .

3 is an operation flowchart of an electronic device (eg, the electronic device 100 or the controller 101 of the electronic device 100) that processes an image in a virtual space according to an embodiment of the present invention.

Referring to FIG. 3 , the electronic device may determine the resolution of a partial area of the display unit corresponding to the virtual area to be provided in the virtual space, which is operation 210 of FIG. 2 described above.

In operation 310, the electronic device may check the distance between the virtual area and the avatar in the virtual space.

According to an embodiment, the virtual space may provide (include) a virtual area (virtual display device), and the electronic device may position the avatar on the virtual space.

For example, the electronic device may position the avatar at a specific position in the virtual space to correspond to the user's position in the real space (based on the user's position in the real space).

For example, the distance between the virtual region and the avatar in the virtual space may correspond to a distance between a position of a partial region of the display unit corresponding to the virtual region and a position of the avatar on the display unit. For example, the electronic device calculates a distance between the virtual region and the avatar in the virtual space based on a distance between the position of the partial region of the display unit corresponding to the virtual region and the position of the avatar on the display unit ( can be confirmed). For example, the electronic device may calculate a distance between the virtual area and the avatar in the virtual space based on coordinate information corresponding to the virtual space.

In operation 320, the electronic device may check the size of the virtual region in the virtual space.

According to an embodiment, in the electronic device, the size of the virtual region in the virtual space may be preset (stored).

In operation 330, the electronic device may identify a designated viewing angle corresponding to the avatar.

According to an embodiment, the viewing angle corresponding to the avatar may be previously designated so as to correspond to (or be similar to) the user's actual viewing angle in real space.

According to an embodiment, the viewing angle corresponding to the avatar may be referred to as the viewing angle of the virtual space camera. For example, when it is assumed that the virtual camera is positioned at a position corresponding to the avatar (or the avatar's eyes) located on the virtual space, and an image in a specified direction in the virtual space is obtained from the position of the virtual camera. , the specified field of view corresponding to the avatar may be a predetermined field of view (FOV) of the virtual camera. For example, a specified field of view corresponding to the avatar (a specified field of view corresponding to the user and/or a predetermined field of view (FOV) of the virtual camera) may be 60 degrees, and may be variously changed according to settings.

In operation 340, the electronic device may determine a size of an area to display the virtual area on the display unit based on the distance between the virtual area and the avatar, the size of the virtual area, and a designated viewing angle corresponding to the avatar.

According to an embodiment, the electronic device is configured to provide a reference distance between a location where the virtual area is to be provided and a location where the avatar is to be provided in the virtual space, a reference size of the virtual area in the virtual space, and/or to the avatar. Reference size information of an area in which the virtual area is to be displayed on the display unit and/or reference resolution information of an area in which the virtual area is displayed on the display unit corresponding to the corresponding reference viewing angle may be stored. The electronic device is configured to: a distance between the virtual area and the avatar in the virtual space using the stored reference size information and/or reference resolution information corresponding to the reference distance, the reference size, and/or the reference viewing angle; Based on the size of the virtual area in the virtual space and a designated viewing angle corresponding to the avatar in the virtual space, the display unit may determine a size of the area in which the virtual area is to be displayed.

According to an embodiment, the reference size information and/or the reference resolution information corresponding to the reference distance, the reference size, and/or the reference viewing angle may be determined (set) by the following operation of the electronic device. .

For example, when the electronic device provides a virtual space providing an image to the user for the first time, the electronic device generates a gap between a location where the virtual area is to be provided and a location where the avatar is to be provided in the virtual space based on the reference viewing angle. It is possible to check (measure) a distance between a location on the display unit at which the virtual area is to be provided and a location at which the avatar is provided, which corresponds to a reference distance. Also, the electronic device may check (measure) a size of a region corresponding to the virtual region on the display unit that corresponds to a reference size of the virtual region in the virtual space based on the reference viewing angle, The checked size may be determined as reference size information of an area in which the virtual area is to be displayed on the display unit. Also, the electronic device may determine reference resolution information of an area in which the virtual area is to be displayed on the display unit based on the resolution information of the display unit.

According to an embodiment, the electronic device may include each of the reference distance, the reference size, and/or the reference viewing angle, the distance between the virtual area in the virtual space and the avatar, and the virtual area in the virtual space. The size and/or the designated viewing angle corresponding to the avatar in the virtual space may be compared to determine the size of the area in which the virtual area is to be displayed on the display unit.

For example, the electronic device may check a reference viewing angle corresponding to a specified viewing angle corresponding to the avatar, and check a reference distance, reference size, reference size information and/or reference resolution information corresponding to the checked reference viewing angle. can The electronic device may compare the identified reference distance with the distance between the virtual region and the avatar, and may compare the identified reference size with the size of the virtual region.

For example, when the distance between the virtual region and the avatar is different from the reference distance, a difference value between the two distances may be checked. The size of the virtual region is adjusted based on a ratio (size adjustment ratio) corresponding to the difference value of the distance, and accordingly, the size of the region in which the size is adjusted and the virtual region to be displayed may be adjusted on the display unit. . Accordingly, the electronic device changes (changes the ratio of the reference size) the reference size of the region in which the virtual region is to be displayed on the display unit according to a ratio (size adjustment ratio) corresponding to the difference value of the distance. The changed reference size may be determined as a size of an area in which the virtual area is to be displayed on the display unit.

For example, when the virtual region size is different from the reference size, the electronic device may check a difference value between the two sizes. The size of the virtual region is adjusted based on a ratio corresponding to the difference in size, and accordingly, the size of the region in which the size-adjusted virtual region is to be displayed may be adjusted on the display unit. Accordingly, the electronic device changes the reference size of an area to display the virtual area in the representation ratio according to a ratio corresponding to the difference value of the size (changes the ratio of the reference size) to obtain the changed reference size The size of the area in which the virtual area is to be displayed on the display unit may be determined.

For example, when the distance between the virtual region and the avatar and the reference distance are different, and the size of the virtual region and the reference size are different, a ratio corresponding to the distance difference value between the two distances and the two sizes According to a ratio corresponding to a size difference between the values, the reference size of an area in which the virtual area is to be displayed on the display unit is changed (by changing the ratio of the reference size), and the changed reference size is displayed on the display unit by the virtual area It can be determined by the size of the area to be used.

In operation 350, the electronic device may determine the resolution of the area of the determined size based on the determined size and the resolution of the electronic device.

According to an embodiment, the electronic device may determine the resolution of the display of the electronic device corresponding to the determined size as the resolution of the area to be displayed, that is, the resolution of the area of the determined size.

4 is an operation flowchart of an electronic device (eg, the electronic device 100 or the controller 101 of the electronic device 100) that processes an image in a virtual space according to an embodiment of the present invention.

Referring to FIG. 4 , the electronic device may determine the resolution of a partial area of the display unit corresponding to the virtual area to be provided in the virtual space, which is operation 210 of FIG. 2 described above.

In operation 410, the electronic device may determine the user's location in the virtual space based on the user's location in the real space.

According to an embodiment, the electronic device uses a camera module (eg, the camera module 109) and/or a laser scanner to confirm the location of the user located in the real space, or the electronic device uses a communication module ( For example: information corresponding to the location of the user in the real space may be obtained using the communication module 107). For example, the electronic device may determine the user's location in the virtual space corresponding to the user's location in the real space based on the user's location in the real space.

In operation 420, the electronic device may identify a distance between the virtual area and the user in the virtual space.

According to an embodiment, the virtual space may provide (include) a virtual area (virtual display device).

According to an embodiment, a distance between the virtual area and the determined user location in the virtual space may be checked.

For example, the distance between the virtual area and the determined location of the user in the virtual space corresponds to the location of a partial area of the display unit corresponding to the virtual area and the determined location of the user in the virtual space, on the display unit may correspond to the distance between the positions of For example, the electronic device may calculate a distance between a location of a partial area of the display unit corresponding to the virtual area and a location on the display unit corresponding to the determined location of the user in the virtual space.

In operation 430, the electronic device may check the size of the virtual region in the virtual space.

According to an embodiment, in the electronic device, the size of the virtual region in the virtual space may be preset (stored).

In operation 440, the electronic device may identify a designated viewing angle corresponding to the user.

According to an embodiment, a viewing angle corresponding to the user may be pre-designated to correspond to (or be similar to) the user's actual viewing angle in real space.

According to an embodiment, the viewing angle corresponding to the user may be referred to as the viewing angle of the virtual space camera. For example, when it is assumed that a virtual camera is positioned at a position corresponding to the user (or the user's eye) located on the virtual space, and an image in a specified direction in the virtual space is obtained from the position of the virtual camera. , the specified field of view corresponding to the user may be a predefined field of view (FOV) of the virtual camera. For example, the designated field of view corresponding to the user (pre-specified field of view (FOV) of the virtual camera) may be 60 degrees, and may be variously changed according to settings.

In operation 450 , the electronic device may determine the size of an area to display the virtual area on the display unit based on the distance between the virtual area and the user, the size of the virtual area, and a designated viewing angle corresponding to the user.

According to an embodiment, the electronic device provides a reference distance between a location where the virtual area is provided on the virtual space and a location of a user in the virtual space, a reference size of the virtual area on the virtual space, and/or the Reference size information of an area in which the virtual area is to be displayed on the display unit and/or reference resolution information of an area in which the virtual area is to be displayed on the display unit corresponding to the reference viewing angle corresponding to the user may be stored. The electronic device is configured to: a distance between the virtual area and the user in the virtual space using the stored reference size information and/or reference resolution information corresponding to the reference distance, the reference size, and/or the reference viewing angle; Based on the size of the virtual area in the virtual space and a designated viewing angle corresponding to the user in the virtual space, the display unit may determine a size of an area to display the virtual area.

According to an embodiment, the reference size information and/or the reference resolution information corresponding to the reference distance, the reference size, and/or the reference viewing angle may be determined (set) by the following operation of the electronic device. .

For example, when the electronic device provides a virtual space providing an image to the user for the first time, the electronic device determines the distance between the location where the virtual area is to be provided and the user's location in the virtual space based on the reference viewing angle. It is possible to check (measure) a distance between a location corresponding to a location on the display unit where the virtual area is to be provided and a location corresponding to the user's location corresponding to the reference distance. Also, the electronic device may check (measure) a size of a region corresponding to the virtual region on the display unit that corresponds to the reference size of the virtual region in the virtual space based on the reference viewing angle, The checked size may be determined as reference size information of an area in which the virtual area is to be displayed on the display unit. Also, the electronic device may determine reference resolution information of an area in which the virtual area is to be displayed on the display unit based on the resolution information of the display unit.

According to an embodiment of the present disclosure, the electronic device may include, each of the reference distance, the reference size, and/or the reference viewing angle, the distance between the virtual area and the user in the virtual space, and the virtual area in the virtual space. The size and/or the designated viewing angle corresponding to the user in the virtual space may be compared to determine the size of the area in which the virtual area is to be displayed on the display unit.

For example, the electronic device may check a reference viewing angle corresponding to a specified viewing angle corresponding to the user, and check a reference distance, reference size, reference size information and/or reference resolution information corresponding to the checked reference viewing angle. can The electronic device may compare the identified reference distance, the distance between the virtual area and the user, and compare the confirmed reference size and the size of the virtual area.

For example, when the distance between the virtual area and the user and the reference distance are different, the electronic device may check a difference value between the two distances, and according to a ratio corresponding to the difference value, the virtual area on the display unit By changing the reference size of the area to display the , the changed reference size may be determined as the size of the area in which the virtual area is to be displayed on the display unit.

For example, when the size of the virtual region and the reference size are different from each other, the electronic device may check a difference value between the two sizes, and display the virtual region in the number ratio according to a ratio corresponding to the difference value of the sizes. By changing the reference size of the region to be determined, the changed reference size may be determined as a size of a region in which the virtual region is to be displayed on the display unit.

For example, when the distance between the virtual area and the user and the reference distance are different and the size of the virtual area and the reference size are different, a ratio corresponding to a distance difference value between the two distances and the two sizes By changing the reference size of an area in which the virtual area is to be displayed on the display unit, the changed reference size may be determined as the size of the area in which the virtual area is to be displayed on the display unit according to a ratio corresponding to a size difference between the values.

In operation 460, the electronic device may determine the resolution of the area of the determined size based on the determined size and the resolution of the electronic device.

According to an embodiment, the electronic device may determine the resolution of the display of the electronic device corresponding to the determined size as the resolution of the area to be displayed, that is, the resolution of the area of the determined size.

5 is a diagram for explaining an operation of selecting an image resolution based on image processing in a virtual space according to an embodiment of the present invention.

Referring to FIG. 5 , the resolution of an image displayed on the display unit 103 of the electronic device 100 and received through the communication module may be different according to the location of the virtual region in the virtual space. The size of the virtual region in the virtual space may be specified, but the size of the virtual region may be changed according to a location of the virtual region in the virtual space.

For example, when the electronic device displays the received image on the A region of the display unit to display the received image on the virtual region provided in the virtual space, the electronic device displays information corresponding to the resolution of the A region. It is transmitted to the server, and an image having a resolution corresponding to the resolution of the area A, for example, an image having an SD resolution, may be received from the server.

For example, when the electronic device displays the received image on the B region of the display unit to display the received image on the virtual region provided in the virtual space, the electronic device displays information corresponding to the resolution of the B region. It is transmitted to the server, and an image having a resolution corresponding to the resolution of the region B, for example, an image having an HD resolution, may be received from the server.

For example, when the electronic device displays the received image on the C region of the display unit to display the received image on the virtual region provided in the virtual space, the electronic device displays information corresponding to the resolution of the C region. It is transmitted to the server, and an image having a resolution corresponding to the resolution of the C region, for example, an image having a FHD resolution, may be received from the server.

Referring to FIG. 5 , when the region A, the region B, and the region C are compared, the position of the region A may be located relatively behind the region B and the region C in a virtual space, The location of the region B may be located relatively behind the region C in the virtual space. Accordingly, as for the size of the region on the display unit of the electronic device 100 corresponding to each region, region A is formed in the smallest size, region B is formed in the middle size, and region C is formed in the largest size. can be formed Accordingly, according to an embodiment of the present invention, when the region formed by the electronic device to provide an image is the region A, an image having a small resolution corresponding to the size of the region A can be received and displayed in the region A have. When the region formed by the electronic device to provide an image is the region B, an image having an intermediate size resolution corresponding to the size of the region B may be received and displayed in the region B. When the region formed by the electronic device to provide an image is the region C, an image having an intermediate size resolution corresponding to the size of the region C may be received and displayed in the region C.

6 is a flowchart of an operation of processing an image in a virtual space of an electronic device (eg, the electronic device 100 or the controller 101 of the electronic device 100) according to an embodiment of the present invention.

In operation 610, the electronic device may determine the resolution of at least a partial area of the display unit corresponding to the virtual area to be provided in the virtual space.

According to an embodiment, the electronic device may identify a location in the virtual space where the avatar corresponding to the user is to be provided, based on location information of the user in the real space. based on at least one of a distance between a location where the virtual area is to be provided and a location where the avatar is to be provided, a size of the virtual area in the virtual space, and a designated viewing angle corresponding to the avatar in the virtual space, The resolution of at least a partial area of the display unit corresponding to the virtual area may be determined.

According to an embodiment, the electronic device may determine the resolution of the display unit corresponding to the virtual area further based on the resolution of the display unit. The electronic device is configured to determine the at least partial area based on at least one of a distance between a location where the virtual area is to be provided and a location where the avatar is to be provided, a size of the virtual area in the virtual space, and a designated viewing angle corresponding to the avatar. may determine a size of , and a resolution of at least a partial area of the display unit corresponding to the virtual area may be determined based on the determined size of the at least partial area and the resolution of the display unit.

According to an embodiment, the electronic device may include a reference distance between a location where the virtual area is to be provided and a location where the avatar is to be provided in the virtual space stored in the memory of the electronic device, and a reference size of the virtual area in the virtual space and further based on resolution information of an area stored corresponding to at least one of the reference viewing angles corresponding to the avatar, the resolution of the at least partial area of the display unit corresponding to the virtual area may be determined.

According to an embodiment, the electronic device determines the location of the user in the virtual space based on location information of the user in the real space, and based on the location of the user in the virtual space, The distance between the location where the virtual area is to be provided and the location of the user is checked in space, and the distance between the location where the virtual area is to be provided and the location of the user corresponds to the size of the virtual area in the virtual space and the user A resolution of at least a partial area of the display unit corresponding to the virtual area may be determined based on at least one of the designated viewing angles.

In operation 620, the electronic device may transmit information corresponding to the determined resolution to the server.

In operation 630, the electronic device may receive image data corresponding to the determined resolution from the server based on the transmission of the information corresponding to the determined resolution.

In operation 640 , while displaying the image corresponding to the virtual space, the electronic device may display an image corresponding to the received image data on at least a partial area of the display unit corresponding to the virtual area.

According to an embodiment, the electronic device provides 360-degree image data ( virtual space image data) and second image data corresponding to the image to be displayed on the display unit may be received.

For example, the electronic device receives the first image data from a first external device (eg, the server 1) through the communication module, and receives the second image data from a second external device (eg, the server 10) Image data can be received. For example, the first image data and the second image data may have different resolutions.

For example, the image corresponding to the first image data may be displayed on the entire area of the screen of the display unit, and the image corresponding to the second image data may be an image displayed on at least a partial area of the display unit. have. The at least partial area is an area overlaid on a part of the entire area, and may be a pre-designated area (an area of a pre-designated size on a pre-specified coordinate). Accordingly, the electronic device is configured to display an image corresponding to the second image data overlaid on a designated area of a screen on which an image corresponding to the first image data, which is an image corresponding to the virtual space, is displayed. An image corresponding to the first image data and an image corresponding to the second image data may be synthesized. For example, when the image corresponding to the first image data and the image corresponding to the second image data have different resolutions, the designated image of the screen in which the image corresponding to the first image data is displayed The resolution of the region may be changed to a resolution corresponding to the second image data, that is, the second resolution according to the synthesis of the two images. For example, when the electronic device synthesizes the image corresponding to the first image data and the image corresponding to the second image data, a specified distortion effect stored in the memory of the electronic device or received through the communication module (distortion effect) related information is available. For example, the designated distortion effect related information may be used to correct an image corresponding to the second image data, and may include a pre-generated and stored distortion map. For example, when the image corresponding to the second image data is overlaid on a partial region of the image corresponding to the first image data, distortion may occur from the center to the edge of the image of the second image data, To solve this problem, the distortion map may be used for distortion correction.

Various embodiments of the present document include instructions stored in a machine-readable storage media (eg, the memory 113 (internal memory or external memory)) readable by a machine (eg, a computer). It may be implemented in software (eg, a program). The device is a device capable of calling a stored command from a storage medium and operating according to the called command, and may include an electronic device (eg, the electronic device 100 ) according to the disclosed embodiments. When the command is executed by the control unit (eg, the control unit 101) (or the processor), the control unit may directly or use other components under the control of the control unit to perform a function corresponding to the command. Instructions may include code generated or executed by a compiler or interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not include a signal and is tangible, and does not distinguish that data is semi-permanently or temporarily stored in the storage medium.

According to an exemplary embodiment, the method according to various embodiments disclosed in this document may be included and provided in a computer program product.

According to an embodiment, as a computer-readable recording medium storing a computer program, the operation of determining a resolution of at least a partial area of a display unit of an electronic device corresponding to a virtual area to be provided in a virtual space, corresponding to the determined resolution information corresponding to the determined resolution, receiving image data corresponding to the determined resolution based on the transmission of information corresponding to the determined resolution, and displaying an image corresponding to the virtual space in the virtual area The method may include instructions for causing the processor to perform a method including displaying an image corresponding to the received image data on the at least partial region of the corresponding display unit.

According to an embodiment, as a computer program stored in a computer-readable recording medium, the operation of determining a resolution of at least a partial area of a display unit of an electronic device corresponding to a virtual area to be provided in a virtual space, corresponding to the determined resolution information corresponding to the determined resolution, receiving image data corresponding to the determined resolution based on the transmission of information corresponding to the determined resolution, and displaying an image corresponding to the virtual space in the virtual area The method may include instructions for causing the processor to perform a method including displaying an image corresponding to the received image data on the at least partial region of the corresponding display unit.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains may make various substitutions, modifications, and changes within the scope not departing from the essential characteristics of the present invention. It will be easy to see that this is possible. That is, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments.

Accordingly, the protection scope of the present invention should be interpreted by the claims described below, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

According to an embodiment of the present invention, when providing content using virtual reality technology, an image may be provided in a virtual space, and accordingly, a technology that allows a user to experience viewing the image in a virtual space Available.

Claims (10)

An electronic device for processing an image in a virtual space, comprising:
a communication module for receiving at least two pieces of image data having different resolutions;
display unit; and
determining a resolution of at least a partial area of the display unit corresponding to a virtual area to be provided in a virtual space;
a control unit for synthesizing and displaying the at least partial area of the display unit corresponding to the virtual area into image data corresponding to the determined resolution among the at least two image data while displaying the image corresponding to the virtual space;
includes,
The control unit is configured to: a distance between a location where the virtual area is to be provided in the virtual space and a location where an avatar corresponding to the user is to be provided in the real space, a size of the virtual area in the virtual space, and a designated viewing angle corresponding to the avatar. determining the resolution of at least a partial area of the display unit corresponding to the virtual area based on at least one
An electronic device for processing images in virtual space. According to claim 1,
The control unit is
An electronic device for processing an image in a virtual space that changes the at least partial region of the display unit to the determined resolution. According to claim 1,
The distortion map further includes a pre-stored memory,
The control unit is
An electronic device for processing an image in a virtual space for correcting edge distortion of the synthesized image data using the distortion map delete According to claim 1,
The control unit is
An electronic device for processing an image in a virtual space for identifying a location where an avatar corresponding to the user is to be provided in the virtual space, based on location information of the user in the real space. An electronic device for processing an image in a virtual space, comprising:
a communication module for receiving at least two pieces of image data having different resolutions;
display unit; and
determining a resolution of at least a partial area of the display unit corresponding to a virtual area to be provided in a virtual space;
a control unit for synthesizing and displaying the at least partial area of the display unit corresponding to the virtual area into image data corresponding to the determined resolution among the at least two image data while displaying the image corresponding to the virtual space;
includes,
The control unit is
The virtual area corresponding to the virtual area is based on at least one of a distance between a location where the virtual area is to be provided and a location of the user on the virtual space, a size of the virtual area in the virtual space, and a designated viewing angle corresponding to the user. An electronic device for processing an image in a virtual space that determines a resolution of at least a partial region of a display unit. 7. The method of claim 6,
The control unit is
Determine the location of the user in the virtual space based on the location information of the user in the real space, and a location where the virtual area is to be provided on the virtual space based on the location of the user in the virtual space and an electronic device for processing an image in a virtual space for checking a distance between the user's location and the user's location. A method for processing an image in a virtual space of an electronic device, the method comprising:
Receiving at least two pieces of image data having different resolutions;
determining a resolution of at least a partial area of a display unit of the electronic device corresponding to a virtual area to be provided in a virtual space; and
synthesizing and displaying the at least partial area of the display unit corresponding to the virtual area into image data corresponding to the determined resolution among the at least two image data while displaying the image corresponding to the virtual space;
includes,
The determining of the resolution may include: a distance between a location where the virtual area is to be provided and a location of the user in the virtual space, and a distance between a location where the virtual area is provided and a location where an avatar corresponding to the user is provided on the virtual space , determining a resolution of at least a partial area of the display unit corresponding to the virtual area based on at least one of a size of the virtual area in the virtual space and a designated viewing angle corresponding to the avatar or the user A method for processing images in virtual space. As a computer-readable recording medium storing a computer program,
The computer program is
A computer-readable recording medium comprising instructions for causing a processor to perform the method according to claim 8 . As a computer program stored in a computer-readable recording medium,
The computer program is
A computer program comprising instructions for causing a processor to perform the method according to claim 8 .

Images