KR20230080285A - Apparatus and method for composing three-dimensional image - Google Patents

Abstract

An apparatus for composing a three-dimensional image may include instructions of modelling a background to be photographed for each position, generating a background model for each location, setting the distance between the first axes of a camera at the corresponding position based on the background model for each position, acquiring a three-dimensional image of the background captured by the camera based on the distance between the first axes, selecting a position corresponding to the object to be photographed, setting the distance between second axes based on the distance between the first axes at the selected position, acquiring a three-dimensional image of the object captured by the camera based on the distance between the second axes, and combining a three-dimensional image of the background and a three-dimensional image of an object. Therefore, a background image and an object image can be combined without disharmony.

Description

Stereoscopic image synthesis apparatus and method {APPARATUS AND METHOD FOR COMPOSING THREE-DIMENSIONAL IMAGE}

The present invention relates to a stereoscopic image synthesizing apparatus and method.

As users become accustomed to online content, they are demanding content in the same form as the fact. According to these user needs, holograms, virtual reality (VR), augmented reality (AR), mixed reality (MR), and extended reality to express content similar to facts , XR) and the like have been proposed.

In particular, in order to provide content similar to reality, a technique for producing a stereoscopic image is required. Although stereoscopic three-dimensional (3D) imaging is used to produce a stereoscopic image, stereoscopic 3D imaging has a problem in that it is more complicated than 2D imaging. Accordingly, a technique of separately capturing a background image and an object image and then combining them has been proposed.

However, the existing technology only synthesizes by enlarging or reducing the size of the object image at the set position of the background image at a set ratio, but does not consider the stereoscopic value, which is an important element of the stereoscopic image, so it is difficult to synthesize a stereoscopic image without discomfort. .

KR 10-1392406 B1

Certain embodiments of the present invention may provide a stereoscopic image synthesizing apparatus and method capable of synthesizing a background image and an object image without discomfort.

According to one embodiment, a method of synthesizing a 3D image performed by a computing device may be provided. The 3D image synthesizing method includes the steps of modeling a background to be photographed for each location to create a background model for each location, setting a distance between a first axis of a camera at a corresponding location based on the background model for each location, Obtaining a stereoscopic image of the background photographed by the camera based on a first axis distance, selecting a target position corresponding to an object to be photographed, and selecting the target based on the first axis distance set at the target position Setting a second axis-to-axis distance at a location, acquiring a stereoscopic image of the object photographed by the camera based on the second axis-to-axis distance, and synthesizing a stereoscopic image of the background and a stereoscopic image of the object steps may be included.

In some embodiments, the setting of the first axis-to-axis distance may include changing the axis-to-axis distance of the camera, checking a three-dimensional value of the background model at the corresponding position according to the changed axis-to-axis distance, Repeating the step of changing the axis-to-axis distance and the step of checking the stereoscopic value until the stereoscopic value is confirmed, and setting the axis-to-axis distance when the optimal stereoscopic value is confirmed as the first axis-to-axis distance steps may be included.

In some embodiments, the stereoscopic image synthesis method may further include storing the first axis-to-axis distance and the stereoscopic image of the background at each location in a database.

In some embodiments, setting the second axis-to-axis distance may include retrieving the first axis-to-axis distance corresponding to the set position from the database.

In some embodiments, the second axis-to-axis distance at the target location may be the same as the first axis-to-axis distance set at the target location.

In some embodiments, the setting of the second axis-to-axis distance includes generating an object model by modeling the object at the target location, and a three-dimensional value of the object model according to the first axis-to-axis distance set at the target location. It may include the step of confirming, and the step of setting the distance between the second axes based on the check result.

The step of setting the second axis-to-axis distance based on the check result is the step of repeating the operation of checking the stereoscopic value while finely adjusting the axis-to-axis distance of the camera until the optimum stereoscopic value is confirmed, and the optimal stereoscopic value. and setting the axis-to-axis distance when the value is confirmed as the second axis-to-axis distance.

In some embodiments, the 3D image synthesizing method may further include storing the second axis-to-axis distance at the target location and the 3D image of the object in a database.

According to another embodiment of the present invention, a stereoscopic image synthesizing apparatus including a memory for storing at least one command and a processor for executing the command may be provided. By executing the command, the processor sets a first axis-to-axis distance of the camera at a plurality of positions of a background to be captured, respectively, and obtains a stereoscopic image of the background captured by the camera based on the first axis-to-axis distance; , Selecting a target position corresponding to an object to be photographed from among the plurality of positions, setting a second axis-to-axis distance at the target position based on the first axis-to-axis distance set at the target position, and determining the second axis-to-axis distance Based on this, a 3D image of the object photographed by the camera may be obtained, and the 3D image of the background and the 3D image of the object may be synthesized.

In some embodiments, the processor generates a plurality of background models by modeling the background at a plurality of positions, respectively, and the first at a corresponding position based on a background model corresponding to each position among the plurality of background models. The axis distance can be set.

In some embodiments, the processor checks the three-dimensional value of the background model at the corresponding position according to the axis-to-axis distance of the camera at each position, and changes the axis-to-axis distance until an optimal three-dimensional value is identified. may be repeated, and the distance between axes when the optimal three-dimensional value is confirmed may be set as the first distance between axes.

In some embodiments, the processor generates an object model by modeling the object at the target location, and the second axis at the target location based on the first axis-to-axis distance set at the target location and the object model. distance can be set.

In some embodiments, the processor checks a three-dimensional value of the object model at the target location according to the axis-to-axis distance of the camera at the target location, and fine-tunes the axis-to-axis distance until an optimal stereoscopic value is identified. The operation may be repeated, and the distance between axes when the optimal three-dimensional value is confirmed may be set as the second distance between axes.

According to another embodiment of the present invention, a stereoscopic image synthesizing method performed by a computing device may be provided. The stereoscopic image synthesizing method includes setting first axis-to-axis distances of cameras at a plurality of positions of a background to be photographed, selecting a target position corresponding to an object to be photographed from among the plurality of positions, set at the target position. Setting a second axis-to-axis distance at the target position based on the first axis-to-axis distance, and based on the second axis-to-axis distance and the stereoscopic image of the background captured by the camera based on the first axis-to-axis distance A step of synthesizing 3D images of the object photographed by the camera may be included.

1 is a diagram showing an example of a stereoscopic image synthesizing system according to an embodiment of the present invention.
2 is a diagram for explaining the relationship between the distance between axes of cameras and a stereoscopic value.
3 is a flowchart illustrating an example of a method for synthesizing a stereoscopic image according to an embodiment of the present invention.
4 is a diagram explaining an example of generating a 3D space in a 3D image synthesizing method according to an embodiment of the present invention.
5 is a diagram explaining an example of generating a 3D object in a 3D image synthesizing method according to an embodiment of the present invention.
6 is a diagram for explaining an example of stereoscopic image synthesis in the stereoscopic image synthesis method according to an embodiment of the present invention.
7 is a diagram illustrating an example of a computing device according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily carry out the present invention. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Expressions written in the singular in the following description may be interpreted in the singular or plural unless explicit expressions such as “one” or “single” are used.

In the flowchart described with reference to the drawings, the order of operations may be changed, several operations may be merged, a certain operation may be divided, and a specific operation may not be performed.

1 is a diagram showing an example of a stereoscopic image synthesizing system according to an embodiment of the present invention.

Referring to FIG. 1 , a stereoscopic image synthesizing system includes a stereoscopic image synthesizing apparatus 100 and a camera 10 .

The camera 10 is a camera capable of taking stereoscopic images. In some embodiments, the camera may be a camera capable of stereoscopic 3D imaging.

The 3D image synthesis apparatus 100 generates a 3D image by synthesizing a background image and an object image captured by the camera 10 . The stereoscopic image synthesizing apparatus 100 includes a background data generator 110, an object data generator 120, a database 130, and a synthesizer 140. In some embodiments, the stereoscopic image synthesis device 100 may be implemented by a computing device.

The background data generation unit 110 divides the three-dimensional space (background) to be photographed by position, and sets an optimal inter-axial distance (IAD) while checking the three-dimensional value at each position. In order to obtain a 3D image of the background at each position, the axis-to-axis distance of the camera 10 is set to an optimal axis-to-axis distance at the corresponding position, and a 3D image of the background photographed by the camera 10 is obtained. The background data generation unit 110 stores the optimal axis-to-axis distance at each position and the 3D image of the background in the database 130. The background data generating unit 110 may repeat this operation for a plurality of locations where the background is divided.

The object data generation unit 120 selects a position corresponding to an object to be photographed and retrieves an optimal axis-to-axle distance from the database 130 at the selected position. The object data generator 120 sets the axis-to-axis distance for the object based on the optimal axis-to-axis distance at the selected position, sets the axis-to-axis distance of the camera 10 as the axis-to-axis distance for the object, and Obtains a stereoscopic image of an object. In some embodiments, the camera 10 may capture an object using a chroma-key technique. The object data generation unit 120 stores the axis-to-axis distance for the object at the selected location and the stereoscopic image of the object in the database 130. The object data generator 110 may repeat this operation for a plurality of objects to be photographed.

The synthesis unit 140 reads the stereoscopic image of the background generated by the background data generator 110 and the stereoscopic image of the object generated by the object data generator 120 from the database 130, and retrieves the stereoscopic image of the background and the object. 3D images are synthesized to generate a final 3D image.

2 is a diagram for explaining the relationship between the distance between axes of cameras and a stereoscopic value.

Generally, when a person sees something, he or she sees a different image due to the distance between the left and right eyes, and this visual difference creates a three-dimensional effect (three-dimensional effect). Factors that produce a three-dimensional effect include binocular parallax according to the distance between the two eyes, binocular convergence, focus control, motion parallax, and psychological factors. In this case, the binocular disparity may correspond to the distance between axes of the camera.

Referring to FIG. 2 , a desired object 240 on the screen 230 is based on the screen 230 by calculating a stereoscopic value according to the distance between the axis of the camera corresponding to the distance between the eyes 210 and 220 (or binocular parallax). It is possible to obtain a 3D image drawn forward or entered based on the screen 230 with . In some embodiments, the solid value may include depth.

In this case, the stereoscopic value (Z _V ) is binocular parallax (t _e ), the distance from the eye 210 to the screen 230 (Z _D ), and the size (P) of the object 240 displayed on the screen 230 , and the binocular parallax (t _e ) can be replaced by the inter-axis distance of the camera. For example, the three-dimensional value (Z _V ) may be defined as in Equation 1.

Therefore, since a stereoscopic value can be calculated according to the axis-to-axis distance, by setting the axis-to-axis distance, an image of the object 240 entering the screen 230 (positive value) or an image coming out of the screen (stereoscopic value is a positive value) value can be negative). Hereinafter, a method of synthesizing a stereoscopic image based on the axis distance will be described with reference to FIGS. 3 to 6 .

3 is a flowchart illustrating an example of a method for synthesizing a stereoscopic image according to an embodiment of the present invention. 4 is a diagram for explaining an example of generating a 3D space in the 3D image synthesis method according to an embodiment of the present invention, and FIG. 5 is an example of 3D object creation in the 3D image synthesis method according to an embodiment of the present invention. FIG. 6 is a diagram for explaining an example of synthesizing a stereoscopic image in a method for synthesizing a stereoscopic image according to an embodiment of the present invention.

Referring to FIGS. 3 and 4 , the apparatus for synthesizing a stereoscopic image obtains a stereoscopic image of a background of a space to be photographed and an axis-to-axis distance for each position. Specifically, the 3D image synthesizing apparatus divides the background by location and models the 3D image of the background by location (S311). In some embodiments, the 3D image synthesizing apparatus may model a 3D image using a well-known 3D modeling algorithm or program. For example, the 3D image synthesis apparatus may model a 3D image using 3D MAX, a commercial 3D modeling program.

The stereoscopic image synthesis apparatus sets an arbitrary inter-axis distance to the background model at each position (S312), and checks the stereoscopic value of the background model at the corresponding position according to the set inter-axis distance (S313). The apparatus for synthesizing stereoscopic images may repeat an operation of confirming a stereoscopic value while changing an axis distance until an optimal stereoscopic value is identified (S314). In some embodiments, the apparatus for synthesizing a stereoscopic image may set a minimum value of the axis-to-axis distance as an initial value of the axis-to-axis distance, and then repeat an operation of confirming the stereoscopic value while increasing the axis-to-axis distance. In some embodiments, after setting the maximum value of the axis-to-axis distance as the initial value of the axis-to-axis distance, the stereoscopic image synthesizing apparatus may repeat an operation of confirming the stereoscopic value while decreasing the axis-to-axis distance. Accordingly, the 3D image synthesis apparatus may obtain an axis-to-axis distance and a 3D space (ie, a 3D background model) optimized for each location.

The 3D image synthesizing apparatus obtains a 3D image of a photographed background based on the optimal inter-axis distance after setting the optimal axis-to-axis distance for the background model at all locations (S315). In some embodiments, the stereoscopic image synthesizing apparatus may set an inter-axis distance when an optimal stereoscopic value is identified as an optimal inter-axis distance. In some embodiments, the 3D image synthesizing apparatus may set the axis-to-axis distance of the camera to an optimal axis-to-axis distance, and acquire a stereoscopic image of a background photographed by the camera from the camera. The stereoscopic image synthesizing device stores the optimal axis-to-axis distance at each position and the stereoscopic image data of the photographed background in a database (S316).

Through this process, the 3D image synthesizing apparatus can acquire the 3D image of the background screen where the 3D value is most well expressed and the axis-to-axis distance for each position that can best represent the 3D value.

Referring to FIGS. 3 and 5 , the stereoscopic image synthesis apparatus obtains an object image based on the axis-to-axis distance of each position stored in a database. Specifically, the stereoscopic image synthesizing apparatus selects a position corresponding to an object to be photographed (S321). The stereoscopic image synthesizing apparatus reads the axis-to-axis distance corresponding to the selected position from the database (S322). The stereoscopic image synthesizing apparatus determines an axis-to-axis distance for an object based on the axis-to-axis distance retrieved from the database (S323 to S325), and obtains a stereoscopic image of the photographed object based on the axis-to-axis distance for the object (S326). In some embodiments, the 3D image synthesizing apparatus may set an axis-to-axis distance of a camera as an axis-to-axis distance for an object, and acquire a stereoscopic image of an object photographed by the camera from the camera. The 3D image synthesizing device stores the 3D image of the object corresponding to each position in the database (S327).

In some embodiments, the axis-to-axis distance for the object may be the same as the axis-to-axis distance retrieved from the database. In some embodiments, the 3D image synthesis apparatus may set the axis-to-axis distance for the object based on the axis-to-axis distance retrieved from the database. In some embodiments, an object may be modeled at the selected location, and a three-dimensional value of the object model at the location according to the axis-to-axis distance read from the database may be checked (S323). The apparatus for synthesizing stereoscopic images may repeat an operation of confirming a stereoscopic value while finely adjusting an axis-to-axis distance until an optimal stereoscopic value is identified (S324 and S325). Through repetition of this process, the 3D image synthesizing apparatus can set the optimal axis-to-axis distance (axis-to-axis distance for objects) of the object at the corresponding position and an optimized 3D object model. In some embodiments, the stereoscopic image synthesizing apparatus may set an inter-axis distance when an optimal stereoscopic value is identified as an optimal object-axis distance. The stereoscopic image synthesizing apparatus may store the distance between axes for objects corresponding to each position in a database (S327).

Referring to FIGS. 3 and 6 , the 3D image synthesizing device selects a location and synthesizes a 3D image of a background corresponding to the selected location stored in a database and a 3D image of an object (S331). The 3D image synthesis apparatus acquires a final 3D image by repeating a process of synthesizing a 3D image of a background and a 3D image of an object at a plurality of positions (S333). In some embodiments, when synthesizing the background 3D image and the 3D image of the object, the apparatus for synthesizing 3D images may check the 3D values of the 3D image of the background and the 3D image of the object again (S332).

Through this process, a stereoscopic image without a sense of incongruity in stereoscopic values can be created. Also, since it is possible to set the optimal camera-axis distance for each position, it is possible to minimize the setting of shooting variables in an actual shooting environment. Accordingly, not only professional producers of 3D images but also ordinary people can easily create 3D image contents. For example, a 3D image in which a performance desired by a user is reproduced in a performance hall selected by a user may be generated by synthesizing a 3D image of a performance hall and a 3D image of the performance.

Next, an exemplary computing device capable of implementing a 3D image synthesizing device or a 3D image synthesizing method according to an embodiment of the present invention will be described with reference to FIG. 7 .

7 is a diagram illustrating an example of a computing device according to an embodiment of the present invention.

Referring to FIG. 7 , the computing device includes a processor 710, a memory 720, a storage device 730, a communication interface 740, and a bus 750. The computing device may further include other general-purpose components.

The processor 710 controls the overall operation of each component of the computing device. The processor 710 may be implemented as at least one of various processing units such as a central processing unit (CPU), a microprocessor unit (MPU), a micro controller unit (MCU), and a graphic processing unit (GPU), and may be implemented as a parallel processing unit. may be Also, the processor 710 may perform an operation on a program for executing the 3D image synthesizing method described above.

Memory 720 stores various data, commands and/or information. The memory 720 may load a computer program from the storage device 730 to execute the 3D image synthesizing method described above. The storage device 730 may non-temporarily store the program. The storage device 730 may be implemented as a non-volatile memory. The storage device 730 may store a database.

The communication interface 740 supports wireless communication of the computing device.

Bus 750 provides communication between components of a computing device. The bus 750 may be implemented with various types of buses, such as an address bus, a data bus, and a control bus.

The computer program may include instructions that, when loaded into the memory 720, cause the processor 710 to perform a stereoscopic image synthesis method. That is, the processor 710 may perform a stereoscopic image synthesizing method by executing a command.

The stereoscopic image synthesizing method described above may be implemented as a computer readable computer program on a computer readable medium. In one embodiment, the computer readable medium may be a removable recording medium or a fixed recording medium. In another embodiment, a computer program recorded on a computer-readable medium may be transmitted to another computing device through a network such as the Internet, and may be installed and executed in the other computing device.

In some embodiments, the computer program models a background to be photographed for each location to generate a background model for each location, sets a distance between a first axis of a camera at a corresponding location based on the background model for each location, and sets the first axis distance of the camera at the corresponding location. Acquire a stereoscopic image of the background captured by the camera based on the first axis distance, select a position corresponding to an object to be photographed, and set a second axis distance based on the first axis distance set at the selected position and acquiring a 3D image of the object photographed by the camera based on the second axis distance, and synthesizing the 3D image of the object with the 3D image of the background.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also included in the scope of the present invention. that fall within the scope of the right.

Claims (14)

A stereoscopic image synthesizing method performed by a computing device,
Creating a background model for each location by modeling the background to be photographed by location;
Setting a first axis distance of a camera at a corresponding position based on the background model at each position;
Obtaining a stereoscopic image of the background captured by the camera based on the first axis-to-axis distance;
Selecting a target location corresponding to an object to be photographed;
Setting a second axis-to-axis distance at the target location based on the first axis-to-axis distance set at the target location;
obtaining a stereoscopic image of the object photographed by the camera based on the second axis-to-axis distance; and
Synthesizing a 3D image of the background and a 3D image of the object
Stereoscopic image synthesis method comprising a. In paragraph 1,
Setting the first axis-to-axis distance
Changing the axis-to-axis distance of the camera;
Checking the three-dimensional value of the background model at the corresponding position according to the changed axis-to-axis distance;
Repeating the step of changing the axis-to-axis distance and the step of checking the stereoscopic value until an optimum stereoscopic value is identified, and
Setting the axis-to-axis distance as the first axis-to-axis distance when the optimal three-dimensional value is confirmed
Stereoscopic image synthesis method comprising a. In paragraph 1,
The stereoscopic image synthesis method further comprising storing the first axis-to-axis distance and the stereoscopic image of the background at each position in a database. In paragraph 3,
The step of setting the second axis-to-axis distance includes reading the first axis-to-axis distance corresponding to the set position from the database. In paragraph 1,
The second axis-to-axis distance at the target position is equal to the first axis-to-axis distance set at the target position. In paragraph 1,
The step of setting the distance between the second axis
generating an object model by modeling the object at the target location;
Checking a three-dimensional value of the object model according to the first axis-to-axis distance set at the target location; and
Setting the second axis-to-axis distance based on the confirmation result
Stereoscopic image synthesis method comprising a. In paragraph 6,
Setting the second axis-to-axis distance based on the confirmation result
Repeating the operation of checking the stereoscopic value while finely adjusting the distance between axes of the camera until the optimal stereoscopic value is confirmed; and
Setting the axis-to-axis distance as the second axis-to-axis distance when the optimal three-dimensional value is confirmed
Stereoscopic image synthesis method comprising a. In paragraph 1,
The method of synthesizing a stereoscopic image further comprising storing the second axis-to-axis distance at the target location and the stereoscopic image of the object in a database. a memory that stores at least one instruction; and
A processor for executing the instructions;
By executing the instructions, the processor
Setting the first axis-to-axis distances of the cameras at a plurality of positions of the background to be photographed, respectively;
Obtaining a stereoscopic image of the background captured by the camera based on the first axis-to-axis distance;
Selecting a target location corresponding to an object to be photographed from among the plurality of locations;
Setting a second axis-to-axis distance at the target position based on the first axis-to-axis distance set at the target position;
Obtaining a stereoscopic image of the object photographed by the camera based on the second axis-to-axis distance;
synthesizing a stereoscopic image of the background and a stereoscopic image of the object
Stereoscopic image synthesizer. In paragraph 9,
The processor
Creating a plurality of background models by modeling the background at a plurality of locations, respectively;
Setting the first axis-to-axis distance at a corresponding position based on a background model corresponding to each position among the plurality of background models
Stereoscopic image synthesizer. In paragraph 10,
The processor
The three-dimensional value of the background model at the corresponding position according to the axis-to-axis distance of the camera at each position is checked, and the operation of changing the axis-to-axis distance is repeated until the optimum stereoscopic value is confirmed, and the optimal stereoscopic value is determined. Setting the axis-to-axis distance when the value is confirmed as the first axis-to-axis distance
Stereoscopic image synthesizer. In paragraph 9,
The processor
modeling the object at the target location to create an object model;
Setting the second axis-to-axis distance at the target location based on the first axis-to-axis distance set at the target location and the object model
Stereoscopic image synthesizer. In paragraph 12,
The processor
The three-dimensional value of the object model at the target position according to the distance between axes of the camera at the target position is checked, and an operation of finely adjusting the distance between axes is repeated until an optimal three-dimensional value is confirmed, and the optimum Setting the axis-to-axis distance when the three-dimensional value is confirmed as the second axis-to-axis distance
Stereoscopic image synthesizer. A stereoscopic image synthesizing method performed by a computing device,
Setting distances between first axes of cameras at a plurality of locations of a background to be photographed, respectively;
Selecting a target location corresponding to an object to be photographed from among the plurality of locations;
Setting a second axis-to-axis distance at the target location based on the first axis-to-axis distance set at the target location; and
synthesizing a stereoscopic image of the background captured by the camera based on the first axis distance and a stereoscopic image of the object captured by the camera based on the second axis distance
Stereoscopic image synthesis method comprising a.

Images