KR102640283B1 - Apparatus and method for composing three-dimensional image - Google Patents

Abstract

The stereoscopic image synthesis device models the background to be photographed for each location, generates a background model for each location, sets the distance between the first axes of the camera at the corresponding location based on the background model at each location, and sets the distance between the first axes of the camera based on the distance between the first axes. Thus, a three-dimensional image of the background captured by the camera is acquired, a position corresponding to the object to be photographed is selected, a second axis distance is set based on the first axis distance set at the selected position, and based on the second axis distance, It may include commands to obtain a three-dimensional image of an object captured by a camera and to combine a three-dimensional image of the background and a three-dimensional image of the object.

Description

Apparatus and method for synthesizing three-dimensional images {APPARATUS AND METHOD FOR COMPOSING THREE-DIMENSIONAL IMAGE}

The present invention relates to a stereoscopic image synthesis device and method.

As users become accustomed to online content, they are demanding content in the same form as the facts. According to these user demands, holograms, virtual reality (VR), augmented reality (AR), mixed reality (MR), and extended reality are used to express content similar to reality. , XR), etc. technologies are being proposed.

In particular, in order to provide content similar to reality, technology for producing three-dimensional images is needed. Stereoscopic 3D (three-dimension) shooting is used to produce three-dimensional images, but stereoscopic 3D shooting has the problem of being more complicated than two-dimensional shooting. Accordingly, a technology was proposed to separate the background image and the object image and then combine them.

However, the existing technology only synthesizes the size of the object image by enlarging or reducing it at a set ratio at the set position of the background image, and does not take into account the stereoscopic value, which is an important element of stereoscopic images, so there is a problem in that it is difficult to synthesize stereoscopic images without discomfort. .

KR 10-1392406 B1

Certain embodiments of the present invention can provide a stereoscopic image synthesis device and method that can synthesize a background image and an object image without discomfort.

According to one embodiment, a stereoscopic image synthesis method performed by a computing device may be provided. The stereoscopic image synthesis method includes generating a background model for each location by modeling the background to be photographed for each location, setting a distance between the first axes of the camera at the corresponding location based on the background model for each location, Obtaining a three-dimensional image of the background captured by the camera based on the first axis distance, selecting a target position corresponding to the object to be photographed, and selecting the target position based on the first axis distance set at the target position. Setting a second inter-axis distance at a position, obtaining a three-dimensional image of the object captured by the camera based on the second inter-axis distance, and combining the three-dimensional image of the background and the three-dimensional image of the object. May include steps.

In some embodiments, setting the first inter-axis distance includes changing the inter-axis distance of the camera, checking a stereoscopic value of the background model at the corresponding position according to the changed inter-axis distance, and determining an optimal Repeating the steps of changing the inter-axis distance and confirming the stereoscopic value until the stereoscopic value is confirmed, and setting the inter-axial distance when the optimal stereoscopic value is confirmed as the first inter-axial distance. May include steps.

In some embodiments, the stereoscopic image synthesis method may further include storing the first inter-axis distance at each location and the stereoscopic image of the background in a database.

In some embodiments, setting the distance between the second axes may include loading the distance between the first axes corresponding to the set position from the database.

In some embodiments, the second axis-to-axis distance at the target location may be equal to the first axis-to-axis distance set at the target location.

In some embodiments, setting the second axis-to-axis distance includes generating an object model by modeling the object at the target position, and a three-dimensional value of the object model according to the first axis-to-axis distance set at the target position. It may include confirming and setting the distance between the second axes based on the confirmation result.

The step of setting the second inter-axis distance based on the confirmation result includes repeating the operation of checking the stereoscopic value while finely adjusting the inter-axis distance of the camera until the optimal stereoscopic value is confirmed. It may include setting the distance between axes when the value is confirmed as the second distance between axes.

In some embodiments, the stereoscopic image synthesis method may further include storing the second inter-axis distance at the target location and the stereoscopic image of the object in a database.

According to another embodiment of the present invention, a stereoscopic image synthesis device may be provided including a memory storing at least one command, and a processor executing the command. By executing the command, the processor sets the first inter-axis distance of the camera at a plurality of positions of the background to be photographed, and obtains a three-dimensional image of the background captured by the camera based on the first inter-axis distance, , select a target position corresponding to the object to be photographed from among the plurality of positions, set a second axis distance at the target position based on the first axis distance set at the target position, and set the second axis distance at the target position. Based on this, a three-dimensional image of the object captured by the camera can be obtained, and the three-dimensional image of the background and the three-dimensional image of the object can be synthesized.

In some embodiments, the processor generates a plurality of background models by respectively modeling the background at a plurality of positions, and the first model at a corresponding position is based on the background model corresponding to each position among the plurality of background models. You can set the distance between axes.

In some embodiments, the processor checks the stereoscopic value of the background model at the corresponding location according to the inter-axial distance of the camera at each location, and changes the inter-axial distance until the optimal stereoscopic value is confirmed. may be repeated, and the inter-axis distance when the optimal stereoscopic value is confirmed may be set as the first inter-axis distance.

In some embodiments, the processor generates an object model by modeling the object at the target location, and generates the second inter-axis distance at the target location based on the first inter-axis distance set at the target location and the object model. You can set the distance.

In some embodiments, the processor determines the stereoscopic value of the object model at the target location according to the inter-axial distance of the camera at the target location, and finely adjusts the inter-axial distance until the optimal stereoscopic value is confirmed. The operation may be repeated, and the distance between axes when the optimal stereoscopic value is confirmed may be set as the second distance between axes.

According to another embodiment of the present invention, a stereoscopic image synthesis method performed by a computing device may be provided. The stereoscopic image synthesis method includes the steps of setting the distance between the first axes of the camera at a plurality of positions of the background to be photographed, selecting a target position corresponding to the object to be photographed from among the plurality of positions, and setting a target position at the target position. Setting a second inter-axis distance at the target location based on the first inter-axis distance, and based on the three-dimensional image of the background captured by the camera based on the first inter-axis distance and the second inter-axis distance It may include synthesizing a three-dimensional image of the object captured by the camera.

1 is a diagram showing an example of a stereoscopic image synthesis system according to an embodiment of the present invention.
Figure 2 is a diagram explaining the relationship between the inter-axis distance of the camera and the stereoscopic value.
Figure 3 is a flowchart showing an example of a stereoscopic image synthesis method according to an embodiment of the present invention.
FIG. 4 is a diagram illustrating an example of stereoscopic space creation in a stereoscopic image synthesis method according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating an example of creating a three-dimensional object in a three-dimensional image synthesis method according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating an example of stereoscopic image synthesis in the stereoscopic image synthesis method according to an embodiment of the present invention.
Figure 7 is a diagram showing an example of a computing device according to an embodiment of the present invention.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

In the description below, expressions described as singular may be interpreted as singular or plural, unless explicit expressions such as “one” or “single” are used.

In the flowchart described with reference to the drawings, the order of operations may be changed, several operations may be merged, certain operations may be divided, and certain operations may not be performed.

1 is a diagram showing an example of a stereoscopic image synthesis system according to an embodiment of the present invention.

Referring to FIG. 1, the stereoscopic image synthesis system includes a stereoscopic image synthesis device 100 and a camera 10.

The camera 10 is a camera capable of shooting three-dimensional images. In some embodiments, the camera may be a camera capable of stereoscopic 3D shooting.

The stereoscopic image synthesis device 100 generates a stereoscopic image by combining a background image and an object image captured by the camera 10. The stereoscopic image synthesis apparatus 100 includes a background data generation unit 110, an object data generation unit 120, a database 130, and a synthesis unit 140. In some embodiments, the stereoscopic image synthesis device 100 may be implemented by a computing device.

The background data generator 110 divides the three-dimensional space (background) to be photographed by location, checks the three-dimensional value at each location, and sets the optimal inter-axial distance (IAD). In order to obtain a three-dimensional image of the background at each location, the inter-axis distance of the camera 10 is set to the optimal inter-axis distance at the corresponding location, and a three-dimensional image of the background captured by the camera 10 is acquired. The background data generator 110 stores the optimal inter-axis distance at each location and the three-dimensional image of the background in the database 130. The background data generator 110 may repeat this operation for a plurality of positions where the background is divided.

The object data generator 120 selects a location corresponding to the object to be photographed and retrieves the optimal inter-axis distance at the selected location from the database 130. The object data generator 120 sets the inter-axis distance for the object based on the optimal inter-axis distance at the selected position, sets the inter-axis distance of the camera 10 as the inter-axis distance for the object, and sets the inter-axis distance for the object. Acquire a three-dimensional image of the object. In some embodiments, the camera 10 may photograph an object using a chroma-key technique. The object data generator 120 stores the inter-axis distance for the object at the selected location and the three-dimensional image of the object in the database 130. The object data generator 110 may repeat this operation for a plurality of objects to be photographed.

The synthesis unit 140 loads the three-dimensional image of the background generated by the background data generating unit 110 and the three-dimensional image of the object generated by the object data generating unit 120 from the database 130, and retrieves the three-dimensional image of the background and the object generated by the background data generating unit 110. The three-dimensional images are synthesized to create the final three-dimensional image.

Figure 2 is a diagram explaining the relationship between the inter-axis distance of the camera and the stereoscopic value.

Generally, when a person looks at something, he or she sees a different image due to the gap between the left and right eyes, and this visual difference creates a three-dimensional effect. Factors that cause a three-dimensional effect include binocular parallax depending on the distance between the eyes, binocular convergence, focus control, motion parallax, and psychological factors. In this case, binocular parallax may correspond to the inter-axial distance of the camera.

Referring to FIG. 2, the stereoscopic value is calculated according to the inter-axis distance of the camera corresponding to the spacing (or binocular parallax) of the eyes 210 and 220, and the desired object 240 on the screen 230 is based on the screen 230. A three-dimensional image can be obtained in the future or based on the screen 230. In some embodiments, the stereoscopic value may include depth.

In this case, the stereoscopic value (Z _V ) is binocular disparity (t _e ), the distance from the eye 210 to the screen 230 (Z _D ), and the size of the object 240 represented on the screen 230 (P). is defined by, and binocular disparity (t _e ) can be replaced by the inter-axis distance of the camera. For example, the three-dimensional value (Z _V ) can be defined as Equation 1.

Therefore, since the stereoscopic value can be calculated according to the distance between axes, by setting the distance between axes, the image in which the object 240 enters based on the screen 230 (stereoscopic value is a positive value) or the image coming out based on the screen (stereoscopic value) A negative value) can be obtained. Below, a method of synthesizing a three-dimensional image based on the inter-axis distance will be described with reference to FIGS. 3 to 6.

Figure 3 is a flowchart showing an example of a stereoscopic image synthesis method according to an embodiment of the present invention. FIG. 4 is a diagram illustrating an example of stereoscopic space creation in a stereoscopic image synthesis method according to an embodiment of the present invention, and FIG. 5 is an example of stereoscopic object creation in a stereoscopic image synthesis method according to an embodiment of the present invention. 6 is a diagram illustrating an example of stereoscopic image synthesis in the stereoscopic image synthesis method according to an embodiment of the present invention.

Referring to Figures 3 and 4, the stereoscopic image synthesis device obtains a stereoscopic image of the background of the space to be captured and the inter-axis distance for each location. Specifically, the stereoscopic image synthesis device divides the background by location and models the stereoscopic image of the background for each location (S311). In some embodiments, a stereoscopic image synthesis device may model a stereoscopic image using a well-known stereoscopic modeling algorithm or program. For example, a stereoscopic image synthesis device can model stereoscopic images using 3D MAX, a commercial stereoscopic modeling program.

The stereoscopic image synthesis device sets a random inter-axis distance to the background model at each location (S312) and checks the stereoscopic value of the background model at that location according to the set inter-axis distance (S313). The stereoscopic image synthesis device may repeat the operation of checking the stereoscopic value while changing the distance between axes until the optimal stereoscopic value is confirmed (S314). In some embodiments, the stereoscopic image synthesis device may set the minimum value of the inter-axis distance as the initial value of the inter-axis distance and then repeat the operation of checking the stereoscopic value while increasing the inter-axis distance. In some embodiments, the stereoscopic image synthesis device may set the maximum value of the inter-axis distance as the initial value of the inter-axis distance and then repeat the operation of checking the stereoscopic value while decreasing the inter-axis distance. Accordingly, the stereoscopic image synthesis device can obtain the inter-axis distance and stereoscopic space (i.e., stereoscopic background model) optimized for each location.

The stereoscopic image synthesis device sets the optimal axis-to-axis distance for the background model at all locations and then acquires a stereoscopic image of the background captured based on the optimal axis-to-axis distance (S315). In some embodiments, the stereoscopic image synthesis device may set the inter-axis distance when the optimal stereoscopic value is confirmed as the optimal inter-axis distance. In some embodiments, the stereoscopic image synthesis device may set the inter-axis distance of the camera to the optimal inter-axis distance and obtain a stereoscopic image of the background captured by the camera from the camera. The stereoscopic image synthesis device stores the optimal inter-axis distance at each location and stereoscopic image data of the photographed background in the database (S316).

Through this process, the stereoscopic image synthesis device can obtain a stereoscopic image of the background screen that best represents the inter-axis distance and stereoscopic value for each position that can best represent the stereoscopic value.

Referring to Figures 3 and 5, the stereoscopic image synthesis device acquires an object image based on the inter-axis distance of each position stored in the database. Specifically, the stereoscopic image synthesis device selects a location corresponding to the object to be photographed (S321). The stereoscopic image synthesis device retrieves the inter-axis distance corresponding to the selected position from the database (S322). The stereoscopic image synthesis device determines the inter-axis distance for the object based on the inter-axis distance retrieved from the database (S323-S325) and acquires a stereoscopic image of the photographed object based on the inter-axis distance for the object (S326). In some embodiments, the stereoscopic image synthesis device may set the inter-axial distance of the camera to the inter-axial distance for the object and obtain a stereoscopic image of the object captured by the camera from the camera. The stereoscopic image synthesis device stores the stereoscopic image of the object corresponding to each location in the database (S327).

In some embodiments, the axis distance for an object may be the same as the axis distance retrieved from the database. In some embodiments, the stereoscopic image synthesis device may set an inter-axis distance for an object based on an inter-axis distance retrieved from a database. In some embodiments, an object can be modeled at a selected location, and the three-dimensional value of the object model at that location according to the inter-axis distance retrieved from the database can be checked (S323). The stereoscopic image synthesis device may repeat the operation of checking the stereoscopic value while finely adjusting the distance between axes until the optimal stereoscopic value is confirmed (S324, S325). Through repetition of this process, the stereoscopic image synthesis device can set the optimal inter-axis distance (inter-axial distance for the object) of the object at the corresponding location and an optimized stereoscopic object model. In some embodiments, the stereoscopic image synthesis device may set the inter-axis distance when the optimal stereoscopic value is confirmed as the optimal inter-axis distance for the object. The stereoscopic image synthesis device can store the inter-axis distance for objects corresponding to each position in the database (S327).

Referring to FIGS. 3 and 6, the stereoscopic image synthesis device selects a location and synthesizes a stereoscopic image of the background and a stereoscopic image of the object corresponding to the selected location stored in the database (S331). The stereoscopic image synthesis device obtains the final stereoscopic image by repeating the process of synthesizing the stereoscopic image of the background and the stereoscopic image of the object for a plurality of positions (S333). In some embodiments, when the stereoscopic image synthesis device synthesizes the stereoscopic image of the background and the stereoscopic image of the object, the stereoscopic value of the stereoscopic image of the background and the stereoscopic image of the object may be re-checked (S332).

Through this process, a three-dimensional image can be created without any discomfort in stereoscopic values. Additionally, since the optimal camera axis distance can be set for each location, the settings of shooting variables can be minimized in an actual shooting environment. Therefore, not only professional producers of 3D video but also ordinary people can easily create 3D video content. For example, by combining a three-dimensional image of a performance hall and a three-dimensional image of a performance, a three-dimensional image of the performance desired by the user can be generated at the performance hall selected by the user.

Next, an exemplary computing device capable of implementing a stereoscopic image synthesis device or a stereoscopic image synthesis method according to an embodiment of the present invention will be described with reference to FIG. 7.

Figure 7 is a diagram showing an example of a computing device according to an embodiment of the present invention.

Referring to FIG. 7 , the computing device includes a processor 710, memory 720, storage device 730, communication interface 740, and bus 750. The computing device may further include other general purpose components.

The processor 710 controls the overall operation of each component of the computing device. The processor 710 may be implemented as at least one of various processing units such as a central processing unit (CPU), microprocessor unit (MPU), micro controller unit (MCU), and graphic processing unit (GPU), and may be implemented as a parallel processing unit. It may be possible. Additionally, the processor 710 may perform operations on a program for executing the stereoscopic image synthesis method described above.

The memory 720 stores various data, commands and/or information. The memory 720 may load a computer program from the storage device 730 to execute the stereoscopic image synthesis method described above. The storage device 730 can store programs non-temporarily. The storage device 730 may be implemented as non-volatile memory. The storage device 730 may store a database.

Communication interface 740 supports wireless communication of the computing device.

Bus 750 provides communication functionality between components of a computing device. The bus 750 may be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

When loaded into the memory 720, the computer program may include instructions that cause the processor 710 to perform a stereoscopic image synthesis method. That is, the processor 710 can perform the stereoscopic image synthesis method by executing the command.

The stereoscopic image synthesis method described above can be implemented as a computer-readable computer program on a computer-readable medium. In one embodiment, the computer-readable medium may be a removable or non-removable recording medium. In another embodiment, a computer program recorded on a computer-readable medium may be transmitted to another computing device through a network such as the Internet, and may be installed and executed on the other computing device.

In some embodiments, the computer program models the background to be photographed for each location, generates a background model for each location, sets a first axis distance of the camera at the corresponding location based on the background model at each location, and sets the first axis distance of the camera at the corresponding location. 1 Obtain a three-dimensional image of the background captured by the camera based on the inter-axis distance, select a location corresponding to the object to be photographed, and set a second inter-axis distance based on the first inter-axis distance set at the selected location. and may include instructions for obtaining a three-dimensional image of the object captured by the camera based on the second inter-axis distance and synthesizing the three-dimensional image of the background and the three-dimensional image of the object.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also possible. It falls within the scope of rights.

Claims (14)

A stereoscopic image synthesis method performed by a computing device, comprising:
Creating a background model for each location by dividing the background to be photographed by location and modeling a three-dimensional image of the background for each location;
setting a first inter-axis distance of the camera at a corresponding location based on the background model at each location;
Obtaining a three-dimensional image of the background captured by the camera based on the first axis distance,
Selecting a target location corresponding to the object to be photographed,
Setting a second inter-axis distance at the target position based on the first inter-axis distance set at the target position,
Obtaining a three-dimensional image of the object captured by the camera based on the second inter-axis distance, and
Combining the three-dimensional image of the background and the three-dimensional image of the object
A stereoscopic image synthesis method comprising: In paragraph 1:
The step of setting the distance between the first axes is
Changing the inter-axis distance of the camera,
Confirming the three-dimensional value of the background model at the corresponding position according to the changed inter-axis distance,
repeating the steps of changing the inter-axial distance and confirming the stereoscopic value until a specific stereoscopic value is confirmed; and
Setting the inter-axis distance when the specific stereoscopic value is confirmed as the first inter-axis distance
A stereoscopic image synthesis method comprising: In paragraph 1:
A stereoscopic image synthesis method further comprising storing the first inter-axis distance and the stereoscopic image of the background at each position in a database. In paragraph 3,
Setting the second inter-axis distance includes loading the first inter-axis distance corresponding to the set position from the database. In paragraph 1:
The method of synthesizing three-dimensional images, wherein the second axis-to-axis distance at the target position is equal to the first axis-to-axis distance set at the target position. In paragraph 1:
The step of setting the distance between the second axes is
generating an object model by modeling the object at the target location;
Confirming the three-dimensional value of the object model according to the first axis distance set at the target position, and
Setting the distance between the second axes based on the confirmation result
A stereoscopic image synthesis method comprising: In paragraph 6:
The step of setting the distance between the second axes based on the confirmation result is
Repeating the operation of checking the stereoscopic value while finely adjusting the inter-axis distance of the camera until a specific stereoscopic value is confirmed, and
Setting the inter-axis distance when the specific stereoscopic value is confirmed as the second inter-axis distance
A stereoscopic image synthesis method comprising: In paragraph 1:
A stereoscopic image synthesis method further comprising storing the second inter-axis distance at the target position and the stereoscopic image of the object in a database. memory to store at least one instruction, and
Includes a processor that executes the instructions,
By executing the instruction, the processor
The background to be photographed is divided by location, a three-dimensional image of the background is modeled for each location to generate a background model for each location, and the first camera at a plurality of positions of the background to be photographed is based on the background model at each location. Set the distance between axes respectively,
Obtaining a three-dimensional image of the background captured by the camera based on the first axis distance,
Select a target location corresponding to the object to be photographed from among the plurality of locations,
Setting a second inter-axis distance at the target position based on the first inter-axis distance set at the target position,
Obtaining a three-dimensional image of the object captured by the camera based on the second inter-axis distance,
Combining the three-dimensional image of the background and the three-dimensional image of the object
Stereoscopic image synthesis device. In paragraph 9:
The processor is
Creating a plurality of background models by modeling each of the backgrounds at a plurality of locations,
Setting the distance between the first axes at the corresponding position based on the background model corresponding to each position among the plurality of background models.
Stereoscopic image synthesis device. In paragraph 10:
The processor is
At each location, the stereoscopic value of the background model at the corresponding position is confirmed according to the inter-axial distance of the camera, and the operation of changing the inter-axial distance is repeated until a specific stereoscopic value is confirmed, and the specific stereoscopic value is confirmed. Setting the distance between axes when confirmed as the first distance between axes
Stereoscopic image synthesis device. In paragraph 9:
The processor is
Generate an object model by modeling the object at the target location,
Setting the second inter-axis distance at the target location based on the first inter-axis distance and the object model set at the target location.
Stereoscopic image synthesis device. In paragraph 12:
The processor is
Confirming the stereoscopic value of the object model at the target location according to the inter-axial distance of the camera at the target location, repeating the operation of finely adjusting the inter-axial distance until a specific stereoscopic value is confirmed, and repeating the operation of fine-tuning the inter-axial distance When this is confirmed, the distance between the axes is set to the second distance between the axes.
Stereoscopic image synthesis device. A stereoscopic image synthesis method performed by a computing device, comprising:
The background to be photographed is divided by location, a three-dimensional image of the background is modeled for each location to generate a background model for each location, and the first camera at a plurality of positions of the background to be photographed is based on the background model at each location. Steps of setting the distance between axes,
Selecting a target location corresponding to the object to be photographed from among the plurality of locations,
setting a second inter-axis distance at the target location based on the first inter-axis distance set at the target location; and
Combining a three-dimensional image of the background captured by the camera based on the first axis distance and a three-dimensional image of the object captured by the camera based on the second axis distance.
A stereoscopic image synthesis method comprising:

Images