KR20190082129A - Apparatus and method for rendering 3dimensional image using video - Google Patents

Abstract

According to an embodiment of the present invention, the present invention provides a device for rendering a three-dimensional image using a video and a method thereof. The device for rendering a three-dimensional image using a video comprises: a communication unit; a storage unit; and a processor connected to be operable with the communication unit and the storage unit. The processor receives a plurality of videos of photographing an object in different directions from a plurality of photographing devices through a communication interface, selects frames of a first set corresponding to a specific view from frames individually forming the plurality of videos, and generates a three-dimensional image by using the selected frames of the first set.

Description

[0001] APPARATUS AND METHOD FOR RENDERING [0002] BACKGROUND OF THE INVENTION [0003]

The present invention provides an apparatus and method for rendering a three-dimensional image using moving images.

Recently, as three-dimensional printers are developed and popularized, the necessity and use of three-dimensional images of real objects such as goods and people are increasing. Even in the case of three-dimensional printing, there is an increasing need to acquire accurate three-dimensional images of real objects simply in game development, entertainment devices, and the like. In addition, not only three-dimensional images but also three-dimensional moving images are utilized in various contents such as movies, games, and advertisements, and various methods for obtaining three-dimensional moving images are used.

In general, a motion capture method can be used to acquire three-dimensional moving images. Sensors such as an infrared ray reflector, a light emitter, an ultrasonic generator, a magnetic sensor, or an optical fiber may be attached to a joint of a subject (e.g., a person) for motion capture. A three-dimensional image rendering apparatus using motion capture can detect a three-dimensional position of an attached sensor to acquire motion data of a subject, and generate a three-dimensional image and / or a three-dimensional image using the acquired motion data.

In this case, since devices for acquiring motion data from the sensors and sensors for attaching to the subject for motion capture and performing three-dimensional image rendering using the motion data are required, It is difficult to obtain a 3D image and / or 3D video. Also, since such equipment is expensive and requires a separate studio for motion capture, it is difficult for a general user to easily obtain a three-dimensional image and / or a three-dimensional image.

Accordingly, there is a need for a method for simply acquiring a three-dimensional image and / or a three-dimensional image without a separate apparatus for motion capture.

An object of the present invention is to provide an apparatus and a method for rendering a three-dimensional image using moving images.

Specifically, it is an object of the present invention to provide a three-dimensional image rendering apparatus and method for acquiring a three-dimensional image and / or a three-dimensional moving image easily and simply without additional equipment for motion capture.

The problems of the present invention are not limited to the above-mentioned problems, and other problems not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided an apparatus and method for rendering a three-dimensional image using a moving image according to an exemplary embodiment of the present invention. An apparatus for rendering a three-dimensional image using moving images according to an embodiment of the present invention includes a communication unit; A storage unit; And a processor operatively connected to the communication unit and the storage unit, wherein the processor receives a plurality of moving pictures photographed in different directions from a plurality of photographing apparatuses via the communication interface, and the plurality Selects a first set of frames corresponding to a specific time among the frames constituting each of the moving images of the moving images, and generates a three-dimensional image using the selected first set of frames.

According to an embodiment of the present invention, there is provided a method of rendering a three-dimensional image using moving images, the method comprising: receiving a plurality of moving images of a subject photographed in different directions from a plurality of photographing devices; Selecting a first set of frames corresponding to a specific time among frames constituting each of the plurality of moving images; And generating a three-dimensional image using the selected first set of frames.

The details of other embodiments are included in the detailed description and drawings.

The present invention does not have separate equipment for motion capture by generating a plurality of three-dimensional images using a moving image photographed through a plurality of photographing apparatuses and generating a three-dimensional moving image using a plurality of three-dimensional images , It is possible to easily and easily generate a three-dimensional moving image while minimizing the cost for generating the three-dimensional moving image.

The effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the present invention.

1 is a schematic diagram for explaining a three-dimensional image rendering system using moving images according to an embodiment of the present invention.
2 is a schematic diagram for explaining a three-dimensional image rendering apparatus according to an embodiment of the present invention.
3 is a schematic flowchart illustrating a method of rendering a three-dimensional image using moving images in a three-dimensional image rendering apparatus according to an embodiment of the present invention.
4 is a schematic flowchart for explaining a method for generating a three-dimensional moving image using a three-dimensional image in a three-dimensional image rendering apparatus according to an embodiment of the present invention.
FIG. 5 is a diagram for explaining a method for acquiring an optimal frame corresponding to a specific time for generating a three-dimensional image in a three-dimensional image rendering apparatus according to an embodiment of the present invention.
FIG. 6 is an exemplary view of an interface screen showing frames acquired for respective positions of a plurality of photographing apparatuses according to an embodiment of the present invention.
FIGS. 7A and 7B are exemplary views of an interface representing a three-dimensional model and a three-dimensional image generated through a three-dimensional image rendering apparatus according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. It should be understood, however, that the invention is not limited to the disclosed embodiments, but is capable of many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, To fully disclose the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims.

Although the first, second, etc. are used to describe various components, it goes without saying that these components are not limited by these terms. These terms are used only to distinguish one component from another. Therefore, it is needless to say that the first component mentioned below may be the second component within the technical spirit of the present invention.

Like reference numerals refer to like elements throughout the specification.

It is to be understood that each of the features of the various embodiments of the present invention may be combined or combined with each other partially or entirely and technically various interlocking and driving is possible as will be appreciated by those skilled in the art, It may be possible to cooperate with each other in association.

Various embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

1 is a schematic diagram for explaining a three-dimensional image rendering system using moving images according to an embodiment of the present invention.

1, a three-dimensional image rendering system 1000 includes a booth 100 for photographing a subject and a three-dimensional image rendering device 200 for generating a three-dimensional image using the moving image. ).

The booth 100 includes a plurality of guiders 110 installed vertically with respect to the subject TS when the subject TS is positioned inside the booth 100 and the guiders 110 So as to form a dome-shaped or cylindrical space. A plurality of photographing apparatuses 120 may be distributedly installed in each of the guiders 110. In the illustrated embodiment, six imaging devices are installed for each guider, but the number of guiding devices and the number of imaging devices may be variously configured. Each of the guiders 110 may be provided with identification information (e.g., ID or number, etc.), and such identification information may be provided to the three-dimensional image rendering apparatus 200. [

Each of the plurality of photographing apparatuses 120 is connected to the three-dimensional image rendering apparatus 200 in a wired manner and is controlled by the three-dimensional image rendering apparatus 200 to control the front of the subject TS located in the booth 100, A plurality of moving images corresponding to various directions such as the back, side, 45 degrees, 60 degrees, and the like can be photographed and the photographed moving image can be transmitted to the three-dimensional image rendering apparatus 200. [ Each of the photographed videos may be a high-quality video that can be used to generate a three-dimensional image. Further, each of the photographed moving images may include at least a part of a subject for rendering as a three-dimensional image. For example, videos are videos that show the front of the head, the front of the upper body, the front of the lower body, the front of the head, the side of the head, the side of the upper body, the side of the lower body, the side of the foot, the back of the head, the back of the upper body, .

 Specifically, each of the plurality of photographing apparatuses 120 may receive a control signal for controlling the three-dimensional image rendering apparatus 200 to photograph a moving image of the subject, and may simultaneously photograph the moving image of the subject according to the control signal. When the photographing is completed, each of the plurality of photographing apparatuses 120 can transmit the photographed moving image to the three-dimensional image rendering apparatus 200. Each of the plurality of photographing apparatuses 120 may be provided with identification information (e.g., ID or number, etc.), and such identification information may be provided to the three-dimensional image rendering apparatus 200 together with the moving image.

Each of the plurality of photographing apparatuses 120 may transmit the position information of each of the plurality of photographing apparatuses 120 to the three-dimensional image rendering apparatus 200 together with the moving pictures of the object. Each of the plurality of photographing apparatuses 120 stores in advance information indicating a position of a guider where each of the plurality of photographing apparatuses 120 is disposed and a height at which each of the plurality of photographing apparatuses 120 are arranged as position information, The position information may be transmitted to the three-dimensional image rendering apparatus 200 together with the moving image. In various embodiments, the three-dimensional image rendering apparatus 200 may display the position of the guider where each photographing apparatus is disposed, the height of the photographing apparatus disposed in the guider, etc. corresponding to the identification information of each of the plurality of photographing apparatuses 120 When the information is previously stored as the position information, each of the plurality of photographing apparatuses 120 may transmit the moving image together with the identification information to the three-dimensional image rendering apparatus 200.

In various embodiments, the booth 100 may further include a control module for controlling each of the plurality of photographing apparatuses 120. The plurality of photographing apparatuses 120, the control module, and the three-dimensional image rendering apparatus 200 may be connected by wire or wirelessly. For example, the control module may receive a request for capturing a moving image for a subject from the three-dimensional image rendering apparatus 200, and control each of the plurality of photographing apparatuses 120 to simultaneously photograph the subject have. When the photographing is completed, the plurality of photographing apparatuses 120 can transmit photographed moving images to the control module.

Each of the plurality of photographing apparatuses 120 may transmit the identification information of the photographing apparatus to the control module together with the moving image. The control module may store the position information in advance in correspondence with the identification information of each of the plurality of photographing apparatuses 120. For example, in the control module, the position information includes information indicating the position of the guider in which each photographing device is disposed, the height of the photographing device disposed in the guider, etc. corresponding to the identification information of each of the plurality of photographing devices 120 It can be stored in advance as information. The control module receiving the moving images and the identification information confirms the position information corresponding to the identification information of each of the photographing apparatuses 120 and transmits the moving images together with the confirmed position information to the three-dimensional image rendering apparatus 200 . In various embodiments, the control module may communicate the moving image to the three-dimensional image rendering apparatus 200 together with the identification information received from each of the plurality of photographing apparatuses 120.

In various embodiments, the three-dimensional image rendering system 1000 may further include a hub device that connects the plurality of imaging devices 120 to the three-dimensional image rendering device 200. The plurality of photographing apparatuses 120 can receive a control signal from the three-dimensional image rendering apparatus 200 through the hub device, and photograph a moving image of the subject according to the control signal. The plurality of photographing apparatuses 120 may transmit photographed moving images to the three-dimensional image rendering apparatus 200 through a hub device.

In various embodiments, each of the plurality of imaging devices 120 may further include a wireless communication module, and may be wirelessly connected to the three-dimensional image rendering device 200 through a wireless communication module. In this case, each of the plurality of photographing apparatuses 120 receives a control signal from the three-dimensional image rendering apparatus 200 through the wireless communication module, captures moving images of the subject in accordance with the control signal, To the three-dimensional image rendering apparatus 200. [

The three-dimensional image rendering apparatus 200 receives the moving images captured by the plurality of photographing apparatuses 120, selects a first set of frames corresponding to a specific time among the frames constituting each of the received moving images, The first set of frames may be used to generate a three-dimensional image. At this time, the generated three-dimensional image can be defined as the first three-dimensional image. Specifically, the three-dimensional image rendering apparatus 200 transmits a moving image capturing request for each of the plurality of image capturing apparatuses 120 to each of the plurality of image capturing apparatuses 120, To the control module that controls the operation. The three-dimensional image rendering apparatus 200 can receive moving images from the plurality of photographing apparatuses 120 or the control module upon request. For example, when there are 30 photographing apparatuses installed in the guider 110, the three-dimensional image rendering apparatus 200 can receive thirty moving pictures photographed by thirty photographing apparatuses. In various embodiments, the three-dimensional image rendering apparatus 200 may receive identification information or location information of each of the plurality of photographing apparatuses 120 together with a plurality of moving images.

The three-dimensional image rendering apparatus 200 can select thirty frames corresponding to a specific time among the frames constituting each of the received 30 moving pictures as the first set of frames. When selecting the first set of frames corresponding to a specific time, the three-dimensional image rendering apparatus 200 can determine whether the frame corresponding to the specific time of each moving image is an optimal frame that can be used for rendering the three-dimensional image. Here, the optimal frame may be an image with high sharpness. The three-dimensional image rendering apparatus 200 may calculate the sharpness value of each frame corresponding to a specific time, and may determine the frame as a frame having a high sharpness if the calculated sharpness value is greater than or equal to a predetermined threshold sharpness value.

When at least a part of frames corresponding to a specific time is not determined as an optimal frame, the 3D image rendering apparatus 200 may use various schemes for obtaining an optimal frame corresponding to a specific time. Here, a frame not determined as an optimal frame may be a low-definition frame, a frame including a blurred portion, or a blacked out frame. For example, when the frame at a specific time is a frame with low definition, the three-dimensional image rendering apparatus 200 performs image processing for increasing the edge strength of the frame at a specific time to improve the sharpness of the frame at a specific time , And thus the image-processed frame can be selected as the optimum frame. In various embodiments, the three-dimensional image rendering apparatus 200 may perform image processing for increasing a color value, a saturation value, or a luminance value for a frame at a specific time, and may select the image-processed frame as an optimal frame. In various embodiments, if the frame includes a blurred portion, the three-dimensional image rendering apparatus 200 may correct the blurred portion of the frame at a specific time based on the previous frame or the next frame, and may also select the corrected frame as an optimal frame have. In various embodiments, the three-dimensional image rendering apparatus 200 can select, as an optimal frame, a frame having a sharpness of a previous frame or a next frame of a frame at a specific time and a pixel difference between the frame of a specific time and a threshold value. The manner of obtaining the optimum frame may be various and is not limited as described above.

The three-dimensional image rendering apparatus 200 can generate a first three-dimensional image based on the optimal frame and the position information selected for each moving image corresponding to a specific time. The generated first three-dimensional image may include a three-dimensional object. Here, the position information may include information indicating the position of the guider where each photographing apparatus is disposed, the height of the photographing apparatus disposed in the guider, and the like. For example, the three-dimensional image rendering apparatus 200 may use the position of the guider in which each photographing apparatus is disposed, the height of the photographing apparatus disposed in the guider, and the like in the front, side, rear, It is possible to identify an optimal frame corresponding to each of the same directions and to render the identified optimal frames to generate a first three-dimensional image.

In various embodiments, the three-dimensional image rendering apparatus 200 may display the position of the guider where each photographing apparatus is disposed, the height of the photographing apparatus disposed in the guider, etc. corresponding to the identification information of each of the plurality of photographing apparatuses 120 Information can be stored in advance as location information. In this case, the three-dimensional image rendering apparatus 200 can confirm the location information previously stored corresponding to the identification information when the identification information of the photographing apparatus that has photographed each moving image together with the plurality of moving images is received. The three-dimensional image rendering apparatus 200 can determine a best frame corresponding to each direction based on the identified position information, and render the determined optimal frames to generate a three-dimensional image.

The three-dimensional image rendering apparatus 200 selects a second set of frames corresponding to different times from a specific time among the frames constituting each of the moving images photographed by the plurality of photographing apparatuses 120, Frames can be used to generate a three-dimensional image. At this time, the generated three-dimensional image can be defined as a second three-dimensional image. The three-dimensional image rendering apparatus 200 may select a second set of frames similar to the process of selecting the first set of frames described above. For example, the three-dimensional image rendering apparatus 200 can select 30 frames corresponding to different times from a specific time among the frames of the received 30 moving pictures as the second set of frames. When selecting the second set of frames corresponding to the different time, the three-dimensional image rendering apparatus 200 can determine whether the frame corresponding to the different time of the moving picture is the optimal frame that can be used for rendering the three-dimensional image.

If there is a frame that is not determined as an optimal frame among frames corresponding to different times, the three-dimensional image rendering apparatus 200 can similarly perform the above-described optimum frame acquisition process to acquire an optimal frame corresponding to another time have. For example, if the frame of different time is a frame with low clarity, the three-dimensional image rendering apparatus 200 performs image processing for increasing the edge strength of the frame at another time to improve the sharpness of the frame at another time , And thus the image-processed frame can be obtained as an optimum frame. In various embodiments, the three-dimensional image rendering apparatus 200 may perform image processing to increase a color value, a saturation value, or a luminance value for a frame at another time, and may acquire the image-processed frame as an optimal frame . In various embodiments, if the frame includes blurry portions, the three-dimensional image rendering apparatus 200 corrects the blurred portion in the frame of another time based on the previous frame or the next frame, and acquires the corrected frame as the optimum frame It is possible. In various embodiments, the three-dimensional image rendering apparatus 200 can acquire, as an optimal frame, a frame in which the sharpness of the previous frame or the next frame of the frame at another time is high and the pixel difference between the frame at another time and the frame is below the threshold. The manner of obtaining the optimum frame may be various and is not limited as described above. The three-dimensional image rendering apparatus 200 may generate a second three-dimensional image based on the optimal frame and position information acquired for each moving image corresponding to another time.

The three-dimensional image rendering apparatus 200 can generate a three-dimensional moving image by arranging the plurality of three-dimensional images thus generated in a time-series manner and using each of the plurality of three-dimensional images arranged in a time series. Specifically, the three-dimensional image rendering apparatus 200 recognizes the three-dimensional object from each of the generated three-dimensional images, and estimates the motion between the recognized three-dimensional objects using the interpolation method. For example, if the subject is a person, the first three-dimensional image shows the subject's arms folded, and the second three-dimensional image shows the subject's hands in the pants pocket, the three- The motion between the figure with the arms folded and the figure with the subject put the hand in the pants pocket can be predicted using the interpolation method. The three-dimensional image rendering apparatus 200 calculates the minutiae points of the three-dimensional object corresponding to the image of the subject's arms folded and the minutiae points of the three-dimensional object corresponding to the image of the subject's hand putting on the pants pocket, So that it is possible to estimate the values located between the minutiae calculated corresponding to the shape of the subject's arms and the calculated minutiae corresponding to the shape in which the subject puts his / her hand into the pants pocket. For example, the interpolation scheme may include, but is not limited to, linear interpolation, and various schemes for estimating values located between minutiae can be used. Using the estimated values, the three-dimensional image rendering apparatus 200 can generate a three-dimensional moving image representing a movement between a state in which the subject is wearing his arms and a state in which the subject puts his / her hands in the pants pocket.

In the illustrated embodiment, the three-dimensional image rendering apparatus 200 generates a first three-dimensional image using a first set of frames selected corresponding to a particular time, and generates a first three-dimensional image using a second set of frames, Although a process of generating a three-dimensional moving image by using first and second three-dimensional images generated after generating a three-dimensional image has been described, the present invention is not limited thereto, and two or more frames corresponding to various time periods can be used.

In various embodiments, the three-dimensional image rendering apparatus 200 may generate a three-dimensional moving image using frames constituting at least a part of moving images captured by each of the plurality of photographing apparatuses 120. For example, the three-dimensional image rendering apparatus 200 can receive thirty moving pictures captured by thirty photographing apparatuses disposed in the plurality of guiders 110. [ The three-dimensional image rendering apparatus 200 includes a first time, a second time, Selecting a frame corresponding to the n-th time to determine a first set of frames related to the first time, a second set of frames related to the second time, The n-th set of frames related to the n-th time can be selected. In this case, n is a natural number. In this case, the three-dimensional image rendering apparatus 200 may include a first set of frames associated with a first time, a second set of frames associated with a second time, A three-dimensional image related to the first time, a three-dimensional image related to the second time, and the like, using each of the n-th set frames related to the n-th time, Dimensional image related to the n-th time, and generates a three-dimensional image related to the first time, a three-dimensional image related to the second time, It is also possible to generate one three-dimensional moving image in which three-dimensional images related to the n-th time are arranged in a time-series manner.

In the illustrated embodiment, the three-dimensional image rendering apparatus 200 is implemented as a single device. However, the present invention is not limited to this, and may be implemented as a plurality of devices, Motion or three-dimensional moving image generation operations may be distributed.

Accordingly, in the present invention, a plurality of three-dimensional images are generated by using a moving image photographed through a plurality of photographing apparatuses, a three-dimensional moving image is generated by using a plurality of three-dimensional images, Therefore, it is possible to easily and easily generate the three-dimensional moving image while minimizing the cost for generating the three-dimensional moving image.

2 is a schematic diagram for explaining a three-dimensional image rendering apparatus according to an embodiment of the present invention.

2, the three-dimensional image rendering apparatus 200 includes a communication unit 210, a display unit 220, a storage unit 230, an input unit 240, and a processor 250, Module 255. < / RTI >

The communication unit 210 connects the three-dimensional image rendering apparatus 200 so that it can communicate with an external apparatus. The communication unit 210 may be connected to the plurality of photographing apparatuses 120 using wire / wireless communication or may be connected to a control module that controls the operation of the plurality of photographing apparatuses 120 to transmit and receive various information. In various embodiments, the communication unit 210 is connected to a hub device, and can transmit and receive various information from the plurality of photographing devices 120 or the control module through the hub device. Specifically, the communication unit 210 may transmit input data for requesting a plurality of moving images of a plurality of photographing apparatuses 120 to a plurality of photographing apparatuses 120 or a control module. The communication unit 210 can receive a plurality of moving images obtained by photographing a plurality of photographing apparatuses 120 from a plurality of photographing apparatuses 120 or a control module. In various embodiments, the communication unit 210 may further receive identification information or location information together with a plurality of moving images.

The display unit 220 may display various contents (e.g., text, images, video, icons, banners or symbols, etc.) to the user. For example, the display unit 220 may display a related user interface (UI) or a graphical user interface (GUI) related to moving images, three-dimensional images, or three-dimensional moving images. Specifically, the display unit 220 may display a moving picture, display frames of a moving picture, or display an interface for selecting and three-dimensional rendering of frames.

The storage unit 230 may store various data used for rendering a three-dimensional image using moving images. In particular, the storage unit 230 may store a plurality of moving images captured by the plurality of photographing apparatuses 120. In various embodiments, the storage unit 230 may store identification information of each of the plurality of photographing apparatuses 120, or may store the position of each of the photographing apparatuses corresponding to the identification information, the height of the photographing apparatuses And so on. In various embodiments, the storage unit 230 may store a three-dimensional image and / or a program for rendering a three-dimensional moving image.

The input unit 240 may generate input data for controlling the operation of the three-dimensional image rendering apparatus 200 by the user. For example, the input unit 240 may generate input data for requesting a plurality of moving images of a plurality of photographing apparatuses 120 by the plurality of photographing apparatuses 240. The input unit 240 may include a key pad, a domeswitch, a touch pad (static / static), a jog wheel, a jog switch, and the like.

Processor 250 is typically capable of controlling the overall operation of the three-dimensional image rendering apparatus 200. For example, processor 250 may perform processing of data or associated control and processing for displaying processed data. The processor 250 may include a graphics module 255 for parallel data processing. The graphic module 255 may be implemented in the control unit 250 or separately from the control unit 250.

The processor 250 receives the moving images photographed by the plurality of photographing apparatuses 120, selects a first set of frames corresponding to a specific time among the frames constituting each of the received moving images, Frames can be used to generate a three-dimensional image. Specifically, the processor 250 may receive a moving image from each of the plurality of photographing apparatuses 120, or may receive a plurality of moving images photographed by the plurality of photographing apparatuses 120 through the control module. The processor 250 may select a first set of frames corresponding to a first time among a plurality of frames constituting each of the received plurality of moving images. For example, if the number of received moving pictures is 30, 30 frames corresponding to the first time can be selected in each of the 30 moving pictures. The processor 250 can verify that each of the selected first set of frames is a suitable optimal frame for generating a three-dimensional image. Here, the optimum frame may be a frame with high clarity. The processor 250 may calculate the sharpness value of each of the first set of frames to determine whether the frame is an optimal frame, and if the calculated sharpness value is greater than or equal to the threshold sharpness value, the processor 250 may determine the frame as a frame with high sharpness. The processor 250 may utilize various schemes to obtain an optimal frame if a frame other than the optimal one of the first set of frames is present. Here, the frame other than the optimal frame may be a low-definition frame, a frame including a blurred portion, or a blacked out frame. Various methods for obtaining an optimal frame include a method of performing image processing for increasing the edge strength of a frame corresponding to a first time to enhance sharpness, a method of enhancing a color value, a saturation value, or a luminance value of a frame corresponding to the first time A method of performing image processing, a method of correcting a blurred portion using a previous frame or a next frame of a frame corresponding to the first time when a blurred portion is included in the frame corresponding to the first time, A method of selecting a frame in which the sharpness of the frame or the next frame is high and the pixel difference between the frame of the first time and the frame is equal to or less than the threshold value, and the like. In this manner, the processor 250 may obtain an optimal frame corresponding to the first time and generate a first three-dimensional image using the first set of frames including the acquired optimal frame.

The processor 250 may select a second set of frames corresponding to a different time from a specific time among frames constituting each of the received moving images and generate a three-dimensional image using the selected second set of frames. Specifically, the processor 250 may select a second set of frames corresponding to a second time among a plurality of frames constituting each of the received plurality of moving images. For example, if 30 received videos are 30 frames, 30 frames corresponding to the second time can be selected in each of the 30 videos. Processor 250 can verify that each of the selected second set of frames is a suitable optimal frame for generating a three-dimensional image. If each of the selected second set of frames is an optimal frame, the processor 250 may generate a second three-dimensional image using the second set of frames. If there is a frame that is not the optimal one among the selected second set of frames, the processor 250 may use various schemes to obtain an optimal frame as described above. Once the optimal frame is obtained, the processor 250 may generate a second three-dimensional image using the second set of frames including the acquired optimal frame.

The processor 250 may generate the three-dimensional moving image using the generated first and second three-dimensional images. Specifically, the processor 250 may time-align the first and second three-dimensional images and recognize the three-dimensional object from each of the first and second three-dimensional images arranged in a time series. The processor 250 may estimate the motion between the recognized three-dimensional objects using an interpolation method and generate a three-dimensional moving image representing the estimated motion. For example, the processor 250 may calculate a plurality of feature points of each of the three-dimensional objects, and estimate first values located between the calculated plurality of feature points. The processor 250 estimates second values located between each of the plurality of feature points and the estimated first values, estimates third values located between each of the plurality of feature points and the estimated second values, It is possible to estimate values located between the three-dimensional objects. Using these estimated values, the processor 250 may generate a three-dimensional moving image representing the motion between the first and second three-dimensional images.

3 is a schematic flowchart illustrating a method of rendering a three-dimensional image using moving images in a three-dimensional image rendering apparatus according to an embodiment of the present invention.

Referring to FIGS. 1 and 3, a three-dimensional image rendering apparatus 200 receives a plurality of moving images of a subject TS from a plurality of photographing apparatuses 120 in a different direction (S300). The three-dimensional image rendering apparatus 200 selects a first set of frames corresponding to a specific time among the frames constituting each of the plurality of moving images (S310). The three-dimensional image rendering apparatus 200 generates a three-dimensional image using the selected first set of frames (S320).

4 is a schematic flowchart for explaining a method for generating a three-dimensional moving image using a three-dimensional image in a three-dimensional image rendering apparatus according to an embodiment of the present invention.

Referring to FIGS. 1 and 4, a three-dimensional image rendering apparatus 200 receives a plurality of moving images of a subject TS in different directions from a plurality of photographing apparatuses 120 (S400). The three-dimensional image rendering apparatus 200 selects a first set of frames corresponding to a specific time among the frames constituting each of the plurality of moving images (S410). The three-dimensional image rendering apparatus 200 generates a first three-dimensional image using the selected first set of frames (S420). The three-dimensional image rendering apparatus 200 selects a second set of frames corresponding to different times from a specific time among the frames constituting each of the plurality of moving images (S430). The three-dimensional image rendering apparatus 200 generates a second three-dimensional image using the selected second set of frames (S440). The three-dimensional image rendering apparatus 200 generates a three-dimensional moving image using the generated first and second three-dimensional images (S450).

Hereinafter, a method for generating a plurality of three-dimensional images using a plurality of moving images photographed through a plurality of photographing apparatuses 120, and generating a three-dimensional moving image using the plurality of generated three- 5 to Fig. 7.

FIG. 5 is a diagram for explaining a method for acquiring an optimal frame corresponding to a specific time for generating a three-dimensional image in a three-dimensional image rendering apparatus according to an embodiment of the present invention.

Referring to FIGS. 1 and 5, the three-dimensional image rendering apparatus 200 may acquire the first moving image 500 and the second moving image 210 captured through the plurality of photographing apparatuses 120. The plurality of frames of the first moving image 500 and the second moving image 210 obtained may be arranged in a time-series manner. In order to generate a three-dimensional image corresponding to a specific time, the three-dimensional image rendering apparatus 200 selects a first frame 502 corresponding to a time t ₂ among a plurality of frames constituting the first moving image 500 , It is possible to select the second frame 512 corresponding to the same time t ₂ among the plurality of frames constituting the second moving picture 510. The 3D image rendering apparatus 200 determines whether the selected first frame 502 and second frame 512 are optimal frames and if the selected first frame 502 and second frame 512 are optimal frames The first three-dimensional image can be generated using the selected first frame 502 and the second frame 512. If the selected first frame 502 is not an optimal frame, the three-dimensional image rendering apparatus 200 may perform image processing to enhance the sharpness of the first frame 502. [ If at least a portion of the selected first frame 502 in various embodiments is blurred, the three-dimensional image rendering device 200 may use the third frame 504, which is the previous frame of the first frame 502, . In various embodiments, the three-dimensional image rendering apparatus 200 calculates the pixel difference between the first frame 502 and the third frame 504, and when the calculated pixel difference is below the threshold, the three- 3 frame 504 may be selected as the optimal frame used to generate the three-dimensional image. The optimum frame obtained corresponding to each of the plurality of photographing apparatuses 120 can be displayed through the interface screen as shown in FIG.

FIG. 6 is an exemplary view of an interface screen showing frames acquired for respective positions of a plurality of photographing apparatuses according to an embodiment of the present invention.

Referring to FIG. 6, the 3D image rendering apparatus 200 displays an interface screen 600 representing an optimal frame among the frames constituting a moving image photographed by each of the plurality of photographing apparatuses 120, . ≪ / RTI > The interface screen 600 includes a first area 610 including a graphic object indicating the position of each of the plurality of photographing apparatuses 120 about the subject TS and a graphic object indicating an optimal frame obtained at each position And a second region 620 included therein. Using the thus obtained optimum frames, the three-dimensional image rendering apparatus 200 can generate the first three-dimensional image corresponding to the time t ₂ . For example, the three-dimensional image rendering apparatus 200 generates a three-dimensional model using the optimum frames corresponding to the time t ₂ , generates a three-dimensional image of the object shape by mapping a texture to the generated three-dimensional model . Here, the texture may be generated using optimal frames along each direction. This will be described in more detail with reference to FIGS. 7A and 7B.

FIGS. 7A and 7B are exemplary views of an interface representing a three-dimensional model and a three-dimensional image generated through a three-dimensional image rendering apparatus according to an embodiment of the present invention.

Referring to FIGS. 1 and 7A, a three-dimensional image rendering apparatus 200 generates a first three-dimensional model using a first set of frames obtained corresponding to a first time (e.g., time t _{2 in} FIG. 5) (E.g., modeling) a second three-dimensional model using the second set of frames obtained corresponding to the second time (e.g., time t _{6 in} FIG. 5). The three-dimensional image rendering apparatus 200 may display the first three-dimensional model 710 and the interface screen 700 showing the second three-dimensional model 720 through the display unit 220. Various schemes can be used to generate a three-dimensional model using each of the first set of frames and the second set of frames. For example, the three-dimensional image rendering apparatus 200 can arrange frames of a direction of a subject and extract feature points related to a subject in each frame. The three-dimensional image rendering apparatus 200 can generate a three-dimensional model through three-dimensional modeling using the extracted minutiae points. The method of generating the three-dimensional model is not limited to the illustrated embodiment, and various methods for generating the three-dimensional model can be used.

Referring to FIGS. 1 and 7B, the 3D image rendering apparatus 200 maps a texture to each of the 3D graphic models 710 and 720 generated as shown in FIG. 7A to generate 3D images 730 and 740 Can be generated. The texture mapped to each of the three-dimensional graphic objects 710 and 720 may be a first set of frames or a second set of frames. For example, the three-dimensional image rendering apparatus 200 can perform texture mapping by mapping the vertices of each of the three-dimensional meshes forming the three-dimensional model and the pixels of each frame corresponding to each vertex. The three-dimensional image rendering apparatus 200 can display the first three-dimensional image 730 generated through the texture mapping and the interface screen 800 representing the second three-dimensional image 740 through the display unit 220 .

When the first three-dimensional image 730 and the second three-dimensional image 740 are generated as described above, the three-dimensional image rendering apparatus 200 recognizes the first three-dimensional object 732 related to the subject in the first three-dimensional image 730 , And extract a plurality of feature points for the recognized first three-dimensional object 732. The three-dimensional image rendering apparatus 200 can recognize a second three-dimensional object 742 about the subject in a second three-dimensional image 740 and extract a plurality of feature points for the second three-dimensional object 742 . The three-dimensional image rendering apparatus 200 uses an interpolation method to calculate a position between a plurality of minutiae extracted in relation to the first three-dimensional object 732 and a plurality of minutiae extracted in relation to the second three- And generate three-dimensional images between the first three-dimensional image 730 and the second three-dimensional image 740 using the estimated values. For example, if the first three-dimensional object 732 recognized in the first three-dimensional image 730 has the shape of a person with arms folded and the second three-dimensional object 742 recognized in the second three- In the case of having a shape of a person with a hand in his / her pocket, the generated three-dimensional images can correspond to a movement of a person with an arm up to move his / her arms into a pants pocket and put his / her hand into the pants pocket. The three-dimensional image rendering apparatus 200 can generate a three-dimensional moving image in which the generated three-dimensional images, the first three-dimensional image 730, and the second three-dimensional image 740 are arranged in a time-series manner. The generated three-dimensional moving image may be a moving image showing a movement of a person with an arm up to move his / her arm-handed person into a pants pocket and put his / her hand into his / her pants pocket.

As described above, according to the present invention, a plurality of three-dimensional images are generated using a moving image captured through a plurality of photographing apparatuses, a three-dimensional moving image is generated using a plurality of three-dimensional images, Therefore, it is possible to easily and easily generate the three-dimensional moving image while minimizing the cost for generating the three-dimensional moving image.

In the illustrated embodiment, a process of generating a three-dimensional image and a three-dimensional image using a booth in which a plurality of photographing devices are arranged has been described. However, the present invention is not limited thereto, and a three- It can also be created. For example, a moving picture for photographing a subject can be obtained through the mobile device. The obtained moving image can be photographed to include all the parts to be rendered as a three-dimensional image. For example, the mobile device can be photographed so as to include the whole head portion of the subject by rotating 360 degrees around the head of the subject. The mobile device may transmit the photographed moving image to the three-dimensional image rendering apparatus 200 for rendering a three-dimensional image. The mobile device may include sensors associated with movement of an acceleration sensor, a proximity sensor, an angle sensor, and the like. The mobile device can transmit the moving image to the three-dimensional image rendering apparatus 200 together with the moving image sensor value associated with the moving time of the moving image. In various embodiments, the subject is located at the center, and the photographing equipment for photographing the subject is seated on a support portion having a certain height, and the support portion can be connected to the rotation drive portion. The rotary drive unit is configured to rotate 360 degrees with respect to the center subject, so that the photographing equipment can be configured to photograph all portions of the center subject. The photographed moving image can be transmitted to the three-dimensional image rendering apparatus 200. At this time, position information such as height of the photographing apparatus may also be transmitted together.

The three-dimensional image rendering apparatus 200 can receive a moving image including a plurality of frames from a mobile device or a photographing apparatus, and can select a frame to be used for acquiring a three-dimensional image among a plurality of frames. The criterion for selection of the frame is not limited, but, for example, a blacked out frame or a blurred frame due to camera shake can be excluded. The method of selecting the optimum frame to be used for the three-dimensional image rendering may be a method based on the three-dimensional image obtained before the rendering. Accordingly, the three-dimensional image rendering apparatus 200 can select a frame that is a two-dimensional image suitable for a rendering method. An optimized rendering algorithm can be applied to extract images that can increase the rendering success rate. RGB, XY coordinate values, and the like can be extracted from the time within the effective range, and spatial data or color data may be guessed or predicted based on previously rendered data. For example, the three-dimensional image rendering apparatus 200 can select a frame by guessing the essential element values in rendering and the image at a specific position. For example, frames at the front, side, back, 45 degrees, 60 degrees can be selected. The three-dimensional image rendering apparatus 200 may render a three-dimensional image based on the selected frame. The rendered three-dimensional image can be used in a game or an entertainment device, or can be used as a file for three-dimensional printing.

In various embodiments, motion capture may be implemented utilizing multiple motion picture imaging devices. The three-dimensional image rendering apparatus 200 can successively acquire a three-dimensional file by extracting and rendering a frame unit for each motion when photographing a dynamic object through a plurality of fixedly positioned moving image devices. It can acquire high productivity compared to existing 3D file acquisition, rigging and motion capture.

In order to implement this, the three-dimensional image rendering apparatus 200 can group and render moving unit-specific frames that can be defined in a simultaneous range. The three-dimensional image rendering apparatus 200 may separately provide a standard for simultaneous photographing based on a time value. In order to set a standard for simultaneous photographing, the three-dimensional image rendering apparatus 200 may classify subtle differences on a frame and determine the same or different frames. For example, the three-dimensional image rendering apparatus 200 may compare the time of the previous frame with the time of the current frame, and consider the frames obtained within a predetermined time period as seconds as simultaneous shooting. The three-dimensional image rendering apparatus 200 determines that the moving image is moved to a coordinate value of each frame, for example, more than a few pixels to the right (for example, left 5 px) in order to distinguish the previous frame from the current frame, The current frame can be determined as a different frame. Alternatively, the three-dimensional image rendering apparatus 200 may determine whether the previous frame matches the current frame or not, based on the difference in the value of the file size per frame or the variation of the RGB value at a specific point in the frame. Based on these pictures, the three-dimensional image rendering apparatus 200 can render a three-dimensional image to generate three-dimensional images substantially close to motion capture.

In various embodiments, a plurality of photographing devices for photographing a subject can be configured to photograph a moving image. The number of the plurality of photographing apparatuses is not limited. The subject can be disposed at the center of the photographing apparatuses. The subject may be disposed on the rotation drive portion. The rotation driving unit can rotate the subject so that all portions of the subject can be photographed as a moving picture. The rotation driving unit may include a supporting part for disposing the subject and a motor for rotation. The shape of the rotation drive portion is not limited. Each of the photographing apparatuses can transmit the photographed moving image to the three-dimensional image rendering apparatus 200. The photographing apparatuses for photographing a subject are seated on a support portion having a predetermined height, and the support portion can be fixed. Position information such as the height of the photographing apparatus may be transmitted together with the moving image.

The three-dimensional image rendering apparatus 200 may receive a moving image including a plurality of frames from each of the plurality of photographing apparatuses, and may select a frame to be used for acquiring a three-dimensional image from the plurality of frames. The three-dimensional image rendering apparatus 200 may be a method based on a three-dimensional image obtained before rendering to select an optimal frame to be used for rendering a three-dimensional image. Accordingly, the three-dimensional image rendering apparatus 200 can select a frame that is a two-dimensional image suitable for a rendering method. The three-dimensional image rendering apparatus 200 can extract an image that can enhance the rendering success rate by applying an optimized rendering algorithm. The three-dimensional image rendering apparatus 200 may render a three-dimensional image based on the selected frame.

In various embodiments, motion capture may be implemented using multiple motion picture imaging devices. When acquiring a dynamic object through a plurality of fixedly positioned moving image devices, the 3D image rendering apparatus 200 extracts a motion-based frame unit and renders it to acquire a 3D image successively.

As described above, according to the present invention, an optimal image is extracted using a frame configured at the time of shooting a moving image and then rendered to generate a three-dimensional image, thereby facilitating acquisition of image information at the level of a multi-camera, There are advantages.

The apparatus and method according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer readable medium may include program instructions, data files, data structures, and the like, alone or in combination.

Program instructions to be recorded on a computer-readable medium may be those specially designed and constructed for the present invention or may be available to those skilled in the computer software arts. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Includes hardware devices specifically configured to store and execute program instructions such as magneto-optical media and ROM, RAM, flash memory, and the like. The above-mentioned medium may also be a transmission medium such as a light or metal wire, wave guide, etc., including a carrier wave for transmitting a signal designating a program command, a data structure and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

Although the embodiments of the present invention have been described in detail with reference to the accompanying drawings, it is to be understood that the present invention is not limited to those embodiments and various changes and modifications may be made without departing from the scope of the present invention. . Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, it should be understood that the above-described embodiments are illustrative in all aspects and not restrictive. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

100: Booth
110: a plurality of guiders
120: a plurality of photographing apparatuses
200: 3D image rendering device
1000: 3D image rendering system
TS: subject

Claims (14)

A communication unit;
A storage unit; And
And a processor operatively coupled to the communication unit and the storage unit,
The processor comprising:
Receiving a plurality of moving pictures photographed from a plurality of photographing apparatuses in different directions through the communication interface,
Selecting a first set of frames corresponding to a specific time among frames constituting each of the plurality of moving images,
And generate a three-dimensional image using the selected first set of frames. 2. The apparatus of claim 1,
Selecting a second set of frames corresponding to a time different from the specific time among frames constituting each of the plurality of moving images,
Generating the generated three-dimensional image and another three-dimensional image using the selected second set of frames,
And to generate a three-dimensional moving image using the generated three-dimensional image and the another three-dimensional image. The image capturing apparatus according to claim 1,
And further configured to photograph the subject at the same time, using the moving image. 2. The apparatus of claim 1,
And perform time synchronization for the plurality of moving images, and generate the three-dimensional image using the plurality of time-synchronized moving images. 2. The apparatus of claim 1,
Determining whether each of the selected first set of frames is an optimal frame used to generate the three-dimensional image,
And generate the three-dimensional image using the selected first set of frames if each of the first set of frames is determined to be the optimal frame. 6. The method of claim 5,
A frame with high definition,
Wherein the processor is further configured to calculate a sharpness value for each of the selected first set of frames and to determine that the frame has a high sharpness value if the calculated sharpness value is greater than or equal to a predetermined threshold sharpness value, A device that renders a dimensional image. 7. The apparatus of claim 6,
And perform image processing to enhance sharpness for the at least some frames if at least some of the frames in the first set are not determined to be the best frame. Comprising: receiving a plurality of moving pictures photographed from a plurality of photographing apparatuses in different directions;
Selecting a first set of frames corresponding to a specific time among frames constituting each of the plurality of moving images; And
And generating a three-dimensional image using the selected first set of frames. 3. A method of rendering a three-dimensional image using a moving image in a three-dimensional image rendering apparatus. 9. The method of claim 8,
Selecting a second set of frames corresponding to a time different from the specific time among frames constituting each of the plurality of moving images;
Generating the generated three-dimensional image and another three-dimensional image using the selected second set of frames; And
And generating a three-dimensional moving image by using the generated three-dimensional image and the another three-dimensional image, wherein the three-dimensional moving image is generated by using the moving image in the three-dimensional image rendering apparatus. 9. The image capturing apparatus according to claim 8,
A method for rendering a three-dimensional image using a moving image in a three-dimensional image rendering apparatus for simultaneously photographing the subject. 9. The method of claim 8,
The method of claim 1, further comprising: performing time synchronization on the plurality of moving images; and rendering a three-dimensional image using the moving image in the three-dimensional image rendering apparatus. 9. The method of claim 8, wherein generating the three-
Determining whether each of the selected first set of frames is an optimal frame used to generate the three-dimensional image; And
And generating the three-dimensional image using the selected first set of frames if each of the frames of the first set is determined to be the optimal frame. To render. 13. The method of claim 12,
A frame with high definition,
Wherein the step of generating the three-
Calculating a sharpness value for each of the selected first set of frames; And
And determining that the frame has a high sharpness if the calculated sharpness value is greater than or equal to a predetermined threshold sharpness value, in the three-dimensional image rendering apparatus. 14. The method of claim 13, wherein generating the three-
Further comprising performing image processing to enhance sharpness for the at least some frames if at least some of the frames in the first set are not determined to be the best frame, How to render a three-dimensional image.

Images