KR20120045269A - Method and apparatus for generating hologram based on 3d mesh modeling and evolution - Google Patents

Abstract

PURPOSE: A method and an apparatus for generating three-dimensional mesh modeling and evolution based hologram are provided to generate a three-dimensional model about a real object and to generate a hologram by using a CGH technology based on a three-dimensional model. CONSTITUTION: A three-dimensional geometry obtaining unit(110) receives image information from one or more cameras. The three-dimensional geometry obtaining unit generates a three-dimensional point cloud by obtaining color information and depth information. A three-dimensional mesh model generating unit(140) applies mesh modeling and mesh evolution to the three-dimensional point cloud. A hologram image generating unit(170) combines two-dimensional images of the generated specific point based on the three-dimensional mesh model.

Description

TECHNICAL AND APPARATUS FOR GENERATING HOLOGRAM BASED ON 3D MESH MODELING AND EVOLUTION

The following embodiments are related to a method and apparatus for generating a holographic image.

An apparatus and method for generating holograms based on 3D mesh modeling and evolution are disclosed.

Computer generated hologram (CGH) technology is a technology for generating holographic images from three-dimensional model information, and is used to represent graphics data as holographic images.

When a method of irradiating a laser beam on a subject is used to capture a holographic image, a problem may occur when the subject is a distant object or a person.

This approach also requires a very stable capture environment in order to obtain the phase difference between the reference laser beam and the object laser beam.

Also, since the capture equipment used in this manner is difficult to miniaturize, problems may arise due to the size of the capture equipment.

An embodiment of the present invention, an apparatus and method for generating a holographic image by generating a 3D model of the photorealistic object using a color and / or range camera, and using the CGH technology based on the generated 3D model Can provide.

According to one embodiment of the present invention, an apparatus and method for generating a holographic image reflecting reflected light by shifting a specular portion of a 3D model according to a change in a view may be provided.

According to one aspect of the present invention, a 3D geometric acquisition unit for generating a 3D point cloud by receiving image information from at least one camera and obtaining color information and depth information from the received image information, mesh modeling and meshing in the 3D point cloud There is provided an imaging apparatus including a 3D mesh model generator for generating 3D mesh information by applying evolution and a hologram image generator for generating a hologram pattern image by combining 2D images of a specific view generated based on the 3D mesh model. .

The 3D geometry acquisition unit may include a depth image acquisition unit obtaining a depth image based on image data provided from a camera set, a color image acquisition unit obtaining a color image based on image data provided from a camera set, and the depth image and the color. It may include a 3D point cloud generator for generating a 3D point cloud based on the image.

The depth image acquisition unit may acquire depth image data based on 2D images from different viewpoints photographed from each of a plurality of cameras constituting the camera set.

The 3D mesh model generation unit,

It may include a mesh modeling unit for generating a mesh model by applying a 3D mesh triangulation to the 3D point cloud and a mesh evolution unit for improving the precision of the mesh model by separating, generating or changing the mesh nodes in the mesh model.

The 3D mesh model generator may further include an illumination information extractor configured to receive the color information from the 3D geometry obtainer and extract illumination information from the color information.

The holographic image generator may detect a reflected light portion in the 3D mesh model based on the illumination information, and apply the reflected light to the 2D images at the specific viewpoint by shifting the reflected light portion in proportion to the movement of the specific viewpoint. can do.

The hologram image generator includes a specific viewpoint image generator for generating 2D images at a specific viewpoint using the 3D mesh model and a hologram pattern image generator for generating a hologram pattern image by combining 2D images at the specific viewpoint. can do.

The hologram pattern image generator may generate hologram pattern image generation information for each of the 2D images at the specific time point using one of a point-based computer generated hologram (CGH) method or a Fourier-based CGH method. The holographic pattern image may be generated by combining pattern image generation information.

The hologram pattern image generation unit may use the hologram pattern image generation information previously generated when the 2D image at the specific viewpoint is the same as the 2D image at the specific viewpoint previously generated.

The hologram pattern image generation unit compares the 2D image at the specific time point and the 2D image at the specific time point previously generated, and applies only to an area affected by a pixel having a change in the 2D image at the specific time point. The hologram pattern image generation information may be recalculated.

According to another aspect of the invention, the 3D geometric acquisition step of generating a 3D point cloud by receiving image information from at least one camera and obtaining color information and depth information from the received image information, mesh modeling and the 3D point cloud 3D mesh model generation step of generating 3D mesh information by applying mesh evolution and 2D images of a specific viewpoint based on the 3D mesh model, and a hologram image of generating hologram pattern images by combining the 2D images of the specific viewpoint There is provided a holographic image generating method comprising a generating step.

The 3D geometric acquiring step may include a depth image acquiring step of acquiring a depth image based on image data provided from a camera set, a color image acquiring step of acquiring a color image based on image data provided from a camera set, and the depth image and the The 3D point cloud generation step of generating a 3D point cloud based on the color image may be included.

The 3D mesh model generation step may include a mesh modeling step of generating a mesh model by applying 3D mesh triangulation to the 3D point cloud, and improving precision of the mesh model by separating, generating, or modifying mesh nodes in the mesh model. And mesh evolution step.

The generating of the 3D mesh model may further include an illumination information extraction step of extracting illumination information from the color information by receiving the color information from the 3D geometry obtainer.

The generating of the hologram image may detect a reflected light portion in the 3D mesh model based on the illumination information, and shift the reflected light portion to be proportional to the movement of the specific view to apply reflected light to the 2D images at the specific view. Applicable

The holographic image generation step may include generating a holographic pattern image by generating a holographic pattern image by combining a 2D image at a specific viewpoint and a specific viewpoint image generating step of generating 2D images at the specific viewpoint using the 3D mesh model. It may include a step.

The generating of the hologram pattern image may include generating hologram pattern image generation information for each of the 2D images at the specific time point using one of a point-based computer generated hologram (CGH) method and a Fourier-based CGH method. And generating the holographic pattern image by combining holographic pattern image generation information.

The generating of the hologram pattern image may use the hologram pattern image generation information previously generated when the 2D image at the specific view point is the same as the 2D image at the specific view point previously generated.

The generating of the hologram pattern image compares the 2D image at the specific point in time and the 2D image at the specific point in time previously generated, so that only the area affected by a pixel having a change in the 2D image at the specific point in time is affected. The hologram pattern image generation information may be recalculated.

An apparatus and method are provided for generating a 3D model of a photorealistic object using a color and / or range camera, and generating a holographic image by using CGH technology based on the generated 3D model.

An apparatus and method are provided for generating a holographic image that reflects reflected light by shifting a portion of the reflected light of the 3D model according to a change in a view.

1 is a structural diagram of an imaging device according to an embodiment of the present invention.
2 illustrates a configuration of a camera set and a 3D geometric acquisition unit for image acquisition according to an embodiment of the present invention.
3 illustrates mesh modeling and evolution according to an embodiment of the present invention.
4 illustrates hologram image generation according to an embodiment of the present invention.
5 is a flowchart illustrating a holographic image generating method according to an embodiment of the present invention.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to or limited by the embodiments. Like reference numerals in the drawings denote like elements.

1 is a structural diagram of an imaging device according to an embodiment of the present invention.

The image processing apparatus 100 may include a 3D geometry acquirer 110, a 3D mesh model generator 140, and a hologram image generator 170.

The 3D geometry acquirer 110 receives image information from one or more (multi) cameras.

The 3D geometry acquirer 110 obtains color information and depth information from the received image information, and generates a 3D point cloud using the obtained color information and depth information.

 The 3D geometry acquirer 110 may include a depth image acquirer 120, a color image acquirer 125, and a 3D point cloud generator 130.

The configuration and operation of the 3D geometry acquisition unit 110 are described below with reference to FIG. 2.

The 3D mesh model generator 140 generates 3D mesh information based on the 3D point cloud generated by the 3D geometry acquirer 110.

The 3D mesh model generator 140 may correct the generated 3D mesh information.

The 3D mesh information may include location information, color information, and lighting information.

The 3D mesh model generator 140 may generate 3D mesh information by applying mesh modeling and mesh evolution to the 3D point cloud.

The 3D mesh model generation unit 140 may include a mesh modeling unit 150, a mesh evolution unit 155, and an illumination information extraction unit 160.

The configuration and operation of the 3D mesh model generator 140 will be described below with reference to FIG. 3.

The hologram image generator 170 generates images of a specific view point based on the 3D mesh model generated by the 3D mesh model generator 140, and combines the generated images of the specific viewpoints to form a hologram pattern image. Create

The hologram image generator 170 may generate a hologram image based on the CGH.

The hologram image generator 170 may include a specific view point image generator 180 and a hologram pattern image generator 190.

The configuration and operation of the hologram image generator 170 will be described below with reference to FIG. 4.

2 illustrates a configuration of a camera set and a 3D geometric acquisition unit 110 for image acquisition according to an embodiment of the present invention.

Various types of camera sets (or sensors) may be used to provide information for obtaining depth images and color images.

The camera set may acquire a 3D real-time image in real time.

The set of cameras may be a fusion system 210, a stereo cameras 220, a camera array 230 and a micro lens array 240 of a color camera and a depth camera.

Fusion system 210 of color cameras and depth cameras includes one or more color cameras 212 and one or more depth cameras 214. Fusion systems 210 of color cameras and depth cameras are called heterogeneous camera sets because they include different kinds of cameras.

The stereo cameras 220 are composed of two cameras photographing a subject at different positions. That is, both cameras constituting the stereo cameras 220 respectively photograph 2D color images from different view points.

Camera array 230 is comprised of a plurality of cameras, arranged in rows and columns, for example. The plurality of cameras constituting the camera array 230 respectively photograph 2D color images from different viewpoints.

The micro lens array 240 is an array of lenses mounted in one camera. The lenses in the micro lens array 240 respectively photograph 2D color images at different viewpoints.

The depth image acquirer 120 obtains a depth image based on image data provided from a camera set.

For example, the depth image acquirer 120 may receive depth image data from the depth camera 214.

For example, the depth image acquirer 120 may acquire a depth image by calculating a depth value based on one or more image data provided from the stereo cameras 220, the camera array 230, or the micro lens array 240. have.

In addition, the depth image acquirer 120 may receive 2D images from different cameras from different sets of cameras, and may generate disparity information of the image based on the provided 2D images.

In general, the depth and variation of pixels representing an object are inversely proportional to each other. Therefore, the depth image acquirer 120 may obtain depth image data based on the variation information of the image.

The color image acquisition unit 125 acquires a color image based on image data provided from a camera set.

For example, the color image acquisition unit 125 may receive color image data from the color camera 212. In addition, the color image acquisition unit 125 may obtain a color image by calculating a color value based on one or more image data provided from the stereo cameras 220, the camera array 230, or the micro lens array 240. have.

As described above, the depth and the variation of pixels representing an object are inversely proportional to each other. Thus, in this embodiment and other embodiments of the present invention, it is obvious that the description of depth can be applied to variations. For example, the depth image acquirer 120 may also be referred to as a disparity image acquirer.

The 3D point cloud generator 130 generates a 3D point cloud based on the generated depth image and color image.

In general, the precision of the 3D geometry represented by the 3D point cloud is affected by the complexity of the captured image. Multi cameras (such as stereo cameras 220, camera array 230 and micro lens array 240) and Tof (Time-of) to obtain high precision dense 3D points Sensors (e.g., depth cameras) of the -Flight type can be used together.

In this case, calibration between the cameras and sensors used together can be performed prior to imaging. Calibration may be to adjust a camera or a sensor so that cameras and sensors photographing the same subject at the same location acquire the same data.

The image information generation method may solve a remote background capture problem, an outdoor capture problem, a laser beam irradiation problem on an object, and the like. In addition, the image information generating method may be performed using a relatively small size of the equipment, it may be performed using a more stable hologram capture equipment.

3 illustrates mesh modeling and evolution according to an embodiment of the present invention.

The first graph 310 represents the 3D point cloud 312 and the hole filling 314 target position generated by the 3D point cloud generator 130.

The mesh modeling unit 150 receives the 3D point cloud 312 from the 3D point cloud generation unit 130.

The mesh modeling unit 150 may perform hole filling.

The hole filling is to create a new 3D point 322 by referring to the information of the surrounding points when there is an empty space between the 3D point clouds 312.

The mesh modeling unit 150 may use a local optimization method or a global optimization method for hole filling.

The local optimization method is to generate a new 3D point based on the color / location continuity of the surrounding points.

The global optimization method is a volume prediction method that generates a new 3D point by predicting an invisible surface based on the currently generated volume.

The mesh modeling unit 150 may calculate a position 314 where a point is to be generated by hole filling according to the above-described method.

The mesh modeling unit 150 generates a new point 322 and generates a final point cloud 320.

The mesh modeling unit 150 generates the mesh model 330 by applying 3D mesh triangulation to the final point cloud 320.

The mesh modeling unit 150 generates a mesh model 330 composed of triangles by connecting points in the final point cloud 320 to an edge 332. The mesh modeling unit 150 may generate the mesh model 330 by using a delaunay triangulation method or a color edge based triangulation method.

The mesh model 330 is a model generated based on the 3D position information generated by the 3D geometry acquirer. However, points in the mesh model 330 contain errors for reasons such as camera characteristics or noise.

The mesh evolution unit 155 improves the precision of the mesh model 330 by separating, generating, or changing mesh nodes in the mesh model 330 with reference to 3D position and color information.

For example, the mesh evolution unit 155 may add a new point 342 and edges 344 connected to the new point 342.

The mesh acquisition unit 155 separates, generates, or changes mesh nodes to generate the final acquisition model 340.

The mesh evolution unit 155 performs a surface matching operation to add color information to the finally obtained model 340.

The surface matching operation produces a final acquisition model with nodes 352 and 354 added with color information.

When the imaging apparatus 100 processes a video, the mesh evolution unit 160 may perform a surface matching considering 3D geometric consistency and color consistency between models generated for each frame to improve image quality of the video. Can be.

The illumination information extractor 160 receives color information (eg, a color image generated by the color image acquirer 125) from the 3D geometric acquirer 110, and extracts illumination information from the color information.

The illumination information extractor 160 may detect a specular portion of the color image.

Among the components representing the color of the material, the diffuse component does not cause a difference according to the viewpoint change. Therefore, when an image of a specific time point is generated, the obtained color value may be used as it is in the portion to which the diffused light component is applied.

However, the reflected light component produces a sensitive difference in accordance with the viewpoint change. Therefore, when an image of a specific viewpoint is generated, the portion to which the reflected light component is applied cannot be processed only by image and depth based view interpolation.

Therefore, the information on the reflected light extracted by the illumination information extractor 160 may be used by the hologram image generator 170 to generate an image at a specific time point.

By processing the mesh evolution unit 155 and the illumination information extractor 160, the 3D mesh model generator 140 may provide 3D model information including the illumination information.

The above-described mesh modeling and mesh evolution processes the 3D geometric precision and color quality directing process based on the generated mesh model, thereby providing higher image quality and higher quality than any image generation method based on images of specific 2D viewpoints such as view interpolation. It is possible to provide a holographic image with precision.

The 3D mesh model generator 140 may consider 3D geometry and color consistency between 3D mesh models generated for each frame when processing the mesh modeling and mesh evolution operations for the frames in the video. In this case, interframe consistency may not be separately considered between 2D images at a specific time point to be generated later.

4 illustrates hologram image generation according to an embodiment of the present invention.

The specific view image generator 180 generates 2D images 420 at a specific view using the 3D mesh model 410 provided from the 3D mesh model generator 140.

Projection in a specific viewpoint 2D image 420 by using a three-dimensional surface model surface ( x , y , z ) in the 3D model 410 to generate the 2D image 420 at the specific viewpoint. The (projection) point ( x _p , y _p ) must be calculated.

The projection point ( x _p , y _p ) can be calculated by one of the following 1) to 3).

1) Orthographic projection: A method of using a parallelepiped observation space. If positive projection is used, the distance between the viewpoint and the object does not affect the projected shape of the object.

In orthographic projection, the projection point can be calculated by Equation 1 below.

및

를 갖는 법선(normal vector)으로 기울어졌다(slanted with).Video plane is angle

And

Slanted with a normal vector with.

2) Perspective projection: A method of connecting objects and viewpoints in space and expressing them on the projection surface. When perspective projection is used, the size of the projected object depends on the distance between the viewpoint and the object.

In perspective projection, the projection point can be calculated by Equation 2 below.

Where ( x ₀ , y ₀ ) is the position of the viewpoint (ie, the camera) and f is the focal length.

3) Angular projection: A projection method using each orthogonal plane.

In each orthogonal projection, the projection point can be calculated by the following equation (3).

When 2D images are generated according to the projection method, the specific view image generator 180 may generate color information corresponding to a calculated geometry by reflecting reflected light material information that changes according to the projection view point.

That is, the specific view image generator 180 may detect the reflected light portion in the 3D mesh model based on the illumination information generated by the illumination information extractor 160.

The specific view image generator 180 may determine a shift distance (ie, the number of pixels to be shifted) from the surface based on the binocular disparity.

The particular view image generator 180 may render a 3D object by applying a pixel shift to the reflected light component. This rendering produces a reflected light image in a particular view.

Therefore, the specific view image generator 180 may apply the reflected light to the 2D image at the specific view by shifting the reflected light portion in proportion to the movement of the view. Accordingly, the specific view image generator 180 may generate a natural image to which the reflected light is applied, and may generate a specific view image of all the diffused light and the reflected light material.

The hologram pattern image generator 190 generates the hologram pattern image 440 by combining the generated 2D images 420 at a specific time point.

The hologram pattern image generator 190 may generate the hologram pattern image 440 by applying various CGH techniques. For example, a point-based CGH scheme or a Fourier-based CGH scheme may be used to generate the hologram pattern image 440.

The hologram pattern image generator 190 may generate the information 430 for generating the hologram pattern image 440 by applying the CGH technology to each of the 2D images 420 at a specific time point. The hologram pattern image 440 may be generated by combining the 430.

In the following, a Fourier type CGH image generation method is described.

Equation 4 below is a formula for calculating the object beam of the hologram capturing process using the Fresnel-Kirchhoff formulation according to the principle of Fresnel diffraction.

및

는 각각 수평 및 수직 방향의 각을 나타낸다. 즉,

는 뷰 각(angle)을 나타낸다.

And

Denotes angles in the horizontal and vertical directions, respectively. In other words,

Represents the view angle.

는

에 대응하는 투영된 이미지이다. 이 때,

는 투영의 좌표 시스템(coordinate system)이다.

Is

Is a projected image corresponding to. At this time,

Is the coordinate system of the projection.

b is a real-valued constant.

s ( m , n ) is the mth and nth perspectives of the object.

The object beam can be calculated by calculating s ( m , n ) for each m and n .

The hologram pattern image generator 190 may generate a fringe pattern of the hologram using the virtual reference beam used in Equation 4 and the object beam calculated by Equation 4. .

When the imaging apparatus 100 plays a video, the hologram image generating unit 170 (or the hologram pattern image generating unit 190) may have a current frame (that is, the 2D image 420) before the previous frame (that is, the previous frame). Only when it is changed compared to the processed 2D image 420, an operation (eg, calculation of hologram pattern image generation information) for generating the hologram pattern image 440 may be performed.

For example, the holographic image generator 170 (or a holographic pattern image generation unit 190) is an object of the p (t- 1) and p (t) only if there is movement (motion) between a particular moment t If it is possible to calculate the beam, there is no movement between p (t- 1) and p (t) may be used as the object beam from the previous calculation time t- 1.

When the image device 100 plays a video, the hologram eternal generation unit 170 (or the hologram pattern image generation unit 190) may display the current frame (ie, the 2D image 420) and the previous frame (that is, the previous frame). The processed 2D image 420) may be compared to extract a pixel in which a change has occurred compared to a previous frame of the current frame.

The hologram eternal generation unit 170 (or the hologram pattern image generation unit 190) generates hologram pattern image information only in an area affected by the pixel in which the change occurs, and the influence of the pixel in which the change occurs is not affected. For regions that are not used, previously generated holographic pattern image generation information may be used.

을 추출할 수 있고, 하기의 수학식 5를 사용하여 현재 시각 t의 S _t (p _mn )을 계산할 수 있다.For example, in the above-described Equation 4, in calculating s ( m , n ), the hologram image generator 170 (or the hologram pattern image generator 190) is a modified portion of each pixel in p _mn . Pixels within

May be extracted and S _t ( p _mn ) of the current time t may be calculated using Equation 5 below.

5 is a flowchart illustrating a holographic image generating method according to an embodiment of the present invention.

The hologram image generating method includes a 3D geometric acquisition step (S520, S525 and S530), a 3D mesh model generation step (S550, S555 and S560) and a hologram image generation step (S580 and S590).

First, the 3D geometry acquisition steps S520, S525, and S530 will be described.

The 3D geometric acquisition steps S520, S525, and S530 generate a 3D point cloud by receiving image information from one or more cameras and obtaining color information and depth information from the received image information.

In a depth image acquisition step (S520), a depth image is obtained based on image data provided from a camera set.

In a color image acquisition step (S525), a color image is obtained based on image data provided from a camera set.

In the 3D point cloud generation step S530, a 3D point cloud is generated based on the depth image acquired in the depth image acquisition step S520 and the color image obtained in the color image acquisition step S525.

Next, 3D mesh model generation steps (S550, S555, and S560) will be described.

In the 3D mesh model generation steps (S550, S555, and S560), 3D mesh information is generated by applying mesh modeling and mesh evolution to the 3D point cloud generated in the 3D geometry acquisition steps (S520, S525, and S530).

In the mesh modeling step S550, a mesh model is generated by applying 3D mesh triangulation to the 3D point cloud.

In mesh evolution step S555, the precision of the mesh model is improved by separating, generating, or changing mesh nodes in the mesh model.

In the illumination information extraction step S560, illumination information is extracted from the color information provided from the 3D geometry acquisition steps S520, S525, and S530.

Next, hologram image generation steps S580 and S590 will be described.

In the hologram image generation steps S580 and S590, 2D images of a specific view are generated based on the 3D mesh model generated in the 3D mesh model generation steps S550, S555 and S560, and the 2D images of the specific view generated are combined. As a result, a holographic pattern image is generated.

In the hologram image generation steps S580 and S590, the reflected light portion in the 3D mesh model may be detected based on the illumination information extracted in the illumination information extraction step S560, and the detected reflected light portion may be proportional to the movement of a specific viewpoint. By shifting, the reflected light may be applied to the 2D images at a specific viewpoint.

In a specific viewpoint image generation step S580, 2D images at a specific viewpoint are generated using the 3D mesh model.

In the hologram pattern image generation step (S590), the hologram pattern image is generated by combining the generated 2D images at a specific time point.

The hologram pattern image generation step (S590) may include generating hologram pattern image generation information for each of the 2D images at a specific time point using one of a point-based CGH scheme or a Fourier-based CGH scheme. The method may include generating the holographic pattern image by combining the generated holographic pattern image generation information.

In the hologram pattern image generation step (S590), when the 2D image at the specific view point is the same as the 2D image at the same specific view point previously generated, previously generated hologram pattern image generation information may be used.

In the hologram pattern image generation step (S590), the 2D image at a specific time point and the 2D image at the same specific time point previously generated may be compared. In addition, the hologram pattern image generation information may be recalculated only in an area affected by a pixel having a change in the 2D image at a specific time point.

Technical contents according to an embodiment of the present invention described above with reference to FIGS. 1 to 4 may be applied to the present embodiment as it is. Therefore, more detailed description will be omitted below.

Method according to an embodiment of the present invention is implemented in the form of program instructions that can be executed by various computer means may be recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

100: video device
110: 3D Geometry
140: 3D mesh model generation unit
170: hologram image generating unit

Claims (20)

A 3D geometry obtainer for generating 3D point clouds by receiving image information from at least one camera and obtaining color information and depth information from the received image information;
A 3D mesh model generator for generating 3D mesh information by applying mesh modeling and mesh evolution to the 3D point cloud; And
Hologram image generation unit for generating a holographic pattern image by combining 2D images of a specific view generated based on the 3D mesh model
Imaging device comprising a. The method of claim 1,
The 3D geometric acquisition unit,
A depth image obtaining unit obtaining a depth image based on image data provided from a camera set;
A color image obtaining unit obtaining a color image based on image data provided from a camera set; And
3D point cloud generating unit for generating a 3D point cloud based on the depth image and the color image
Including, the imaging device. The method of claim 2,
The depth image obtaining unit obtains depth image data based on 2D images from different viewpoints photographed from each of a plurality of cameras constituting the camera set. The method of claim 1,
The 3D mesh model generation unit,
A mesh modeling unit generating a mesh model by applying 3D mesh triangulation to the 3D point cloud; And
Mesh evolution unit that improves the precision of the mesh model by separating, generating or changing mesh nodes in the mesh model
Including, the imaging device. The method of claim 4, wherein
The 3D mesh model generation unit,
An illumination information extracting unit which receives the color information from the 3D geometry obtaining unit and extracts lighting information from the color information.
Further comprising, the imaging device. The method of claim 5,
The hologram image generator detects the reflected light portion in the 3D mesh model based on the illumination information, and applies the reflected light to the 2D images at the specific viewpoint by shifting the reflected light portion in proportion to the movement of the specific viewpoint. Imaging device. The method of claim 1,
The hologram image generator,
A specific view image generation unit generating 2D images at a specific view using the 3D mesh model; And
Hologram pattern image generation unit for generating a hologram pattern image by combining the 2D images at the specific time point
Including, the imaging device. The method of claim 7, wherein
The hologram pattern image generating unit generates hologram pattern image generation information for each of the 2D images at the specific time point using one of a point-based computer generated hologram (CGH) method or a Fourier-based CGH method, and the hologram pattern image And generating the holographic pattern image by combining generation information. The method of claim 8,
And the hologram pattern image generation unit uses the hologram pattern image generation information previously generated when the 2D image at the specific viewpoint is the same as the 2D image at the specific viewpoint previously generated. The method of claim 8,
The hologram pattern image generation unit compares the 2D image at the specific time point and the 2D image at the specific time point previously generated, and applies only to an area affected by a pixel having a change in the 2D image at the specific time point. An image device for recalculating holographic pattern image generation information. 3D geometry acquisition step of generating a 3D point cloud by receiving image information from at least one camera and obtaining color information and depth information from the received image information;
A 3D mesh model generation step of generating 3D mesh information by applying mesh modeling and mesh evolution to the 3D point cloud; And
Generating hologram images based on the 3D mesh model to generate 2D images of a specific viewpoint and generating a holographic pattern image by combining the 2D images of the specific viewpoint.
Including, holographic image generation method. The method of claim 11,
The 3D geometry acquisition step,
A depth image obtaining step of obtaining a depth image based on image data provided from a camera set;
Obtaining a color image based on the image data provided from the camera set; And
3D point cloud generation step of generating a 3D point cloud based on the depth image and the color image
Including, holographic image generation method. The method of claim 11,
The 3D mesh model generation step,
A mesh modeling step of generating a mesh model by applying 3D mesh triangulation to the 3D point cloud; And
Mesh evolution step of improving the precision of the mesh model by separating, generating or modifying mesh nodes in the mesh model
Including, holographic image generation method. The method of claim 13,
The 3D mesh model generation step,
An illumination information extraction step of extracting illumination information from the color information by receiving the color information from the 3D geometric acquisition unit;
Further comprising a, holographic image generation method. The method of claim 14,
The generating of the hologram image detects the reflected light portion in the 3D mesh model based on the illumination information, and applies the reflected light to the 2D images at the specific viewpoint by shifting the reflected light portion in proportion to the movement of the specific viewpoint. , Holographic image generation method. The method of claim 11,
The hologram image generating step,
A specific viewpoint image generation step of generating 2D images at the specific viewpoint using the 3D mesh model; And
Hologram pattern image generation step of generating a hologram pattern image by combining the 2D images at the specific time point
Including, holographic image generation method. The method of claim 16,
The hologram pattern image generation step,
Generating holographic pattern image generation information for each of the 2D images at the specific viewpoint using one of a point-based computer generated hologram (CGH) method and a Fourier-based CGH method; And
Generating the hologram pattern image by combining the hologram pattern image generation information
Including, holographic image generation method. The method of claim 17,
The generating of the hologram pattern image uses the hologram pattern image generation information previously generated when the 2D image at the specific view point is the same as the 2D image at the specific view point previously generated. The method of claim 17,
The generating of the hologram pattern image compares the 2D image at the specific point in time and the 2D image at the specific point in time previously generated, so that only the area affected by a pixel having a change in the 2D image at the specific point in time is affected. And recalculating the hologram pattern image generation information. A computer-readable recording medium containing a program for performing the method for generating the hologram image of any one of claims 11 to 19.

Images