WO2013165046A1

WO2013165046A1 - 3d warping method for suppressing formation of holes and image processing apparatus adopting same

Info

Publication number: WO2013165046A1
Application number: PCT/KR2012/003841
Authority: WO
Inventors: 송혁; 최병호; 김제우
Original assignee: 전자부품연구원
Priority date: 2012-05-04
Filing date: 2012-05-16
Publication date: 2013-11-07
Also published as: KR101373603B1; KR20130123891A

Abstract

Provided are a 3D warping method for suppressing the generation of holes and an image processing apparatus adopting the same. An image processing method according to an embodiment of the present invention divides an image into multiple regions and 3D warps the image by units of regions. Thus, holes may not be generated in 3D warping, and may suppress the generation of holes as much as possible. Thus, 3D-warped RGB images may have correct pixel values and 3D-warped Z-images may have correct depth information.

Description

[Specification】

[Name of invention]

3D warping method for suppressing hole generation and image processing device using the same

Technical Field

The present invention relates to a 3D-warping method and apparatus, and more particularly, to a 3D-warping method and apparatus for moving a viewpoint of an image to another viewpoint.

Background Art

3D-warping refers to image processing for moving a viewpoint of an image to another viewpoint. To match the view of the Z-camera that generates the Z-image containing the depth information and the RGB-camera that produces the RGB image containing the color information, the view of the Z-image is RGB- 3D-warping is typically used when moving to the point of view of the image.

In addition, 3D-warping is used to generate a multi-view image by moving the viewpoint of the RGB image generated by the RGB camera to another viewpoint.

In both the former and the latter, 3D-warped images may have undefined data, which may be filled with inaccurate data even if the data is filled by the hole removal technique. It causes dropping problems. [Detailed Description of the Invention]

[Technical problem]

The present invention has been made to solve the above problems, the present invention The present invention has been made to solve the above problems, and an object of the present invention is to produce a 3D-warping method and apparatus that can be suppressed as much as possible, or does not generate a hole in performing the 3D-warping In providing.

Technical Solution

According to one or more exemplary embodiments, an image processing method includes: dividing an image into a plurality of regions; And 3D-warping the image in units of the regions generated in the dividing step.

The dividing step may include extracting feature points from the image; And generating the plurality of regions by connecting the feature points extracted in the extraction step.

The extracting step may further include detecting edges in the image; And extracting nodes by the edges, wherein the generating may include connecting the nodes to generate the plurality of regions.

And, the plurality of regions may be spirits of the same shape.

In addition, the shape may be a triangle.

And the 3D-warping step comprises: 3D-warping the feature points; And 3D-warping the regions of the image into regions generated in the 3D-warped feature points, respectively.

In addition, the image may be a color image or a depth image. On the other hand, according to another embodiment of the present invention, an image processing apparatus includes a camera for generating an image; And an image processor for dividing an image generated by the camera into a plurality of regions and warping the image in units of regions.

Advantageous Effects

As described above, according to the present invention, in performing the 3D-warping, it is possible not to generate a hole or to suppress it as much as possible. Accordingly, in the case of the 3D-warped RGB image, it is possible to maintain accurate pixel values for (almost) all pixels, thereby maintaining the best image quality when generating a multiview image. In addition, in the case of 3D-warped Z-images, accurate depth information is retained for (almost) all pixels, thereby enabling accurate three-dimensional representation.

[Brief Description of Drawings]

1 is a block diagram of an image processing apparatus according to a preferred embodiment of the present invention, FIG. 2 is a flowchart provided in detail of a 3D-warping process of an RGB image by the image processing apparatus shown in FIG.

3 is a diagram provided for further explanation of edge detection;

4 is a view provided for further explanation of node extraction, and

5 is a diagram provided for further explanation of RGB-image segmentation using nodes. Best Mode for Implementation of the Invention

Hereinafter, with reference to the drawings will be described the present invention in more detail. 1 is a block diagram of an image processing apparatus according to a preferred embodiment of the present invention. The image processing apparatus according to the present exemplary embodiment performs 3 warping to generate a point image again using one RGB image and a Z image, and does not generate holes in the 3D warping process.

As shown in FIG. 1, an image processing apparatus that performs such a function includes an RGB camera 110, a Z camera 120, an image processor 130, and an image processor 140. ᅳ

The RGB camera 110 generates an RGB image, which is a color image, by photographing, and the Z camera 120 generates a Z image, which is a depth image for the RGB image.

The image processor 130 generates images of various views by using the RGB image generated by the RGB camera 110 and the Z image generated by the Z camera 120. The image processor 130 that performs the above functions includes an RGB image correction / alignment unit 131, an edge detection unit 132, a node extraction unit 133, an RGB image division unit 134, and an RGB- Image 3D-warping section 135, Z-image correction / alignment section 136, Z ᅳ image 3D-warping section 137, Z-image inpainting section 138 and Z-image upsampling section 139 ).

RGB—Image calibration / alignment unit 131 calibrates and aligns the RGB image with the baseline according to the internal and extrinsic parameters of the RGB camera 110. .

Edge detector 132 is RGB-corrected / aligned by RGB-image correction / aligner 131 Detect an edge on the phase. The node extractor 133 extracts nodes using the edges detected by the edge detector 132.

The RGB image segmentation unit 134 connects the nodes extracted by the node extraction unit 133 to generate triangular regions (triangular regions) to divide the RGB image into a plurality of triangular regions.

Meanwhile, the Z-image calibration / alignment unit 136 corrects the Z-image according to the internal and external factors of the Z-camera 120 and aligns it with the reference line. The Z-image tri-warping unit 137 3D-warps the corrected / aligned Z-image to the viewpoint of the RGB image in the Z-image correction / alignment unit 136.

The Z 'image inpainting unit 138 inpaints the 3D-warped Z-image in the Z-image 3D-warping unit 137 to remove holes present in the 3D-warped Z-image. The Z-image upsampling unit 139 upsamples the size of the inpainted Z-image all RGB-images. Upsampling takes into account that the resolution of the Z-image is lower than that of the RGB image.

The RGB-image 3D-warping section 135 uses a RGB-image output from the RGB-image correction / alignment section 131 and a Z-image output from the Z-image upsampling section 139 to provide various views. Generates RGB images.

To this end, the RGB image must be 3D-warped to various viewpoints, wherein the RGB image 3D-warping unit 135 is an RGB image divided by a plurality of triangular regions by the image segmentation unit 134. Image '(hereinafter, referred to as' divided image'). Specifically, RGB shooting The image 3D-warping unit 135 performs 3D-warping in units of triangular regions of the divided image.

The image processor 140 may display images of various views generated by the image processor 130 on a display or store the images on a storage medium, and may perform various applications such as object recognition using the images.

Hereinafter, the 3D-warping process of the RGB image by the image processing apparatus shown in FIG. 1 will be described in more detail with reference to FIG. 2.

As shown in FIG. 2, first, the edge detector 132 detects an edge in an RGB image that has been generated by the RGB camera 110 and then corrected / aligned by the RGB image correction / alignment unit 131. (S210).

As edge detection in step S210, an edge detection technique using a window (eg, Sobel technique), an edge detection technique (eg, Canny technique), etc. considering a connection relationship at a color boundary may be applied.

The left side of FIG. 3 shows the result of edge detection by Sobel technique for a specific RGB image, and the right side of FIG. 3 shows the edge detection result by Canny technique for a specific RGB image.

Thereafter, the node extractor 133 extracts nodes using the edges detected in step S210 (S220). The nodes extracted in step S220 are points where the edges change direction rapidly, points where the edges are discontinuous, and other points exhibiting unusual characteristics.

On the left side of FIG. 4, the node extraction result for the edge image shown on the left side of FIG. 4 illustrates a node extraction result for the edge image shown on the right side of FIG. 3.

Next, the RGB image segmentation unit 134 connects the nodes extracted by the node extraction unit 133 to generate triangular regions, thereby dividing the RGB image into a plurality of triangular regions (S230).

The triangular region generated in step S230 is generated in a direction in which an angle which is the smallest of the internal angles is maximized, and is not included in the interior. The RGB image is divided into a plurality of triangular regions by the RGB image divider 134.

Since these triangular regions were created using nodes extracted based on the edges, one triangular region is located inside the foreground or background. That is, the triangular area generated by step S230 is not located over the foreground and the background.

The left side of FIG. 5 shows a segmentation result of an RGB image using the nodes shown on the left side of FIG. 4, and the right side of FIG. 5 shows a segmentation result of an RGB image using the nodes shown on the right side of FIG. 4.

Afterwards, the RGB-image 3D-warping unit 135 3D-warps the nodes shown in the RGB image to another viewpoint (S240), and then triangular regions of the RGB-image by the 3D-warped nodes in step S240. 3D-warping each of the generated triangular regions (S250).

Since the 3D-warping in operation S250 is not 3D-warping between pixels and pixels, but 3D-warping between regions and regions, holes are not generated. That is, 3D-warping at step S250 is zero It can be said to be 3 warping of inverse units.

Up to now, the present invention has been described in detail with reference to a preferred embodiment of a technique that does not generate holes in warping the RGB image.

In the above embodiment, it is assumed that the RGB image is divided into triangular regions, but this is merely an example for convenience of explanation, and it is also possible to divide the image into regions of different shapes (for example, a rectangle and a pentagon). It is possible.

In this case, although the shapes of the divided regions are all the same, it is also possible to vary the shapes of the regions according to needs and specifications.

In addition, the above embodiment has been illustrated and described as segmenting the RGB image based on the nodes, which is also merely an example for convenience of description. It is also possible to segment the RGB image on the basis of other feature points other than nodes.

In addition, in the above embodiment, 3D-warping for the RGB image is assumed, but the technical idea of the present invention is also applicable to the Z-image. In FIG. 1, 3D-warping for the Z-image is assumed to be 3D-warping on a pixel-by-pixel basis, but it is possible to replace the Z-image by performing 3D-warping on a region-by-region basis by dividing the Z-image into a plurality of regions.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the specific embodiments described above, the technology to which the invention belongs without departing from the spirit of the invention claimed in the claims Various modifications can be made by those skilled in the art, as well as such modifications. It should not be understood individually from technical ideas or prospects.

Claims

[Range of request]

[Claim 1]

Dividing the image into a plurality of regions; And

3D—warping the image in units of the regions generated in the dividing step.

[Claim 2]

The method of claim 1,

The dividing step,

Extracting feature points from the image; And

And connecting the feature points extracted in the extracting step to generate the plurality of regions.

[Claim 3]

The method of claim 2,

The extraction step,

Detecting edges in the image; And

Extracting nodes by the edges; and

The generating step,

Connecting the nodes to generate the plurality of regions Image processing method.

[Claim 4]

The method of claim 2,

The plurality of areas,

Image processing method, characterized in that the spirits of the same shape.

[Claim 5]

The method of claim 4,

The shape is,

Image processing method characterized in that the triangle.

[Claim 6]

The method of claim 2,

The 3D-warping step,

3D-warping the feature points; And

D-warping each of the regions of the image into regions generated in 3D-warped feature points.

[Claim 7] The method of claim 1,

The video is

Color -image or depth—image processing method characterized in that the image.

[Claim 8]

A camera for generating an image; And

And an image processor for dividing an image generated by the camera into a plurality of regions, and 3D-warping the image in units of regions.