KR20110058476A - Method and apparatus for detecting depth of object - Google Patents

Abstract

PURPOSE: A depth detecting method and apparatus are provided to extract the depth of an object without prepared reference image from the depth included in a three-dimensional image. CONSTITUTION: A depth detecting apparatus comprises a first pickup part, a restoring part(140), a second pickup part, a relation degree calculation part(160), a maximum relation degree detection part(170), and a depth detection part(180). The first pickup part photographs an object and creates a reference image. The storing part restores the reference image to create plane images according to depth. The second pickup part picks up the plane images to create an object image. The relation degree calculation part calculates the relation degree between unit blocks of the reference image and the object image. The maximum relation degree detecting part detects the maximum relation degree among relation degrees corresponding to the unit blocks of the reference image. The depth detection part calculates the number of detection of the maximum relation degree according to depth and determines the depth, where the number of detection of the maximum relation degree is the maximum, as the depth of the object.

Description

Method and apparatus for detecting depth of object

The present invention relates to 3D imaging technology, and more particularly, to a depth detection method and apparatus for an object included in a 3D image.

Recently, a lot of researches have been conducted on the research on the 3D object recognition and the system implementation. In particular, integrated image technology capable of recording and reconstructing 3D images for 3D object recognition has been introduced.

Integrated imaging techniques include pickup and retrieval processes. The pickup process is a step of storing in the form of an elemental image array (EIA) through a lens array for a three-dimensional object.

The reconstruction process is a step of displaying the element image array and restoring the 3D image to its original position by passing the lens array again.

The 3D image that has been reconstructed may be represented as a planar image for each depth. In this case, the conventional object recognition method recognizes an object indicated by the reference image included in the 3D image by using a correlation between the planar images and the reference image representing the specific object.

Such an object recognition method has a limitation of storing a reference image in advance.

An object of the present invention is to pick up a planar reconstructed image reconstructed from an integrated image by using a computer pickup method, thereby providing a depth of an object without a previously stored reference image.

Technical problems other than the present invention will be easily understood through the following description.

According to an aspect of the present invention, a first pickup unit for photographing an object to generate a reference image; a restoration unit for generating a planar reconstruction image for each depth by reconstructing the reference image; A second pickup unit which picks up the planar reconstructed image and generates a target image; A correlation calculator calculating a correlation between the reference image and the unit block of the target image; A maximum correlation detector configured to detect a maximum correlation that is a maximum value among the correlations corresponding to the unit block of the reference image; And a depth detector configured to calculate a maximum correlation detection number corresponding to each depth, and detect a depth having the maximum number of correlation detection times among the depths as the depth of the object.

According to another aspect of the present invention, a depth detecting apparatus detects a depth of an object included in a stereoscopic image, the method comprising: photographing an object to generate a reference image; reconstructing the reference image to generate a planar reconstruction image for each depth Generating a target image by picking up the planar reconstruction image; calculating a correlation between the reference image and a unit block of the target image; a maximum of the correlations corresponding to a unit block of the reference image Detecting a maximum correlation that is a value; And calculating a maximum correlation detection number corresponding to each depth, and detecting, as the depth of the object, a depth having the maximum correlation detection frequency among the depths as the depth of the object.

According to an embodiment of the present invention, by providing a depth of an object included in a 3D image without a previously stored reference image, a depth of an object for which a reference image is not prepared in advance may be extracted.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

In the following description of the present invention with reference to the accompanying drawings, the same components are denoted by the same reference numerals regardless of the reference numerals, and redundant explanations thereof will be omitted. In describing the present invention, when it is determined that the detailed description of the related well-known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

1 is a diagram illustrating a depth detection apparatus according to an embodiment of the present invention.

Referring to FIG. 1, the object recognition apparatus includes a lens unit 110, an optical pickup unit 120, a restoration unit 140, a computer pickup unit 150, a correlation calculator 160, and a maximum correlation detector 170. ) And the depth detector 180.

The lens unit 110 includes a plurality of lenses or pinhole arrays for capturing and displaying an integrated image.

The optical pickup unit 120 generates the image data by sensing the projected light passing through the lens unit 110. That is, the optical pickup unit 120 detects the light projected through the lens unit 110 and generates a reference image which is an integrated image including a plurality of elemental images. Hereinafter, the element image constituting the reference image is referred to as a reference element image. The optical pickup unit 120 may include a CCD or a CMOS.

The reconstruction unit 140 generates a plane reconstruction image (POI) using the computer-based reconstruction method for the reference image. Hereinafter, the planar reconstructed image from which the reference image is reconstructed will be referred to as a reference reconstructed image.

The computer pickup unit 150 generates an integrated image by simulating a process of generating each planar reconstructed image restored by the reconstructor 140 through a lens array or a pinhole array. Hereinafter, the integrated image generated through the simulation will be referred to as a target image. That is, the computer pickup unit 150 corresponds to each planar reconstructed image by using a computer pickup method of generating an integrated image through a simulation of virtually detecting a virtual computer graphic model through a lens array. A target image that is an integrated image is generated. In this case, the element image constituting the target image is referred to as a target element image. A process of generating a target image of the planar reconstructed image will be described in detail later with reference to FIG. 5. The computer pickup unit 150 transmits the picked-up target image to the correlation calculator 160.

The correlation calculator 160 receives a reference image from the optical pickup unit 120 and receives a target image from the computer pickup unit 150.

The correlation calculator 160 calculates a correlation between the reference image and the corresponding target image. In the present embodiment, the correlation is described as being normalized cross-correlation (NCC), but it is apparent to those skilled in the art that the correlation may be calculated by a correlation method other than NCC.

The correlation calculator 160 may calculate a correlation between the target image and the reference image through Equation 1 below.

[Equation 1]

In this case, S is a value of a specific pixel included in the target image, T is a value of a specific pixel included in the reference image, S and T are values of pixels located at the same coordinates, and NCC (S, T) is a target. Correlation between pixels of an image and a reference image, E (T) is the average of the values of pixels included in the reference image containing T, and E (S) is the value of the pixels included in the reference image containing S Average.

That is, the correlation calculator 160 calculates a correlation between pixels of the same coordinates included in the target image for each depth and the reference image by using Equation 1. The correlation calculator 160 transmits the calculated correlation to the position detector 170.

The maximum correlation detector 170 detects the maximum correlation for each element image. For example, the maximum correlation detector 170 groups the correlation between the pixels constituting the target element image and the reference element image. The maximum correlation detector 170 calculates a representative correlation of the corresponding group, which is the maximum value among the correlations of each group. That is, the maximum correlation detector 170 calculates a representative correlation degree corresponding to the pair of the target element image and the reference element image. The depth detector 170 groups the representative correlation diagrams having the same location as the representative correlation diagram and the corresponding reference element image. Next, the depth detector 170 selects the maximum correlation, which is the maximum value among the representative correlations of each group.

The depth detector 180 detects the depth of the object by analyzing the maximum correlation detected by the maximum correlation detector 170. For example, when the maximum correlation detector 170 detects the maximum correlation, the depth detector 180 counts the maximum correlation detection frequency of the depth corresponding to the maximum correlation. At this time, the maximum correlation detection frequency is the number of detection of the maximum correlation corresponding to each depth. The depth detector 170 detects the depth of the maximum depth as the depth of the object among the total depth.

2A illustrates a pickup process according to an embodiment of the present invention, and FIG. 2B illustrates a restoration process according to an embodiment of the present invention.

Referring to FIG. 2A, the light reflected from the 3D object is projected to the optical pickup unit 120 through the lens unit 110, and the optical pickup unit 120 projects the projected light through a sensor such as a CCD or a CMOS. Sensing and creating a digital image. An image generated by the optical pickup unit 120 is an integrated image including a plurality of element images, and when the image is reversely projected through the lens unit 110, a stereoscopic image may be displayed.

Referring to FIG. 2B, when the integrated image is projected onto the lens unit 110 through a display device such as a display panel, the image picked up through the optical pickup unit 120 is disposed on the front surface of the lens unit 110. Stereoscopic images may be displayed.

The integrated image reconstruction method may include an optical integrated image reconstruction method (OIIR) or a computer integrated image reconstruction method (CIIR: Computational Integral Imaging Reconstruction).

The restoration method illustrated in FIG. 2B is the above-described optical integrated image restoration method. The optically integrated image reconstruction method is used to display low resolution three-dimensional images due to insufficient overlap between element images caused by physical limitations of the optical device such as rotation and aberration, and deteriorated image quality of the displayed three-dimensional image. There is a problem. To compensate for this drawback, computer integrated image reconstruction, which reconstructs three-dimensional images to a computer through digital simulation of geometric optics, is used as the reconstruction method. Hereinafter, a computer integrated image restoration method will be described with reference to FIG. 3.

3 is a conceptual diagram illustrating a computer integrated image restoration method according to an embodiment of the present invention.

Referring to FIG. 3, the restoration unit 140 projects an integrated image onto a restoration plane through a pinhole arrangement. In this case, it is apparent that the integrated image may be projected onto the reconstruction plane by using the lens array instead of the pinhole array. As illustrated in FIG. 3, the reconstructor 140 overlaps a portion of each element image projected on the reconstruction plane separated by z from the pinhole array to generate a planar reconstruction image POI. At this time, the restoration unit 140 may generate a plurality of planar restoration images by changing the depth z. That is, the restoration unit 140 may generate a plurality of depth-specific planar restoration images.

The reconstructor 140 may generate a planar reconstructed image for each depth without using a pinhole array or a lens array through the digital simulation of the above-described process.

4 is a diagram illustrating a method of detecting a depth of an object by a depth detecting apparatus according to an embodiment of the present invention.

Referring to FIG. 4, the optical pickup unit 120 picks up an object whose distance is z from the lens unit 110 to generate a reference image 410.

The reconstructor 140 generates the reference reconstructed image 420 for each depth by using the computer integrated image reconstruction method (CIIR) described above with reference to FIG. 3.

Subsequently, the computer pickup unit 150 picks up each reference reconstructed image using a computer pickup method. That is, the computer pickup unit 150 designates a reference reconstructed image, which is a planar image, as an object, and generates a target image 430 by picking it up through a virtual simulation. A function performing process of the optical pickup unit 120, the restoration unit 140, and the computer pickup unit 150 will be described in detail later with reference to FIG. 5.

The correlation calculator 160 calculates a correlation between the reference image 410 and each target image 430.

The maximum correlation detector 170 detects a maximum correlation which is the maximum value among the correlations between the reference image 410 and each target image 430. For example, as illustrated in FIG. 4, the depth detector 170 may be included in a reference element image located at (4, 5) of the reference image 410 and a target element image located at (4, 5) of the target image 431. A representative correlation is detected which is the maximum value of the correlation corresponding to the pixel. The depth detector 170 detects representative correlations corresponding to reference element images positioned at (4, 5) of the reference image 410 and target element images positioned at (4, 5) of the remaining target image 431, respectively. do. Representative correlations are graphically illustrated as shown in 440 according to depth. Next, the depth detector 160 detects the maximum correlation that is the maximum value among the detected representative correlations. For example, according to 440, the representative correlation is greatest at a depth of 75 mm. Therefore, the depth detector 170 detects the maximum correlation corresponding to the depth of 75mm.

At this time, the depth detector 180 increases the number of detections of the maximum correlation corresponding to the depth of 75 mm by one. The depth detector 180 performs the above-described maximum correlation detection process for all element images, and counts the number of detections of the maximum correlation for each depth according to each maximum correlation detection. As shown in FIG. 450, the number of detections of the maximum correlation may be represented by a graph according to depth.

The depth detector 180 detects the depth where the maximum correlation detection count is greatest as the depth where the object is located. For example, since the maximum correlation detection frequency in the graph 450 is the largest at 75 mm, the depth detector 180 may detect the depth of the object at 75 mm.

5 is a diagram illustrating an optical pickup and computer pickup process according to an embodiment of the present invention.

Referring to FIG. 5, the optical pickup unit 120 generates a reference image by detecting light reflected from an object passing through the lens unit 110.

Subsequently, the reconstruction unit 140 generates a reference reconstruction image that is a planar reconstruction image corresponding to the depth of the reference image z _i (i = 1, ..., n, n is an arbitrary natural number).

The computer pickup unit 150 sets the reference reconstructed image from the reconstructor 140 as a flat object located at a depth z _i to perform the pickup process. In this case, since the reference reconstructed image is in the form of digital data, the reference reconstructed image may be picked up using a computer pick-up method. That is, the computer pickup unit 150 may generate a target image, which is one integrated image corresponding to one reference reconstructed image, by using the computer pickup method.

Although the depth detection apparatus described above with reference to FIGS. 1 to 5 has been described as calculating a representative correlation degree based on the element image, the target image and the reference image are divided into blocks having a specified size according to an implementation method, and for each block. It will be apparent to those skilled in the art that a depth detection process may be performed by calculating a representative correlation.

FIG. 6A is a graph illustrating a representative correlation chart calculated in units of depths of an element image according to an embodiment of the present disclosure, and FIG. 6B is a block having a size obtained by dividing an element image by 2 × 2 according to another embodiment of the present invention. This is a graph showing the representative correlations calculated in units of depth.

Compared to the method of calculating the representative correlation for each element image as shown in FIG. 6A, FIG. 6B generates a representative correlation for each block smaller than the element image, thereby increasing the number of representative correlations to more accurately determine the depth of the object. Can be detected.

7 is a flowchart illustrating a process of detecting a depth of a depth detection apparatus according to an embodiment of the present invention.

The depth detecting apparatus generates a reference image by using an optical pickup method (710).

The depth detecting apparatus restores the reference image to the planar reconstructed image by using the computer integrated image reconstructing method (720).

The depth detection apparatus picks up the reconstructed plane reconstructed image by using a computer pick-up method and generates a target image (730). For example, the depth detecting apparatus generates a plurality of target images by picking up each of the flattened restored images restored in operation 720 using a computer pickup method.

The depth detection apparatus calculates a correlation between the target image and the reference image (740). For example, the depth detecting apparatus calculates a correlation between each pixel of the target image and the same pixel among pixels of the reference image.

The depth detection apparatus groups the correlations by unit block (750). That is, the depth detection apparatus divides the pixels of the target image and the reference image by unit blocks, and groups the correlations corresponding to the pixels by unit blocks.

The depth detecting apparatus detects the maximum correlation for each unit block and counts the maximum correlation detection number corresponding to each depth (760). The depth detecting apparatus detects a maximum value among correlations grouped for each unit block and adds 1 to the maximum correlation detection frequency for a depth corresponding to the correlation.

The depth detecting apparatus detects a depth at which the object has the largest correlation detection number as the depth at which the object is located (770).

So far I looked at the center of the embodiment for the present invention. Many embodiments other than the above-described embodiments are within the claims of the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. The disclosed embodiments should, therefore, be considered in an illustrative rather than a restrictive sense. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

1 is a diagram illustrating an object recognition apparatus according to an embodiment of the present invention.

2A illustrates a pickup process in accordance with one embodiment of the present invention.

2B is a diagram illustrating a restoration process according to an embodiment of the present invention.

3 is a conceptual diagram illustrating a computer integrated image restoration method according to an embodiment of the present invention.

4 is a diagram illustrating a method of detecting a depth of an object by a depth detecting apparatus according to an embodiment of the present invention.

5 illustrates an optical pickup and computer pickup process in accordance with an embodiment of the present invention.

FIG. 6A is a graph illustrating a representative correlation chart calculated in units of element images according to depths according to an embodiment of the present invention. FIG.

FIG. 6B is a graph illustrating representative correlations calculated according to depths of blocks having a size obtained by dividing an element image into 2 × 2 units according to another embodiment of the present invention. FIG.

7 is a flowchart illustrating a process of detecting a depth of a depth detecting apparatus according to an embodiment of the present invention.

Claims (15)

A first pickup unit photographing an object to generate a reference image; A reconstruction unit reconstructing the reference image to generate a planar reconstruction image for each depth; A second pickup unit which picks up the planar reconstructed image and generates a target image; A correlation calculator calculating a correlation between the reference image and the unit block of the target image; A maximum correlation detector configured to detect a maximum correlation that is a maximum value among the correlations corresponding to the unit block of the reference image; And And a depth detector for calculating a maximum correlation detection number corresponding to each depth, and detecting, as the depth of the object, a depth having the maximum number of correlation detection times among the depths. The method of claim 1, And a lens unit including a lens array or a pinhole array. 3. The method of claim 2, And the first pickup unit photographs the object through the lens unit to generate the integrated image. The method of claim 1, And the second pickup unit generates the target image by using a computer pickup method for the planar restored image. The method of claim 1, And the second pickup unit generates one target image per planar reconstructed image. The method of claim 1, And the maximum correlation detection number is a number of times the maximum correlation corresponding to each depth is detected. The method of claim 1, And the correlation calculator calculating a correlation between the unit blocks at the same position in the reference image and the target image. The method of claim 7, wherein The correlation calculator calculates a correlation by comparing pixels included in the unit block corresponding to the reference image with pixels included in the unit block corresponding to the target image, and calculates a maximum correlation among the correlations corresponding to the pixel. And a value is calculated as a correlation between the unit blocks. In the method for detecting the depth of the object included in the stereoscopic image in the depth detection device, Photographing an object to generate a reference image; Restoring the reference image to generate a planar reconstruction image for each depth; Picking up the planar reconstructed image to generate a target image; Calculating a correlation between the reference image and the unit block of the target image; Detecting a maximum correlation which is the maximum value among the correlations corresponding to the unit block of the reference image; And Calculating a maximum correlation detection number corresponding to each depth, and detecting, as the depth of the object, a depth having the maximum number of detections of the maximum correlation among the depths. The method of claim 9, The generating of the reference image may include generating the integrated image by photographing the object through a lens array or a pinhole array. The method of claim 9, The generating of the target image may include generating the target image by using a computer pick-up method for the planar reconstructed image. The method of claim 9, The generating of the target image may include generating one target image per planar reconstruction image. The method of claim 9, And the maximum correlation detection number is a number of times the maximum correlation corresponding to each depth is detected. The method of claim 9, The calculating of the correlation between the unit blocks is a step of calculating the correlation between the unit blocks of the same position in the reference image and the target image. 15. The method of claim 14, The calculating of the correlation between the unit blocks may be performed by comparing the pixels included in the unit block corresponding to the reference image with the pixels included in the unit block corresponding to the target image, and calculating the correlation. And calculating a maximum value among corresponding correlations as a correlation between the unit blocks.

Images