KR20110058437A - Method and apparatus for detecting object - Google Patents

Abstract

PURPOSE: A method and apparatus for detecting an object are provided to extract a depth about an object of which a pre-stored reference image is not prepared. CONSTITUTION: A pick up unit(120) creates a reference image. A restoration unit(140) restores the reference frame. A second pick-up part(150) picks up a plane restoration image. A correlation calculator(160) produces the correlation between a unit block of the object image and the reference image. A classification unit(180) creates the classification integration image by depth.

Description

Method and apparatus for detecting objects {Method and apparatus for detecting object}

The present invention relates to three-dimensional imaging technology, and more particularly, to a method and apparatus for recognizing an object included in a three-dimensional image.

Recently, a lot of researches have been conducted on the research on the 3D object recognition and the system implementation. In particular, integrated image technology capable of recording and reconstructing 3D images for 3D object recognition has been introduced.

Integrated imaging techniques include pickup and retrieval processes. The pickup process is a step of storing in the form of an elemental image array (EIA) through a lens array for a three-dimensional object.

The reconstruction process is a step of displaying the element image array and restoring the 3D image to its original position by passing the lens array again.

The 3D image that has been reconstructed may be represented as a planar image for each depth. In this case, the conventional object recognition method recognizes an object indicated by the reference image included in the 3D image by using a correlation between the planar images and the reference image representing the specific object.

This object recognition method has a constraint that a reference image must be stored in advance.

An object of the present invention is to pick up a planar reconstructed image reconstructed from an integrated image by using a computer pickup method, thereby providing a depth of an object without a previously stored reference image.

Another object of the present invention is to provide a planar reconstructed image with low noise by classifying unit blocks by depth.

Technical problems other than the present invention will be easily understood through the following description.

According to an aspect of the present invention, a first pickup unit for photographing an object to generate a reference image; a restoration unit for generating a planar reconstruction image for each depth by reconstructing the reference image; A second pickup unit which picks up the planar reconstructed image and generates a target image; A correlation calculator calculating a correlation between the reference image and the unit block of the target image; A depth detector for detecting a depth corresponding to a maximum correlation among the correlations corresponding to a unit block of the reference image as a depth corresponding to the unit block; And a classification unit classifying the unit blocks by depth and generating a classification integrated image for each depth including the classified unit blocks.

According to another aspect of the present invention, a method for recognizing an object by an object recognition apparatus, the method comprising: generating a reference image by photographing an object; Restoring the reference image to generate a planar reconstruction image for each depth; Picking up the planar reconstructed image to generate a target image; Calculating a correlation between the reference image and the unit block of the target image; Detecting a depth corresponding to a maximum correlation among the correlations corresponding to a unit block of the reference image as a depth corresponding to the unit block; And classifying the unit blocks according to depths, and generating a depth classification image including the classified unit blocks.

According to an embodiment of the present invention, by providing three-dimensional coordinates of an object included in the three-dimensional image without a reference image stored in advance, it is possible to extract the depth of the object for which the reference image is not prepared.

According to an embodiment of the present invention, as the noise of the planar reconstruction image is removed, the shape of the object may be clearly displayed.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

In the following description of the present invention with reference to the accompanying drawings, the same components are denoted by the same reference numerals regardless of the reference numerals, and redundant explanations thereof will be omitted. In the following description of the present invention, if it is determined that the detailed description of the related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

1 is a diagram illustrating an object recognition apparatus according to an embodiment of the present invention.

Referring to FIG. 1, the object recognition apparatus includes a lens unit 110, an optical pickup unit 120, a restoration unit 140, a computer pickup unit 150, a correlation calculation unit 160, a depth detection unit 170, and The classification unit 180 is included.

The lens unit 110 includes an array of a plurality of lenses or pinholes for capturing and displaying an integrated image.

The optical pickup unit 120 generates the image data by sensing the projected light passing through the lens unit 110. That is, the optical pickup unit 120 detects the light projected through the lens unit 110 and generates a reference image which is an integrated image including a plurality of elemental images. Hereinafter, the element image constituting the reference image is referred to as a reference element image. The optical pickup unit 120 may include a CCD or a CMOS.

The reconstruction unit 140 generates a plane reconstruction image (POI) using the computer-based reconstruction method for the reference image. Hereinafter, the planar reconstructed image from which the reference image is reconstructed will be referred to as a reference reconstructed image.

The computer pickup unit 150 generates an integrated image by simulating a process of generating each planar reconstructed image restored by the reconstructor 140 through a lens array or a pinhole array. Hereinafter, the integrated image generated through the simulation will be referred to as a target image. That is, the computer pickup unit 150 corresponds to each planar reconstructed image by using a computer pickup method of generating an integrated image through a simulation of virtually detecting a virtual computer graphic model through a lens array. A target image that is an integrated image is generated. In this case, the element image constituting the target image is referred to as a target element image. A process of generating a target image of the planar reconstructed image will be described in detail later with reference to FIG. 5. The computer pickup unit 150 transmits the picked-up target image to the correlation calculator 160.

The correlation calculator 160 receives a reference image from the optical pickup unit 120 and receives a target image from the computer pickup unit 150.

The correlation calculator 160 calculates a correlation between the reference image and the corresponding target image. In the present embodiment, the correlation is described as being normalized cross-correlation (NCC), but it is apparent to those skilled in the art that the correlation may be calculated by a correlation method other than NCC.

The correlation calculator 160 may calculate a correlation between the target image and the reference image through Equation 1 below.

[Equation 1]

In this case, S is a value of a specific pixel included in the target image, T is a value of a specific pixel included in the reference image, S and T are values of pixels located at the same coordinates, and NCC (S, T) is a target. Correlation between pixels of an image and a reference image, E (T) is the average of the values of pixels included in the reference image containing T, and E (S) is the value of the pixels included in the reference image containing S Average.

That is, the correlation calculator 160 calculates a correlation between pixels of the same coordinates included in the target image for each depth and the reference image by using Equation 1. The correlation calculator 160 transmits the calculated correlation to the position detector 170.

The depth detector 170 detects the maximum correlation for each reference element image, and detects the depth of the target element image corresponding to the maximum correlation as the depth of the reference element image. For example, the depth detector 170 groups the correlation between the pixels constituting the target element image and the reference element image. The depth detector 170 detects a maximum value among correlations of each group as a representative correlation of the corresponding group. That is, the depth detector 170 detects a representative correlation degree corresponding to the pair of the target element image and the reference element image. The depth detector 170 groups the representative correlation diagrams having the same location as the representative correlation diagram and the corresponding reference element image. Next, the depth detector 170 detects a maximum correlation, which is the maximum value among the representative correlations of each group. The depth detector 170 detects the depth of the target element image corresponding to each maximum correlation as the depth of the reference element image corresponding to the maximum correlation.

The classifier 180 classifies the reference element image based on the depth of each reference element image. That is, the classification unit 180 classifies and classifies reference element images having the same depth among all the reference element images. Subsequently, the classifying unit 180 arranges the classified element images at coordinates corresponding to each element image, and arranges empty element images (eg, element images having zero values of all pixels) at empty coordinates. (Hereinafter referred to as classification integrated image) is generated. Therefore, the classification unit 180 generates a classification integrated image for each depth.

2A illustrates a pickup process according to an embodiment of the present invention, and FIG. 2B illustrates a restoration process according to an embodiment of the present invention.

Referring to FIG. 2A, the light reflected from the 3D object is projected to the pickup unit 120 through the lens unit 110, and the pickup unit 120 detects the projected light through a sensor such as a CCD or a CMOS. Generate digital images. The image generated by the pickup unit 120 is an integrated image including a plurality of element images, and when the image is reversely projected through the lens unit 110, a stereoscopic image may be displayed.

Referring to FIG. 2B, when the image picked up through the pickup unit 120 is projected onto the lens unit 110 through a display device such as a display panel, a stereoscopic image is formed on the front surface of the lens unit 110. An image can be displayed.

The integrated image reconstruction method may include an optical integrated image reconstruction method (OIIR) or a computer integrated image reconstruction method (CIIR: Computational Integral Imaging Reconstruction).

The restoration method illustrated in FIG. 2B is the above-described optical integrated image restoration method. The optically integrated image reconstruction method is used to display low resolution three-dimensional images due to insufficient overlap between element images caused by physical limitations of the optical device such as rotation and aberration, and deteriorated image quality of the displayed three-dimensional image. There is a problem. In order to compensate for this drawback, a computer integrated image reconstruction method, which reconstructs a 3D image to a computer through digital simulation of geometric optics, is used as a reconstruction method. Hereinafter, a computer integrated image restoration method will be described with reference to FIG. 3.

3 is a conceptual diagram illustrating a computer integrated image restoration method according to an embodiment of the present invention.

Referring to FIG. 3, the restoration unit 150 projects an integrated image onto a restoration plane through a pinhole array. In this case, it is apparent that the integrated image may be projected onto the reconstruction plane by using the lens array instead of the pinhole array. As illustrated in FIG. 3, the reconstructor 150 generates a planar reconstructed image (POI) by overlapping a part of each element image projected on the reconstruction plane separated by z from the pinhole array. In this case, the reconstructor 150 may change z to generate a plurality of planar reconstructed images. That is, the restoration unit 150 may generate a plurality of depth-specific planar restoration images.

The reconstructor 150 may generate a planar reconstructed image for each depth without using a pinhole array or a lens array through the digital simulation of the above-described process.

4 is a diagram illustrating a method of detecting a depth of each element image by an object recognition apparatus according to an embodiment of the present invention.

Referring to FIG. 4, the optical pickup unit 120 picks up an object whose distance is z from the lens unit 110 to generate a reference image 410.

The reconstructor 140 generates the reference reconstructed image 420 for each depth by using the computer integrated image reconstruction method (CIIR) described above with reference to FIG. 3.

Subsequently, the computer pickup unit 150 picks up each reference reconstructed image using a computer pickup method. That is, the computer pickup unit 150 designates a reference reconstructed image, which is a planar image, as an object, and generates a target image 430 by picking it up through a virtual simulation. A function performing process of the optical pickup unit 120, the restoration unit 140, and the computer pickup unit 150 will be described in detail later with reference to FIG. 5.

The correlation calculator 160 calculates a correlation between the reference image 410 and each target image 430.

The depth detector 170 detects a maximum correlation that is the maximum value among the correlations between the reference image 410 and each target image 430. For example, as illustrated in FIG. 4, the depth detector 170 may be included in a reference element image located at (4, 5) of the reference image 410 and a target element image located at (4, 5) of the target image 431. The representative correlation, which is the maximum value among the correlations corresponding to the pixels, is detected. The depth detector 170 detects representative correlations corresponding to reference element images positioned at (4, 5) of the reference image 410 and target element images positioned at (4, 5) of the remaining target image 431, respectively. do. Representative correlations are graphically illustrated as shown in 440 according to depth. Next, the depth detector 160 detects the maximum correlation that is the maximum value among the detected representative correlations. For example, according to 440, the representative correlation is greatest at a depth of 30 mm. Therefore, the depth detector 170 detects a maximum correlation corresponding to the depth of 30 mm.

The depth detector 170 detects the depth of each reference element image as the depth of the target element image having a maximum correlation with the corresponding reference element image. A depth map representing the depth of each reference element image in color may be represented as 450.

5 is a diagram illustrating an optical pickup and computer pickup process according to an embodiment of the present invention.

Referring to FIG. 5, the optical pickup unit 120 generates a reference image by detecting light reflected from an object passing through the lens unit 110.

Subsequently, the reconstructor 140 generates a reference reconstructed image that is a planar reconstructed image corresponding to the reference image depth z _i (i = 1, ..., n, n is an arbitrary natural number).

The computer pickup unit 150 sets the reference reconstructed image from the reconstructor 140 as a flat object located at a depth z _i to perform the pickup process. In this case, since the reference reconstructed image is in the form of digital data, the reference reconstructed image may be picked up using a computer pick-up method. That is, the computer pickup unit 150 may generate a target image, which is one integrated image corresponding to one reference reconstructed image, by using a computer pickup method.

6 is a diagram illustrating a process of classifying, by a classifier, a reference image with reference to a depth map, according to an exemplary embodiment.

Referring to FIG. 6, the classifier 180 classifies each reference element image for each depth according to the depth of each reference element image referring to the depth map. For example, the classifier 180 classifies the reference image and classifies the image by depth. The classifier 180 generates classified classification images in which classified reference element images are arranged at coordinates located on the reference image, and empty element images are disposed at coordinates in which the reference element image is not disposed.

According to an embodiment of the present invention, if a planar reconstructed image having a depth corresponding to each classification integrated image is reconstructed, a sharp object may be detected as compared to the planar reconstructed image reconstructing a general integrated image. For example, when the restoration unit 140 generates a planar restoration image obtained by restoring (b) at a depth of 30 mm, the restoration unit 140 may acquire an image of a specific object such as (g).

Although the object recognition apparatus described above with reference to FIGS. 1 to 5 has been described as calculating a representative correlation based on the element images, the target image and the reference image are divided into blocks having a specified size according to an implementation method, and for each block. It will be apparent to those skilled in the art that a depth detection process may be performed by calculating a representative correlation.

FIG. 7A is a graph illustrating a representative correlation chart calculated in units of element images according to depths, and FIG. 7B is a block having a size obtained by dividing an element image by 2 × 2 according to another embodiment of the present invention. This is a graph showing the representative correlations calculated in units of depth.

Compared to the method of calculating the representative correlation for each element image as shown in FIG. 7A, FIG. 7B generates a representative correlation for each block smaller than the element image, thereby increasing the number of representative correlations to more accurately determine the depth of the object. Can be detected.

8 is a flowchart illustrating a process of detecting a depth of an object recognition apparatus according to an embodiment of the present invention.

The object recognition apparatus generates a reference image by using an optical pickup method in operation 810.

The object recognition apparatus restores the reference image to the planar reconstructed image by using the computer integrated image reconstruction method (820).

The object recognition apparatus picks up the reconstructed plane reconstructed image by using a computer pick-up method and generates a target image in operation 830. For example, the object recognition apparatus generates a plurality of target images by picking up each of the flattened restored images restored in operation 820 using a computer pickup method.

The object recognition apparatus calculates a correlation between the target image and the reference image (840). For example, the object recognizing apparatus calculates a degree of correlation between each pixel of the target image and the same pixel among pixels of the reference image.

The object recognition apparatus groups the correlations for each unit block (850). That is, the object recognizing apparatus divides the pixels of the target image and the reference image by unit blocks, and groups the correlations corresponding to the pixels by unit blocks.

The object recognition apparatus detects a maximum correlation for each unit block (860).

The object recognition apparatus detects a depth corresponding to each unit block of the reference image (870). That is, the object recognizing apparatus detects the depth of the target image in which the maximum correlation corresponding to each unit block of the reference image is detected as the depth of the reference image.

The object recognition apparatus classifies an image of each unit block according to a depth corresponding to each unit block (880).

In operation 890, the object recognition apparatus generates a classification integrated image by arranging an image of each unit block classified according to depth.

The object recognition apparatus generates a planar reconstruction image by reconstructing the corresponding classification integrated image at a depth corresponding to each classification integrated image (operation 895). In this case, the generated planar reconstructed image includes a clearer image than a reconstructed image of a general integrated image.

9 is a view for explaining the effect of the object recognition apparatus according to an embodiment of the present invention, Figure 10a is a diagram illustrating a planar reconstruction image reconstructed a general integrated image, Figure 10b is an embodiment of the present invention 11A and 11B illustrate a correlation between a planar reconstructed image reconstructed from a planar reconstructed image and a general reconstructed reconstructed image according to an embodiment of the present invention. It is a graph.

Referring to FIGS. 9, 10A, and 10B, when an integrated image obtained by optically picking up a mark having a distance of 30 mm and a mark having a distance of 45 mm from a pinhole array or a lens array and an object is generally restored, a planar reconstruction image for each depth as shown in FIG. 9A. Is generated. In FIG. 9A, an object not in focus is blurred in each planar reconstruction image.

In contrast, 9b, which is a planar reconstructed image reconstructing the classification integrated image according to an embodiment of the present invention, shows a sharper image than FIG. 9A. Therefore, the object recognizing apparatus according to the embodiment of the present invention provides an effect of providing an image of a clear object by removing an object that is not in focus. Referring to FIGS. 11A and 11B, a correlation diagram of a planar reconstruction image for each depth of a mark having a distance of 30 mm and a mark having a distance of 30 mm from a pinhole array or a lens array and an object is shown in FIG. It shows a sharp increase in depth of 30mm and depth of 45mm. That is, the object recognition method according to the exemplary embodiment of the present invention may calculate an accurate depth of the object as compared with the general planar restoration method.

So far I looked at the center of the embodiment for the present invention. Many embodiments other than the above-described embodiments are within the claims of the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

1 is a diagram illustrating an object recognition apparatus according to an embodiment of the present invention.

2A illustrates a pickup process in accordance with one embodiment of the present invention.

2B is a diagram illustrating a restoration process according to an embodiment of the present invention.

3 is a conceptual diagram illustrating a computer integrated image restoration method according to an embodiment of the present invention.

4 is a diagram illustrating a method of detecting a depth of each element image by an object recognition apparatus according to an embodiment of the present invention.

5 illustrates an optical pickup and computer pickup process in accordance with an embodiment of the present invention.

6 is a diagram illustrating a process of classifying, by a classifier, a reference image with reference to a depth map, according to an exemplary embodiment.

FIG. 7A is a graph illustrating a representative correlation chart calculated in units of element images according to depths according to an embodiment of the present invention. FIG.

7B is a graph illustrating representative correlations calculated according to depths of blocks having a size obtained by dividing an element image into 2 × 2 units according to another embodiment of the present invention.

8 is a flowchart illustrating a process of detecting a depth of an object recognition apparatus according to an embodiment of the present invention.

9 is a view for explaining the effect of the object recognition apparatus according to an embodiment of the present invention.

10A is a diagram illustrating a planar reconstructed image obtained by reconstructing a general integrated image.

10B is a diagram illustrating a planar reconstructed image reconstructed with a classification integrated image according to an embodiment of the present invention.

11A and 11B are graphs illustrating a correlation between a planar reconstructed image and a planar reconstructed image obtained by reconstructing a general integrated image, according to an exemplary embodiment.

Claims (15)

A first pickup unit photographing an object to generate a reference image; A reconstruction unit reconstructing the reference image to generate a planar reconstruction image for each depth; A second pickup unit which picks up the planar reconstructed image and generates a target image; A correlation calculator calculating a correlation between the reference image and the unit block of the target image; A depth detector configured to detect a depth corresponding to a maximum correlation among the correlations corresponding to a unit block of the reference image as a depth corresponding to the unit block; And And a classification unit classifying the unit blocks by depths and generating a classification integrated image for each depth including the classified unit blocks. The method of claim 1, An object recognizing apparatus further comprising a lens unit including a lens array or a pinhole array. 3. The method of claim 2, And the first pickup unit generates the integrated image by photographing the object through the lens unit. The method of claim 1, And the second pickup unit generates the target image by using a computer pickup method for the planar restored image. The method of claim 1, And the correlation calculator calculates a correlation between the unit blocks at the same position in the reference image and the target image. The method of claim 1, The correlation calculator calculates a correlation by comparing pixels included in the unit block corresponding to the reference image with pixels included in the unit block corresponding to the target image, and calculates an average value of correlations corresponding to the pixels. Calculating a degree of correlation between the unit blocks. The method of claim 1, And a reference image of a predetermined monochrome is located in an area other than an area in which the unit block of the classification integrated image is disposed. The method of claim 1, The reconstructor generates a planar reconstructed image by reconstructing the classification integrated image, And only a planar reconstructed image having a depth corresponding to the classification integrated image is generated. In the object recognition apparatus recognizes an object, Photographing an object to generate a reference image; Restoring the reference image to generate a planar reconstruction image for each depth; Picking up the planar reconstructed image to generate a target image; Calculating a correlation between the reference image and the unit block of the target image; Detecting a depth corresponding to a maximum correlation among the correlations corresponding to a unit block of the reference image as a depth corresponding to the unit block; And Classifying the unit blocks by depth and generating a depth classification image including the classified unit blocks. The method of claim 9, The generating of the reference image may include generating the reference image by photographing the object through a lens array or a pinhole array. The method of claim 9, The generating of the target image may include generating the target image by using a computer pickup method for the planar reconstructed image. The method of claim 9, The calculating of the correlation may include calculating a correlation between the unit blocks having the same position in the reference image and the target image. The method of claim 12, The calculating of the correlation may include calculating a correlation by comparing pixels included in the unit block corresponding to the reference image with pixels included in the unit block corresponding to the target image, and correlating with the pixel. Calculating an average value of degrees as a correlation between the unit blocks. The method of claim 9, And a reference image of a predetermined monochrome is located in an area other than an area in which the unit block of the classification integrated image is disposed. The method of claim 9, The generating of the planar reconstructed image may include generating a planar reconstructed image reconstructing the classification integrated image. And generating only a planar reconstructed image having a depth corresponding to the classification integrated image.

Images