JP2004086484A - Image processor and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor and an image processing method capable of easily and precisely restoring an object. <P>SOLUTION: An object restoring device 1 is provided with an image pickup part 11, a plane defining part 12, a projection converting part 13, a projecting point comparing part 14, and an object storing part 17. The imaging part 11 photographs an object A. The plane defining part 12 defines a plurality of planes on a three-dimensional space. The projection converting part 13 projection-converts each photographed image of the object A into a plurality of planes. The projecting point comparing part 14 compares the pixel values of the respective projection-converted images. The object restoring part 17 restores the group of points whose pixel values are matched as the object B on the three-dimensional space as the result of comparison by the projecting point comparing part 14. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing device and an image processing method.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, a technique for measuring a three-dimensional position of an object using an image captured by a stereo camera has been proposed. For example, Japanese Patent Application Laid-Open No. 2000-293693 discloses the following obstacle detection method and apparatus. The method and the apparatus associate a single point in a three-dimensional space between images captured by a plurality of stereo cameras (search for a corresponding point) and measure a three-dimensional position using a well-known triangulation method. Is what you do.
[0003]
[Problems to be solved by the invention]
However, in order to realize such a measurement technique, various data such as the position of each stereo camera, the imaging direction, the focal length of the lens, and the relative position between the cameras, as well as the parameters of the relational expression using each data, are used. It must be calculated in advance. However, in order to execute a process of calculating a parameter (calibration), it is necessary to prepare a number of known points in a three-dimensional space, which requires a great deal of labor and time.
[0004]
In view of such difficulties, a planar projection stereo method capable of detecting an obstacle without performing complicated calibration has been proposed. Since the plane projection stereo method is a well-known and conventional position measurement technique, detailed description is omitted, but for points on a plane defined in a three-dimensional space, corresponding points are calculated using a predetermined conversion formula, and This is a technique for detecting an object that is not located on the plane based on the difference between pixel values of points.
[0005]
Thus, for example, when a road is defined as a plane, a set of points not located on the plane is detected as an obstacle.
However, even with such a method, it is necessary to search for a corresponding point. In addition, even though the object can be detected, the object existing in the three-dimensional space cannot be accurately restored.
[0006]
Therefore, an object of the present invention is to provide an image processing apparatus and an image processing method that can easily and accurately restore an object.
[0007]
[Means for Solving the Problems]
The present invention has the following features to solve the above problems.
An image processing apparatus according to the present invention includes first and second photographing means for photographing an object, definition means for defining a plurality of planes to which the image of the object is projected and converted, and first and second photographing means. Projection conversion means for projecting and transforming each image of the object photographed by the photographing means to a plurality of planes defined by the definition means; and a pixel value of each image projected and transformed by the projection transformation means for each pixel. And a restoring means for restoring a set of points whose pixel values match as a result of the comparison by the comparing means as an object in space.
[0008]
In the image processing method according to the present invention, the image processing apparatus may include a photographing step of photographing an object by first and second photographing means, and the image processing apparatus may include a plurality of images to be projected and converted to an image of the object. A definition step of defining a plane, and the image processing apparatus, a projection conversion step of projecting and converting each image of the object captured in the imaging step to a plurality of planes defined in the definition step, A comparison step in which the image processing device compares pixel values of each image projected and transformed in the projection transformation step for each pixel, and a result of the comparison in the comparison step, in which the pixel values match. Restoring a set of points as an object in space.
[0009]
According to these inventions, first, a plurality of planes are defined in the space. Subsequently, each image of the same object photographed by the first and second photographing means is projected and transformed into a plurality of defined planes. Then, the pixel values of the projected and transformed images are compared for each pixel, and a set of points on each of the images at which the pixel values match is restored as an object in the space. Therefore, when the object existing in the space is restored, it is not necessary to search for the corresponding point which has been conventionally required. Further, since the number and direction of the planes defined in the space are arbitrary, the restoration accuracy of the object is improved by taking many definition planes or taking the direction of the definition plane parallel to the boundary surface of the object. As a result, the object can be easily and accurately restored.
[0010]
Preferably, the image processing apparatus according to the present invention further includes a detecting unit that detects an edge of the object, and the defining unit defines a plurality of planes using the edge detected by the detecting unit.
[0011]
According to the present invention, an edge of an object is detected, and a plurality of planes are defined based on the edge. That is, by detecting the edge, the end of the object becomes clear, and the range in which the plane should be defined can be accurately recognized. Therefore, by defining a plane within the recognized range (with the end of the object as a limit), a wasteful process of defining a plane on which a captured image of the object is not projected and transformed is omitted. it can. As a result, the object restoration processing efficiency is improved.
[0012]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a block diagram illustrating a functional configuration of an object restoration device 1 according to the present embodiment. As shown in FIG. 1, the object restoration device 1 (corresponding to an image processing device) includes an imaging unit 11, a projection conversion unit 13 (corresponding to a projection conversion unit), a plane definition unit 12 (corresponding to a definition unit), It includes a projection point comparison unit 14 (corresponding to a comparison unit), a voting unit 15, a definition plane control unit 16, and an object restoration unit 17 (corresponding to a restoration unit). Preferably, the object restoration device 1 further includes an edge detection unit 18 (corresponding to a detection unit). These units are electrically connected so that various signals can be input and output.
[0013]
The imaging unit 11 includes two parallel stereo cameras 11a and 11b (corresponding to first and second photographing units) that can photograph the same imaging target A (corresponding to the target) from different viewpoints. The cameras 11a and 11b have the same angle of view and the number of pixels, respectively, and the distance between them in the direction perpendicular to the imaging direction is variable. The imaging unit 11 outputs a stereo image captured by each of the cameras 11a and 11b to the projection conversion unit 13.
[0014]
The plane definition unit 12 defines a plurality of YZ planes having a predetermined interval in the X-axis direction on the XYZ three-dimensional space where the object A exists. The predetermined interval varies depending on the size of the imaging target A, the scale of the captured image, and the like, but is preferably a short interval from the viewpoint of improving the accuracy of the object restoration processing. Similarly, the plane defining unit 12 defines a plurality of XZ and XY planes having predetermined intervals in the Y and Z axis directions on the XYZ three-dimensional space.
[0015]
The projection conversion unit 13 performs a projection conversion of each stereo image input from the imaging unit 11 onto each of YZ, XZ, and XY planes defined by the plane definition unit 12. A specific method of the projection conversion will be described later in the description of the operation.
[0016]
The projection point comparison unit 14 compares the pixel values of all points (hereinafter, referred to as “projection points”) projected from each stereo image at substantially the same position on each of the YZ, XZ, and XY planes. The projection point comparison unit 14 detects a projection point having a similar pixel value difference for all the planes defined by the plane definition unit 12.
[0017]
The voting unit 15 votes the pixel values at all positions in the three-dimensional space corresponding to the projection points detected by the projection point comparison unit 14.
The definition plane control unit 16 determines whether it is necessary to move the definition plane of the projection transformation destination in the layer direction (for example, in the case of the YZ plane, the X-axis direction). The definition plane control unit 16 moves the definition plane by a predetermined distance in the layer direction according to the result of the determination.
[0018]
The object restoring unit 17 restores a set of points whose pixel values match as a result of the comparison by the projection point comparing unit 14 as an object B (corresponding to an object) in the three-dimensional space.
The edge detection unit 18 detects an edge from the image of the object A captured by the imaging unit 11 by a well-known method, and specifies an end of the object A.
[0019]
Next, the operation of the object restoration device 1 according to the present embodiment will be described with reference to FIGS. In addition, each step of the image processing method according to the present invention will be described.
First, in S1 of FIG. 2, the imaging unit 11 captures an image of the object A using the stereo cameras 11a and 11b. As a result, two stereo images are obtained.
[0020]
In the following S2, the plane definition unit 12 defines a YZ plane perpendicular to the X axis on the three-dimensional space of XYZ.
In S3, the projection conversion unit 13 performs projection conversion of one stereo image (the image captured by the stereo camera 11a) onto the YZ plane defined in S2 by the plane definition unit 12 by the plane projection stereo method.
[0021]
Since such projection conversion processing is a well-known and commonly used coordinate system conversion processing, detailed description including illustration is omitted, and a preferred processing example will be briefly described below. That is, the coordinate system transformation matrix H when projecting an image on a plane is represented by H = R + tn ^T / d. Here, R is a rotation matrix between the stereo cameras 11a and 11b, t is an amount of translation between the stereo cameras 11a and 11b, n is a normal vector of the calibration plane, and d is a distance from 0 to the calibration plane. It is.
[0022]
The coordinate system conversion by the plane projection stereo method can be performed by a conversion equation g ′ = Hg, where g is the coordinate before the conversion and g ′ is the coordinate after the conversion. This coordinate system conversion matrix H can be calculated from the coordinates of four or more points on the defined YZ plane. Here, R can be ignored since the defined YZ plane moves in the X-axis direction. Therefore, by changing d every fixed amount, the defined YZ plane can be moved. The coordinate system conversion is not limited to the YZ plane, but can be similarly performed on the XZ plane and the XY plane.
[0023]
In S4, the projection conversion unit 13 uses the same technique as S3 to apply the other stereo image (the image captured by the stereo camera 11b) to the YZ plane defined in S2 by the plane definition unit 12 by the planar projection stereo method. Perform projection transformation.
[0024]
In S5, the projection point comparison unit 14 compares the pixel values of all the projection points projected at substantially the same position on the YZ plane from each stereo image in S3 and S4. Here, the points at substantially the same position include not only points whose coordinates are completely the same but also peripheral points projected within a predetermined range (for example, within 5 pixels) from one point. The pixel value is, for example, luminance (brightness), saturation, or the like.
As a result of the comparison in S5, a projection point having a pixel value difference equal to or smaller than a certain value is detected with respect to the plane defined in S2.
[0025]
In S6, the voting unit 15 votes the pixel values at all positions in the three-dimensional space corresponding to the detected projection points. FIG. 3 is a diagram conceptually showing a state in which a plurality of projection points D1, D2, and D3 detected on the definition plane C are voted on the three-dimensional space E. As shown in FIG. 3, the three-dimensional space E is composed of a large number of cubes (hereinafter, referred to as “voxels”). The projection points D1, D2, D3 are voted for voxels having the same coordinates in the three-dimensional space E.
[0026]
In S7, the definition plane control unit 16 sets the definition plane that is the projection conversion destination at this time in the layer direction (for example, the X axis direction in the case of the YZ plane, the Y axis direction in the case of the XZ plane, and the XY plane). In this case, it is determined whether or not it is necessary to move in the Z-axis direction). As a result of the determination, if movement in the layer direction is necessary (S7; Y), the definition plane control unit 16 moves the definition plane by a predetermined distance in the layer direction (S8).
[0027]
FIG. 4 is a diagram illustrating an example of the defined YZ plane. As shown in FIG. 4, a plurality of YZ planes can be defined by arbitrarily setting the value of X. As described above, it is desirable that as many YZ planes as possible are defined within a range in which the processing load and the speed of the object restoring device 1 are considered in order to increase the voting points and increase the object restoration accuracy. The distance d1 between the YZ planes X1 and X2 and the distance d2 between the YZ planes X2 and X3 do not necessarily have to be the same. Further, d1 and d2 are determined independently of one side of the voxel.
After the movement, the process returns to S3, and the above-described series of processing from S3 to S7 is performed on the moved definition plane.
[0028]
As a result of the determination in S7, when the movement in the layer direction becomes unnecessary (S7; N), the process proceeds to S9 and subsequent steps. Here, a preferred example of a criterion for determining whether or not the definition plane needs to be moved will be described with reference to FIG. FIG. 5 is a diagram conceptually showing a positional relationship between a target object and a YZ plane defined so as to be in contact with the detected edge. In FIG. 5, the edge (boundary line) of the object H is detected by the edge detection unit 18, and the ends G1 and G2 are specified.
[0029]
The YZ planes X4 and X5 are defined so as to be in contact with the ends G1 and G2. In other words, the longest distance L of the object H in the X-axis direction is equal to the distance L 'between the YZ planes X4 and X5. That is, the target object H does not exist in the positive X-axis direction than the YZ plane X4, and does not exist in the negative X-axis direction than the YZ plane X5. Therefore, by defining the YZ planes X4 and X5 as defining planes (movement limits) at both ends in the X-axis direction, there is no need to define a plane where a projection point cannot exist. As a result, an efficient object restoration process can be realized using the minimum necessary definition plane.
Although the YZ plane has been representatively described, the same effect can be obtained by performing the same processing on the XZ and XY planes.
[0030]
In S9, the definition plane control unit 16 sets the definition plane that is the projection conversion destination at the present time in the center axis direction (for example, the Z axis is used as the rotation axis in the case of the YZ plane, and the X axis is used in the case of the XZ plane). (As a rotation axis). As a result of the determination, if rotation in the axial direction is necessary, the definition plane control unit 16 rotates the definition plane about the axis by a predetermined angle (for example, 90 degrees) (S10).
After the rotation, the process returns to S3 again, and the above-described series of processing of S3 to S9 is performed on the rotated definition plane.
[0031]
For example, when the rotation angle to the Z axis in S10 is 90 degrees (or 270 degrees) and the movement distance in the Y axis direction in S8 is d3, d4, as shown in FIG. Planes Y1, Y2, and Y3 are defined.
Note that d3 and d4 need not necessarily be the same. Further, d3 and d4 are determined independently of one side of the voxel.
[0032]
Similarly, when the rotation angle to the X axis in S10 is 90 degrees (or 270 degrees) and the movement distance in the Z axis direction in S8 is d5, d6, as shown in FIG. An XY plane Z1, Z2, Z3 is defined.
Note that d5 and d6 need not necessarily be the same. Further, d5 and d6 are determined independently of one side of the voxel.
[0033]
Returning to FIG. 3, for example, when the movement amount e of each definition plane is equal to the side length f of the voxel (e = f), the definition plane is set in the three-axis direction, and as a result, the voxels forming the three-dimensional space E are set. Voting is performed three times per voxel. The pixel values of the voxel by these three votes are pixel values obtained from the same point in the space of the voting source, and therefore, the same pixel value should be obtained for the same voxel. For this reason, when a different pixel value is obtained, a point in the voting source space is detected as noise.
[0034]
Regarding the same voxel, even when the number of votes is four or more or two or less, a point in the voting source space is detected as noise. As a result, only points whose reliability is equal to or more than a predetermined value are voted for each voxel in the three-dimensional space E, and the voting points having high reliability constitute the restored object B.
[0035]
Further, for example, when one side length f of a voxel is longer than the movement amount e of each definition plane (e <f), a plurality of pixel values obtained from the same definition plane are voted for one voxel. Therefore, the influence of noise can be reduced.
As a result of the determination in S9, when the rotation about the predetermined axis becomes unnecessary (S9; N), the series of object restoration processing ends.
[0036]
As described above, the present invention provides a technique for detecting a plane area using a plane projection stereo method in view of the point that coordinates at which an object is present are projected and converted to substantially the same position on a definition plane. Applied to the restoration of That is, the object restoring device 1 according to the present invention defines the planes YZ, XZ, and XY in the three-dimensional space where the object A exists, and the object A captured by the two stereo cameras 11a and 11b. Each image is projected onto a plurality of defined planes. Then, the pixel values of each of the projected and transformed images are compared, and a set of points having the same pixel value is restored as the object B in the three-dimensional space E.
[0037]
Therefore, when the object existing in the space is restored, it is not necessary to search for the corresponding point which has been conventionally required. Also, since the number and direction of the planes to be defined are arbitrary, the restoration accuracy of the object is improved by adopting many definition planes or by taking the direction of the definition plane parallel to the boundary surface of the object. For example, by applying the object restoring device 1 to obstacle detection when a vehicle executes an automatic parking process, a parking route can be planned easily and accurately.
[0038]
Note that the present invention is not limited to the above-described embodiment, and it is possible to appropriately adopt modified embodiments without departing from the gist of the present invention. For example, in the present embodiment, one plane is first defined, and the movement process and the rotation process are repeatedly performed on the plane, thereby voting the pixel values from a plurality of planes. However, a plurality of planes having a predetermined distance interval and direction may be defined in advance, and pixel values may be simultaneously voted from each plane.
[0039]
【The invention's effect】
According to the present invention, it is possible to easily and accurately restore an object.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a functional configuration of an object restoration device according to the present invention.
FIG. 2 is a flowchart illustrating an object restoration process.
FIG. 3 is a diagram conceptually showing a state of voting a projection point having an approximate pixel value on a three-dimensional space.
FIG. 4 is a diagram illustrating an example of a defined YZ plane.
FIG. 5 is a diagram conceptually showing a positional relationship between a target object and a YZ plane defined so as to be in contact with a detected edge.
FIG. 6 is a diagram illustrating an example of a defined XZ plane.
FIG. 7 is a diagram illustrating an example of a defined XY plane.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Object restoration apparatus, 11 ... Imaging part, 11a, 11b ... Stereo camera, 12 ... Plane definition part, 13 ... Projection conversion part, 14 ... Projection point comparison part, 17 ... Object restoration part, 18 ... Edge detection part, A … Imaging object, B… Reconstructed object

Claims (3)

First and second photographing means for photographing an object;
Definition means for defining a plurality of planes to be the projection transformation destination of the image of the object,
Projecting conversion means for projecting and converting each image of the object photographed by the first and second photographing means to a plurality of planes defined by the defining means;
A comparison unit that compares pixel values of each image projected and converted by the projection conversion unit for each pixel,
An image processing apparatus comprising: a restoration unit that restores a set of points having the same pixel value as a result of the comparison by the comparison unit as an object in space. Further comprising a detecting means for detecting an edge of the object,
The image processing apparatus according to claim 1, wherein the definition unit defines a plurality of planes using the edge detected by the detection unit. A photographing step in which the image processing device photographs an object by the first and second photographing means;
The image processing apparatus, a definition step of defining a plurality of planes to be a projection transformation destination of the image of the object,
A projection conversion step in which the image processing apparatus projects and converts each image of the target object shot in the shooting step to a plurality of planes defined in the definition step;
A comparison step in which the image processing device compares pixel values of each image projected and transformed in the projection transformation step for each pixel;
A restoring step of restoring, as a spatial object, a set of points whose pixel values match as a result of the comparison in the comparing step.

Images