JPH0814858A - Data acquisition device for three-dimensional object - Google Patents

Abstract

PURPOSE:To reduce the cost of a device for obtaining three-dimensional data from an object, by using a spacial light source so as to prevent detrimental affections to an object to be picked up, while simultaneously obtaining three dimensional data and surface texture data from the object, and by simplifying the structure of the device. CONSTITUTION:The motion of a feature point on an object 2 to be picked up is detected from an image (two-dimensional image) per unit turn angle picked up from the object 2 to be picked up by a video camera 5 while the turn angle of a stepping motor 4 is controlled by a computer 6, and from a turn angle of the stepping motor 4, so as to seek the rotating axis of the object to be picked up. Further, three-dimensional data of the object 3 to be picked up are obtained on the basis of the rotating axis, and further, surface texture data are obtained from thus obtained threedimensional data and a two-dimensional image per unit turn angle.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention provides material information for compression of moving images, object recognition, computer graphics creation, etc., using only two-dimensional images taken under general shooting conditions and angle information at the time of shooting. Accurate three-dimensional shape information of three-dimensional objects and accurate surface texture (Surface Tex)
The present invention relates to a three-dimensional object data acquisition device configured to obtain information.

SUMMARY OF THE INVENTION According to the present invention, a three-dimensional shape information of a three-dimensional object and a surface corresponding to the three-dimensional shape information are obtained from a photographed image which is a two-dimensional image without using a special illumination or a special sensor to assist the measurement. The present invention relates to an apparatus for obtaining texture information, and while extracting an object feature point from a two-dimensional image obtained by shooting an image with a video camera or the like while placing the object on a turntable and rotating the object,
By repeatedly calculating the relative inclination of the rotation axis with respect to the lens optical axis of the video camera based on the motion information of each of these feature points, the three-dimensional space in which the subject exists and the two-dimensional camera imaging plane Considering the correspondence as a central projection, using the highly accurate rotation axis information obtained by iterative calculation and known rotation angle information, it is necessary to use a light source such as a laser slit light or accurate camera position information. Compared to the method described above, the three-dimensional shape information of the three-dimensional object and the surface texture information corresponding thereto are obtained with a simpler configuration.

[0003]

2. Description of the Related Art Three-dimensional objects used as material information for compressing moving images, recognizing objects, creating computer graphics, etc.
Conventionally, as methods for obtaining dimensional shape information and surface texture information, active measurement methods that use special light sources such as laser slit light and special sensors, and passive measurement methods that use captured images such as video cameras There is.

An example of the former is a slit light projection method, and in this slit light projection method, three-dimensional shape information of a three-dimensional object is obtained as described below.

First, as shown in FIG. 7, a table 103 is provided at a position separated from a table 102 on which a subject 101 is placed by a predetermined distance, a laser light source 104 is arranged on the table 103, and the laser light source is arranged. A cylindrical lens 105 for converting laser light emitted from 104 into slit light (laser slit light) S is arranged, and the subject 101 is further provided.
The camera 106 is arranged at a position separated by a predetermined distance from the table 102 on which is mounted.

Then, with the subject 101 placed on the base 102, laser light is emitted from the laser light source 104, and this laser light is converted into slit light (laser slit light) S by the cylindrical lens 105, The subject 101 is illuminated and the subject 10 is captured by the camera 106.
1 is captured, and a slit image of the subject 101 corresponding to the laser slit light S is captured on the imaging surface 107 of the camera 106.

As a result, of the three-dimensional space coordinates showing the surface shape of the subject 101, the three-dimensional space coordinates of the point P on the plane formed by the laser slit light S are the point P and the lens center O of the camera 106. It is projected onto a certain point P ′ on the image pickup surface 107 via a straight line L passing through and is cut out on the image pickup surface 107 by the plane formed by the laser slit light S of the surface shape of the subject 101. The outline of the cross section (slit image) is projected.

Hereinafter, the above-mentioned photographing is performed while moving the laser emission direction of the laser light source 104 little by little in the horizontal direction, and the laser slit light S among the surface shapes of the subject 101 is formed on each image pickup surface 107. The outline of the cross section cut by each plane is projected.

Then, with respect to the entire subject 101, each contour line on each slit image obtained when the above-mentioned photographing processing is completed is used as the three-dimensional information of the subject 101 to compress a moving image and recognize an object. , And supplies it to a processing device that creates computer graphics, etc., and causes it to process each of the outlines.

As the latter example, there is a three-dimensional boarding method. In this three-dimensional boarding method, three-dimensional shape information of a three-dimensional object is obtained as described below.

First, a subject is photographed by a camera from a plurality of viewpoints, and as shown in FIG. 8, points which are the features of the subject from the respective images obtained (two-dimensional feature points).
Is extracted, and the line of sight passing through each of these two-dimensional feature points and the lens center of the camera is obtained.

Next, the space in which the subject is present is divided into three-dimensional pixels (hereinafter referred to as voxels), and a predetermined value is added to each voxel through which each line of sight passes, and By obtaining the integrated value for each voxel, the voxels where many lines of sight intersect each other are extracted as the three-dimensional feature points of the subject, and each of these three-dimensional feature points is used as the three-dimensional information of the subject in the moving image. The data is supplied to a processing device that performs compression, object recognition, computer graphics creation, etc., and causes processing of each of the three-dimensional feature points.

[0013]

However, the slit light projection method and the three-dimensional boarding method described above have the following problems.

First, in the slit light projection method shown in FIG.
A laser light source 10 is used as a light source to improve measurement accuracy.
4 is used to narrow the width of the laser slit light S, so that when the subject 101 is a work of art such as paper, the surface of the work of art is dried by the heat of the laser slit light S. There is a possibility that the object 101 will be discolored or discolored, and since only the three-dimensional shape information of the object 101 can be obtained, the surface of the object is photographed separately,
There is a problem that the surface texture information of the subject 101 must be obtained.

Further, in the three-dimensional boarding method shown in FIG. 8, when a plurality of three-dimensional feature points are simultaneously extracted, the processes for the respective three-dimensional feature points interfere with each other to generate false three-dimensional feature points. Due to image quantization error and the like, feature points whose added values are below a certain threshold value cannot be extracted as three-dimensional feature points, and the shape of the shape is changed depending on the quantization degree of the set voxel space. There was a problem that the accuracy was decided.

Further, in principle, accurate information on the camera position is required, but it is very difficult to accurately determine the center position of the camera lens, so it is difficult to extract accurate three-dimensional feature points. , At the moment, it is not a technically established method.

In view of the above circumstances, the present invention prevents adverse effects on a subject due to the use of a special light source,
A three-dimensional object data acquisition apparatus capable of acquiring accurate three-dimensional shape information of a subject and surface texture information at the same time, and greatly simplifying the apparatus configuration and significantly reducing the manufacturing cost of the apparatus. Is intended to provide.

[0018]

In order to achieve the above object, a three-dimensional object data acquiring apparatus according to the present invention is, in claim 1, a rotary drive mechanism for rotating an object in a predetermined angle unit, and the rotary drive mechanism. An imaging mechanism that images the subject and outputs a two-dimensional image for each rotationally driven rotational angle, a two-dimensional image for each rotational angle obtained by the imaging mechanism, and a rotational angle of the rotational drive mechanism. Based on, by detecting the movement of the feature points on the surface of the subject,
A rotation axis of the subject is searched for, and three-dimensional shape information of the subject is created based on the rotation axis and the feature points 3
Surface texture information based on a three-dimensional shape information acquisition processing unit, three-dimensional shape information of the subject obtained by the three-dimensional shape information acquisition processing unit, and a two-dimensional image for each rotation angle obtained by the photographing mechanism. And a surface texture information acquisition processing unit for creating

According to a second aspect of the present invention, in the three-dimensional object data acquisition apparatus according to the first aspect, the three-dimensional shape information acquisition processing section extracts feature points from a two-dimensional image for each rotation angle, and each of these After detecting the movement of the characteristic points and detecting the rotation axes corresponding to the respective characteristic points, the respective rotation axes are linearly arranged to determine the positions of the respective characteristic points in the three-dimensional space, and the object is determined. 3D shape information is created, and the surface texture information acquisition processing unit detects a rotation angle at which the surface area surrounded by each feature point faces the imaging mechanism based on the 3D shape information of the subject. The surface texture information is created based on the pattern of the two-dimensional image of this rotation angle.

[0020]

In the above structure, in the three-dimensional object data acquiring device according to claim 1, the object is photographed by the photographing mechanism for each rotation angle while rotating the subject by a predetermined angle unit by the rotation driving mechanism, and the object is two-dimensionally measured. While generating an image, the three-dimensional shape information acquisition processing unit detects the movement of the feature point on the surface of the subject based on the two-dimensional image for each rotation angle and the rotation angle of the rotation drive mechanism, After searching the rotation axis of the subject, three-dimensional shape information of the subject is created based on the rotation axis and the feature points, and further obtained by the surface texture information acquisition processing unit by the three-dimensional shape information acquisition processing unit. By creating the surface texture information based on the three-dimensional shape information of the subject and the two-dimensional image for each rotation angle obtained by the photographing mechanism, The accurate production of 3D shape information and surface texture information of the subject is simultaneously obtained while preventing the adverse effect on the subject due to the use of various light sources, and the configuration of the device is greatly simplified. Is significantly reduced.

According to a second aspect of the present invention, in the three-dimensional object data acquisition apparatus according to the first aspect, the three-dimensional shape information acquisition processing section extracts feature points from a two-dimensional image for each rotation angle, and After detecting the movement of each feature point and detecting the rotation axis corresponding to each feature point, these rotation axes are linearly arranged to determine the position of each feature point in the three-dimensional space. Based on the three-dimensional shape information of the subject, the three-dimensional shape information of the subject is created, and based on the three-dimensional shape information of the subject, the surface area surrounded by each feature point faces the photographing mechanism directly. Is detected and surface texture information is created based on the pattern of the two-dimensional image of this rotation angle, thereby preventing adverse effects on the object due to the use of a special light source, as in claim 1. Positive A three-dimensional shape information, and a surface texture information at the same time, acquires, very simplified device structure significantly reduces the cost of the manufacturing apparatus.

[0022]

FIG. 1 is a block diagram showing an embodiment of a three-dimensional object data acquisition device according to the present invention.

The three-dimensional object data acquisition device 1 shown in this figure is
A turntable 3 on which the subject 2 is placed, a stepping motor 4 for rotating the turntable 3 by a constant angle, and a video camera 5 arranged at a position separated from the subject 2 by a predetermined distance to photograph the subject 2. ,
While controlling the operation of the stepping motor 4, a computer 6 that captures an image obtained by the video camera 5 and simultaneously obtains accurate three-dimensional shape information of the subject 2 and surface texture information is provided. There is.

Then, while controlling the rotation angle of the stepping motor 4 by the computer 6, an image (two-dimensional image) for each rotation angle obtained by photographing the subject 2 by the video camera 5 and the stepping motor Based on the rotation angle of 4, the movement of the characteristic points on the surface of the subject 2 is detected, the rotation axis of the subject 2 is searched, and the three-dimensional shape of the subject 2 is obtained based on the information of the rotation axis. While creating information, this three-dimensional shape information,
A process of creating surface texture information based on the two-dimensional image for each rotation angle and performing compression of a moving image, recognition of an object, creation of computer graphics, etc., with the three-dimensional shape information and the surface texture information. Supply to the device.

Next, the processing of the computer 6 shown in FIG. 1 will be described in detail with reference to the block diagram shown in FIG.

First, the computer 6, as shown in FIG. 2, has a photographing and angle control unit 10, a feature point detection unit 11, a motion detection unit 12, a rotation axis search unit 13, a reprojection unit 14, and a reprojection unit 14.
The function is divided into the texture geometric transformation and averaging unit 15, and the image capturing and angle control unit 10 performs image acquisition processing for each rotation angle, and the feature point detection unit 11
The motion detection unit 12, the rotation axis search unit 13, and the reprojection unit 14 perform the acquisition processing of the three-dimensional shape information, and the texture geometric conversion and averaging unit 15 performs the acquisition processing of the surface texture information.

<< Image Acquisition Processing for Each Rotation Angle >><Photographing and Angle Control Unit 10> Photographing and Angle Control Unit 1
0 is a rotation angle unit such as a one-step unit or a half-step unit, and the stepping motor 4 is rotationally driven to rotate the turntable 3 on which the subject 2 is placed, and at the same time, with the video camera 5 for each rotation angle. A subject 2 is photographed and an image (two-dimensional image) of the subject 2 is captured, and as shown in FIG. 3, a two-dimensional array in which the vertical direction of the subject 2 is a row and the horizontal direction of the subject 2 is a column. The pixel is retained as a pixel. The brighter the pixel data (pixel data) arranged two-dimensionally, the larger the numerical value.

Then, a two-dimensional image for each rotation angle obtained by rotating the subject 2 once is used by the feature point detection unit 11,
It is supplied to the motion detection unit 12 and the texture geometric transformation and averaging unit 15.

<< Acquisition Processing of Three-Dimensional Shape Information >><Characteristic Point Detection Unit 11> The characteristic point detection unit 11 captures the two-dimensional image for each rotation angle output from the photographing and angle control unit 10 and After dividing each pixel of the three-dimensional image into a plurality of areas by dividing each pixel by 8 pixels in the row direction and the column direction, the contour strength of each image data is obtained, and the square sum of the contour strengths included in each of these areas is calculated. , The center points of all regions that are equal to or greater than a preset threshold value are extracted as feature points of this two-dimensional image, and these feature points are used as original feature points and the motion detection unit 12 and the rotation axis Search unit 12
And the reprojection unit 14.

<Motion Detection Unit 12> The motion detection unit 12 outputs the original feature point for each rotation angle output from the feature point detection unit 11 and the rotation angle output for each rotation angle output from the photographing and angle control unit 10. While capturing the two-dimensional image, based on the original feature points on the one two-dimensional image, the most similar content portion in the other two-dimensional image for which correspondence should be obtained is extracted,
Block matching is performed using a method in which this portion is used as feature point coordinates in each two-dimensional image, and each feature point of the subject 2 rotates to determine how the image plane of the video camera 5 moves. Calculate as a value.

Actually, based on the original feature points on one two-dimensional image, an area in another two-dimensional image for which correspondence should be obtained is selected, and the difference in pixel brightness between the same coordinates in this area is selected. Is calculated for all the pixels in the area, and the portion where the sum of squares (evaluation value) of each pixel obtained thereby is the minimum is set as the characteristic point corresponding to the original characteristic point.

Then, the coordinates of the corresponding original feature points are subjected to block matching with the two-dimensional images at different angles, and the corresponding original feature points of the respective two-dimensional images are assigned to the corresponding original feature points. The feature points are put together and the coordinates (the coordinates on each two-dimensional image) of each of these original feature points are supplied to the rotation axis search unit 13 and the texture geometric transformation / averaging unit 15 as a feature point coordinate group.

However, since the corresponding original feature point may not exist depending on the two-dimensional image, when the evaluation value is less than a certain threshold value, the corresponding two-dimensional image is taken on the two-dimensional image photographed at the rotation angle. The process of determining that the original feature points do not exist is performed, and only when three or more original feature points correspond to each other, the original feature points are used as the feature point coordinate group, the rotation axis search unit 13, the texture geometric transformation, and Supply to the averaging unit 15.

<Rotation Axis Search Unit 13> The rotation axis search unit 13 includes each feature point coordinate group output from the motion detection unit 12,
The original feature points output from the feature point detection unit 11 are fetched, one feature point coordinate group is selected from each feature point coordinate group, and each of the original feature points constituting the feature point coordinate group is selected. After extracting three original feature points from among them, as shown in FIG. 4, one plane including three straight lines passing through each of these original feature points and the center of the lens of the video camera 5 (a certain vector is a normal line). Plane) is assumed.

As shown in FIG. 5, each projection point obtained by projecting these three original feature points on each plane through each straight line is one of the hypotenuses with the rotation axis of the subject 2 as the center. Since the two isosceles triangles having the same shape are formed, the lengths L1 and L2 of the bases are equal, and the rotation angle of the turntable 3 when obtaining each two-dimensional image is θ.
Then, since the sum of the interior angles of each of these isosceles triangles is π, the following equation holds and θ + φ / 2 + φ / 2 = π where φ / 2: the base angle of the isosceles triangle , The following equation is obtained.

[Pi]-[theta] = [phi] Therefore, if a projection point satisfying the condition 1 shown in the following equation is selected, it is possible to find the projection point of the moving original feature point caused by the rotation of the turntable 3.

Condition 1: π-θ = φ L1 = L2 where φ: angle formed by two straight lines connecting projection points of each original feature point L1: distance between projection points of two original feature points L2: two Distance between projection points of original feature points Then, the rotation axis direction is detected based on the position of the projection point of each of these original feature points, and this is used as an axis having a relative inclination with respect to the lens optical axis of the video camera 5. The vector is supplied to the reprojection unit 14.

In this embodiment, the rotation direction of the subject 2 is not input at the time of shooting, but the movement of the projection point on the plane whose normal is the axis vector obtained in this sequence is the same as that at the time of shooting. It is the direction of rotation with respect to the axis vector.

Thereafter, the above-described processing is performed on each feature point coordinate group to detect the axis vector corresponding to each original feature point forming each feature point coordinate group, and this is reprojected by the reprojection unit 14.
Supply to.

<Reprojection Unit 14> The reprojection unit 14 takes in each axis vector output from the rotation axis search unit 13 and each original feature point output from the feature point detection unit 11 and also outputs each original point. Regarding the feature point coordinate group in which three or more feature point coordinates are obtained based on the feature point information, the points equidistant from each projection point on the plane whose axis is the normal line are the reference points, and With respect to the feature point coordinate group having two point coordinates, the axes from the rotation axis search unit 13 are equidistant from the projection point on the plane having the axis vector as the normal and when the point is the center. A point that satisfies the condition of the rotation direction with respect to the vector is set as a reference point.

After that, as shown in FIG. 6, the reference points obtained for the respective feature point coordinate groups are in the same direction as the axis vector in the three-dimensional space, and the normal direction is held so that they are aligned in a straight line. In this state, the projection plane is moved, shape data is obtained based on the projection points of the respective original feature points obtained thereby, and this is supplied to the texture geometric transformation and averaging unit 15 as the three-dimensional shape information of the subject 2. At the same time, the image data is supplied to a processing device that compresses moving images, recognizes objects, creates computer graphics, and the like.

Actually, a straight line whose direction is equal to the axis vector passing through the reference point obtained from a certain characteristic point coordinate group
The plane to be projected is moved so that the reference points of the other feature point coordinate groups are closest to each other, recalculation is performed, and the projection points of the respective original feature points obtained by this are used as the three-dimensional shape information of the subject 2. There is.

<< Process for Obtaining Surface Texture Information >><Texture Geometric Transformation and Averaging Unit 15> The texture geometric transformation and averaging unit 15 includes the three-dimensional shape information of the subject 2 output from the reprojection unit 14, and Each axis vector output from the motion detection unit 12 and each two-dimensional image output from the shooting and angle control unit 10 are captured, and at each of three feature points based on the three-dimensional shape information of the subject 2. The enclosed surface area is a three-dimensional space, and the rotation angle facing the lens optical axis of the video camera 5 is detected, and the pattern of the two-dimensional image of this rotation angle and another two-dimensional image corresponding to this rotation angle. After geometrically transforming (two-dimensional image including the surface area) to obtain a surface pattern, these are averaged to obtain an average pattern of the surface area.

When the positions of the surrounding characteristic points are different, the overlapping portions are averaged to obtain an average pattern.

Hereinafter, the above-described processing is performed on the other surface area surrounded by the three feature points on the surface of the subject 2 to obtain an average pattern of the entire surface of the subject 2, Is supplied as surface texture information to a processing device that performs moving image compression, object recognition, computer graphics creation, and the like.

As described above, in this embodiment, the computer 6 controls the rotation angle of the stepping motor 4 while the video camera 5 controls the object 2 to be photographed.
Image (two-dimensional image) for each rotation angle obtained by shooting
And based on the rotation angle of the stepping motor 4,
The movement of the characteristic points on the surface of the subject 2 is detected, the rotation axis of the subject 2 is searched, and the three-dimensional shape information of the subject 2 is created based on this rotation axis. Since the surface texture information is created based on the information and the two-dimensional image for each rotation angle, the following effects can be obtained.

First, unlike the conventional slit light projection method, the subject 2 is rotated and photographed without using a special light source such as a laser slit light, thereby obtaining three-dimensional shape information. , Surface texture information can be obtained at the same time.

Furthermore, as in the conventional three-dimensional boarding method, the integrated value 3 is less than the threshold value.
Dimensional feature points were unsuccessfully extracted, accuracy was reduced due to difficulty in obtaining accurate camera position information, precision limit due to use of quantized voxel space, and all feature points were processed simultaneously. False 3 that sometimes occurs
Accurate 3 while eliminating inconveniences such as 3D feature point extraction
It is possible to extract only the dimensional feature points and obtain accurate surface texture information.

Further, in the above-described embodiment, the feature point detection unit 11 divides the two-dimensional image for each rotation angle output from the photographing and angle control unit 10 into eight pixels to form a plurality of regions. However, the regions may be divided into other regions by other division methods, and the feature points may be extracted by a method other than the sum of squares.

[0050]

As described above, according to the present invention, in claims 1 and 2, accurate three-dimensional shape information of a subject is obtained while preventing adverse effects on the subject due to the use of a special light source. Surface texture information can be acquired at the same time, and the device configuration is greatly simplified,
The manufacturing cost of the device can be significantly reduced.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an embodiment of a three-dimensional object data acquisition device according to the present invention.

FIG. 2 is a block diagram showing a functional configuration example of a computer shown in FIG.

FIG. 3 is a schematic diagram showing a format example of a two-dimensional image output from the shooting and angle control unit shown in FIG.

FIG. 4 is a schematic diagram showing an example of a rotation axis search process of a rotation axis search unit shown in FIG.

5 is a schematic diagram showing an example of a rotation axis search process of a rotation axis search unit shown in FIG.

6 is a schematic diagram showing an example of reprojection processing by the reprojection unit shown in FIG.

FIG. 7 is a configuration diagram for explaining a slit light projection method among active measurement methods conventionally known as a method for acquiring three-dimensional shape information of a three-dimensional object.

FIG. 8 is a diagram showing a method of acquiring three-dimensional shape information of a three-dimensional object, which is one of the three known passive measurement methods.
It is a block diagram for explaining a dimensional boarding method.

[Explanation of symbols]

1 three-dimensional object data acquisition device 2 subject 3 turntable (rotational drive mechanism) 4 stepping motor (rotational drive mechanism) 5 video camera (shooting mechanism) 6 computer 10 shooting and angle control unit (rotational drive mechanism, shooting mechanism) 11 feature check Output unit (3D shape information acquisition processing unit) 12 Motion detection unit (3D shape information acquisition processing unit) 13 Rotation axis search unit (3D shape information acquisition processing unit) 14 Reprojection unit (3D shape information acquisition processing unit) ) 15 Texture geometric transformation and averaging section (surface texture information acquisition processing section)

Claims (2)

[Claims] 1. A rotary drive mechanism for rotating a subject in units of a predetermined angle, and a photographing mechanism for photographing the subject and outputting a two-dimensional image for each rotation angle rotationally driven by the rotary drive mechanism. Based on the two-dimensional image for each rotation angle obtained by the photographing mechanism and the rotation angle of the rotation drive mechanism, the movement of the characteristic point on the surface of the subject is detected to search the rotation axis of the subject, Based on this rotation axis and the characteristic points,
A three-dimensional shape information acquisition processing unit that creates three-dimensional shape information of the subject, and three-dimensional shape information of the subject obtained by the three-dimensional shape information acquisition processing unit and each rotation angle obtained by the imaging mechanism. A three-dimensional object data acquisition device, comprising: a surface texture information acquisition processing section that creates surface texture information based on the two-dimensional image. 2. The three-dimensional object data acquisition device according to claim 1, wherein the three-dimensional shape information acquisition processing unit extracts a feature point from a two-dimensional image for each rotation angle, and calculates the movement of each feature point. After detecting and detecting the rotation axis corresponding to each feature point, these rotation axes are arranged in a straight line to determine the position of each feature point in the three-dimensional space to obtain the three-dimensional shape information of the subject. Further, the surface texture information acquisition processing unit detects a rotation angle at which the surface area surrounded by each feature point faces the photographing mechanism based on the three-dimensional shape information of the subject,
A three-dimensional object data acquisition device characterized in that surface texture information is created based on the pattern of a two-dimensional image of this rotation angle.