JPH1079029A - Stereoscopic information detecting method and device therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a stereoscopic information detecting method detecting stereoscopic shape information on a subject and texture information on the subject from plural pieces of picture data. SOLUTION: Video signals inputted from cameras 3a to 3c are converted into a picture format easily handled by a stereoscopic information detecting part 8 by a video signal part 5. The result is written in a memory 6 as one sheet of picture data by each view point by CPU 9. In addition CPU 9 writes a camera parameter including subject position information previously stored within a storage device 7, view point position information of the respective cameras 3a to 3c and information on a focal distance, a picture angle, etc., and information on the position and the characteristic of a light source 2 in the memory 6 with picture data. A stereoscopic information detecting part 8 executes processing for detecting stereoscopic information consisting of stereoscopic shape information of the subject 1 and the characteristic (texture information) of a subject surface.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional information detecting method for detecting three-dimensional information comprising three-dimensional shape information of a subject and texture information such as surface reflectance from an image signal obtained by photographing the subject. Regarding the device.

[0002]

2. Description of the Related Art Conventionally, as one of methods for detecting three-dimensional shape information of an arbitrary subject, a subject is photographed from a plurality of different viewpoints to acquire image data, and a relative viewpoint positional relationship at the time of photographing and an image. There is a method of detecting a distance to a subject from the position of the subject in the data using the principle of triangulation, and integrating the distance to detect three-dimensional shape information of the subject.

FIG. 8 is an explanatory diagram showing the principle of the above-mentioned conventional method. In the figure, when one point A of an object is photographed by two cameras 3a and 3b (viewpoint positions OL and OR), a shift (PR-PL) occurs in the position in the image data due to the parallax between the two viewpoints. By calculating the amount of this shift, the position of the point A in an arbitrary space can be determined based on the optical axis direction, the focal length, and the relative positional relationship between the two cameras 3a and 3b at the time of shooting. .

The actual processing is performed as follows. First, the subject is photographed from a plurality of viewpoint positions, and at this time, the relative positional relationship between each viewpoint position and the subject position is recorded. Next, from a plurality of pieces of image data captured from a plurality of viewpoint positions, a correspondence relationship in pixel units between the plurality of pieces of image data is obtained using a known method such as a template matching method. Then, parallax is obtained from the obtained correspondence and the viewpoint position recorded at the time of shooting, and an integrated object shape on arbitrary coordinates can be obtained using the principle of triangulation.

When a correspondence between a plurality of image data is obtained, a correspondence at a luminance level of a pixel is used in a template matching method or the like. The reflection of light on the surface of the subject during photographing is performed by the light source 2 that irradiates the subject with light and the viewpoint position O.
When the positional relationship between L and OR is as shown in FIG.
It can be represented by the following equation model.

I = I0 · Ka · cos θ + I0 · Kb · (cosφ) ^Kc + C (1) where Ka is the diffuse reflection coefficient, Kb is the specular reflection coefficient, and K
c is a constant representing the spread of specular reflection due to surface roughness, C is contribution by environmental light, I0 is the luminance of the light source, and I is the luminance detected after reflection. In FIG. 9, N is the surface normal direction, θ is the angle between the light source direction and the normal direction, and φ is the angle between the light source reflection direction and the viewpoint direction.

From the above equation, the diffused light is determined by the angle θ between the light source direction and the subject surface, and does not depend on the angle φ between the viewpoint and the reflection direction. Therefore, when the surface of the subject is a uniform diffuse reflection surface (Kb ≒ 0), it can be considered that the luminance level of an arbitrary surface with respect to the same light source is substantially equal regardless of the viewpoint position. It is possible to correctly determine the correspondence between the two.

[0008]

However, when the object surface has a surface other than the diffuse reflection surface, that is, when the luminance of the same surface greatly changes with respect to the viewpoint, a method of using the luminance level of the image data as it is. Could not find the correct correspondence. That is, in the conventional method of obtaining the shape of the subject by extracting the corresponding points, the accuracy of the method depends on the surface characteristics of the subject, and the subjects that can be determined are limited.

When the three-dimensional shape information of a subject is used, texture information of the subject may be actually required, and if the surface color, reflectance, etc. are unknown, the data is insufficient. .

The present invention has been made in view of such a situation, and a three-dimensional information detecting method and a three-dimensional information detecting method capable of detecting three-dimensional information including three-dimensional shape information of a subject and texture information of the subject from a plurality of image data. It is intended to provide a device.

[0011]

According to a first aspect of the present invention, there is provided a method for detecting three-dimensional information, comprising: obtaining image data obtained by photographing a subject illuminated by a light source from a plurality of viewpoint positions by a photographing device; A stereoscopic information detection method for detecting stereoscopic information of the subject from the image data, wherein the obtained image data is divided into a plurality of small areas, and light source information regarding the light source and the plurality of viewpoints are provided for each of the plurality of viewpoint positions. Using the photographing posture information of the photographing device at each position, the stereoscopic information of each of the plurality of small regions is detected, and the stereoscopic information of each of the plurality of small regions detected for each of the plurality of viewpoint positions is arbitrarily determined. The three-dimensional information of the subject is detected by integrating the coordinates on a coordinate system.

According to a second aspect of the present invention, in the three-dimensional information detection method according to the first aspect, the three-dimensional information comprises three-dimensional shape information of the subject and texture information of a surface of the subject. The information detection method obtains three-dimensional shape information of the subject based on the shooting attitude information of the shooting device at the plurality of viewpoint positions and the obtained image data, and obtains the three-dimensional shape information of the subject and light source information regarding the light source. Texture information of the surface of the subject is obtained based on the obtained information, and the obtained three-dimensional shape information and the obtained texture information are integrated as three-dimensional information in an arbitrary coordinate system.

According to a third aspect of the present invention, in the three-dimensional information detecting method according to the first or second aspect, based on the obtained image data, the photographing posture information of the photographing device at each viewpoint position is obtained. Is detected.

According to a third aspect of the present invention, in the three-dimensional information detecting method according to the third aspect, the step of detecting the photographing posture information of the photographing apparatus at each of the viewpoint positions includes the step of: The method further includes a step of photographing the drawn sample together with the subject.

According to a fifth aspect of the present invention, in the three-dimensional information detecting method according to any one of the first to fourth aspects, the light source information of the light source is detected based on the obtained image data. It is characterized by including a step.

According to a third aspect of the present invention, in the three-dimensional information detecting method according to the fifth aspect, in the step of detecting the light source information of the light source, the light source information includes:
It is characterized by being detected based on image data obtained by photographing a sample having a plurality of regions with known optical characteristics together with a subject.

According to a third aspect of the present invention, in the three-dimensional information detecting method according to the second aspect, the texture information includes a reflectivity of a surface of the subject, and the three-dimensional information detecting method includes the reflectivity. Correcting the obtained three-dimensional shape information of the subject using

According to a third aspect of the present invention, there is provided a three-dimensional information detecting method according to the seventh aspect, further comprising the step of correcting the reflectance using the corrected three-dimensional shape information. I do.

According to a ninth aspect of the present invention, there is provided a three-dimensional information detecting device, comprising: input means for inputting image data obtained by photographing a subject illuminated by a light source by photographing devices provided at a plurality of viewpoint positions; The divided image data is divided into a plurality of small areas, and for each of the plurality of viewpoint positions, stereoscopic information of each of the plurality of small areas is obtained by using light source information regarding the light source and shooting attitude information of the plurality of shooting devices. And stereoscopic information detecting means for detecting stereoscopic information of the subject by integrating stereoscopic information of each of the plurality of small areas detected for each of the plurality of viewpoint positions on an arbitrary coordinate system. It is characterized by.

According to a tenth aspect of the present invention, in the three-dimensional information detecting apparatus according to the ninth aspect, the three-dimensional information includes three-dimensional shape information of the subject and texture information of a surface of the subject. The information detecting means obtains three-dimensional shape information of the subject based on the shooting attitude information of the shooting device at the plurality of viewpoint positions and the input image data, and obtains the three-dimensional shape information of the subject and light source information regarding the light source. Texture information on the surface of the object based on the obtained three-dimensional shape information and the obtained texture information are integrated as three-dimensional information in an arbitrary coordinate system to detect the three-dimensional information. It is characterized by the following.

According to an eleventh aspect of the present invention, there is provided the three-dimensional information detecting apparatus according to the ninth or tenth aspect, wherein the three-dimensional information detecting apparatus captures the image at each of the viewpoint positions based on the image data obtained by the capturing apparatus. It is characterized by comprising photographing posture information detecting means for detecting posture information.

According to a twelfth aspect of the present invention, in the three-dimensional information detecting apparatus according to the eleventh aspect, the photographing posture information detecting means stores photographing posture information of the photographing device at each viewpoint position on a surface. The apparatus is characterized in that it is configured to detect a specimen on which a known pattern is drawn by photographing the specimen together with the subject.

According to a thirteenth aspect of the present invention, in the three-dimensional information detecting device according to any one of the ninth to twelfth aspects, the light source information of the light source is based on image data obtained by the photographing device. Is provided with light source information detecting means for detecting

According to a fourteenth aspect of the present invention, in the three-dimensional information detecting apparatus according to the thirteenth aspect, the light source information detecting means has a plurality of regions whose optical characteristics are known. It is characterized in that detection is performed based on image data obtained by photographing a sample together with a subject.

According to a three-dimensional information detecting device of the fifteenth aspect, in the three-dimensional information detecting device according to the tenth aspect, the texture information includes a reflectance of a surface of the subject, and the three-dimensional information detecting device includes: A stereoscopic shape information correcting unit configured to correct the obtained stereoscopic shape information of the subject using a ratio.

According to a sixteenth aspect of the present invention, in the three-dimensional information detection apparatus according to the fifteenth aspect, the three-dimensional information detection apparatus further includes a reflectance correction unit that corrects the reflectance using the corrected three-dimensional shape information. It is characterized by.

[0027]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) First, a first embodiment of the present invention will be described with reference to FIGS.

FIG. 1 is a block diagram showing a schematic configuration of a photographing system employing the stereoscopic information detecting method according to the present embodiment.

In FIG. 1, a subject 1 having a triangular pyramid shape is in a state of being uniformly illuminated by a light source 2 which is sufficiently far away, and cameras 3a, 3b,
3c. Cameras 3a, 3b, 3c
Are input to the stereoscopic information detecting device 4 and processed.

The three-dimensional information detecting device 4 includes cameras 3a, 3
b, a video signal unit 5 for converting a video signal input from 3c into an arbitrary image format, a stereoscopic information detecting unit 8 for detecting stereoscopic information of a subject from a plurality of image data, and control of each unit in the apparatus. CPU 9, a memory 6 for storing image data, and a recording unit 7 for recording information such as detected stereoscopic information.

The video signals input from the cameras 3a, 3b and 3c are converted into an image format which can be easily handled by the three-dimensional information detector 8 in the video signal unit 5, for example, 10-bit gray scale bitmap data. This conversion converts the RGB signals from the cameras 3a, 3b, 3c into 1:
The addition is performed at a ratio of 2: 1, and the result is written into the memory 6 as one image data for each viewpoint by the CPU 9. In addition, the CPU 9 stores the subject position information and the camera 3 previously recorded in the recording unit 7.
Camera parameters including information on the viewpoint positions a, 3b, and 3c, information on the focal length, the angle of view, and the like, and information on the position and characteristics of the light source 2 are written into the memory 6 together with the image data. The image data and information written in the memory 6 are used for processing described later.

The three-dimensional information detector 8 performs a process for detecting three-dimensional information comprising the three-dimensional shape information of the subject 1 and the characteristics (texture information) of the subject surface.

FIG. 2 is a flowchart showing the procedure of the three-dimensional information detecting process performed by the three-dimensional information detector 8 for detecting three-dimensional information including the three-dimensional shape information of the subject 1 and texture information such as the characteristics of the subject surface. Note that a program for executing this flowchart is stored in a memory (not shown) and executed by the CPU 9.

First, in step S1, it is determined whether or not processing has been completed for all image data. At first, since this answer is negative (NO), the process proceeds to step S2, and arbitrary two viewpoints OL (XL, YL, ZL), O
R (XR, YR, ZR) is selected, and corresponding point extraction processing is performed to extract corresponding points between image data captured from one viewpoint position and image data captured from the other viewpoint position.

Assuming a certain size of the subject, the two viewpoint positions selected in step S2 are known, so that the overlapping portion of the subject region in the two pieces of image data is determined by the camera parameters and the subject / viewpoint. It can be calculated geometrically based on the position information. In order to reduce the processing load, the corresponding point extraction processing is performed only on this overlapping area. In the present embodiment, a template matching method is used as a corresponding point extraction processing method.

FIGS. 3A and 3B are explanatory diagrams for explaining the template matching method. FIG. 3A shows image data of the subject 1 photographed from the left viewpoint position, and FIG. 3B shows the photograph data from the right viewpoint position. This is the image data of the subject 1 that has been performed. As shown in the figure, the template matching method
An area (template) of an arbitrary size including a target image in one image data is set, and an image included in this template (hereinafter, referred to as a template image)
This is a method of searching for the position of the image data 11 in the other image data, and using the area having the highest matching degree as the corresponding point.

As the image data used for the corresponding point extraction processing, the case where the luminance data at the time of photographing is used as it is,
Image data that has been subjected to a filtering process or the like for edge detection may be used. In any case, since the luminance distribution is used, the specular reflection in the above equation (1) depends on the surface state of the subject 1. , Which causes large errors. Since it is difficult to perform a process in consideration of the surface state of the subject 1, in the corresponding point extraction process in step S2, the subject surface is assumed to perform uniform diffuse reflection, and a response including an error due to a specular reflection component is performed. Points are required.

When the corresponding point extraction processing is completed, step S
Proceeding to 3, the three-dimensional shape information of the subject is obtained by using the principle of triangulation based on the result of the corresponding point extraction processing, the relative relationship between the viewpoint positions, and the camera parameters at the time of shooting. The obtained three-dimensional shape information is moved to a predetermined coordinate system in order to unify the three-dimensional information described later.

Further, in step S3, a process of dividing the obtained three-dimensional shape information into small areas (planes) of an arbitrary size is performed. As described above, here, the corresponding point extraction processing is performed assuming that the surface of the subject has only a diffuse reflection component. As can be seen from equation (1), since the specular reflection component rapidly decreases as the viewpoint position deviates from the reflection direction of the light rays, when the subject is viewed from one visual field position, the surface other than the surface having a specific normal direction is observed. Has a small specular reflection component,
Even if ignored, the error is small. Therefore, when only the diffuse reflection is considered, the equation (1) can be approximated as follows.

I = I0 · Ka · cos θ + C (2) If the material in the region does not change, I depends only on the angle θ between the normal direction of the surface and the light source direction. Therefore, by performing small area division under the condition that the change in luminance is within a certain value,
The inside of a small region can be treated as one surface in which the normal directions of the planes included therein are almost equal, and one diffuse reflectance can correspond to one small region. As an actual division method, I is normalized in one image data, taking into account the previously obtained three-dimensional shape information, and division is performed so that its variation is equal to or less than a predetermined value or the area is equal to or less than a predetermined value. Done.

When the process in step S3 is completed, the process proceeds to step S4, where the positional relationship between the small area divided in step S3 and the light source 2 is obtained. As described above, the information on the light source direction is detected in advance and stored in the memory 6, and by placing this information in the above-described arbitrary coordinate system, it is created by the light from the light source 2 and the surface included in the small area. The angles θ and φ can be calculated.

After calculating the angles θ and φ in step S4, in step S5, a region where the luminance difference is large in the input image data is detected. This is because this portion is considered to have an erroneous response due to the specular reflection component. That is, from equation (1), since the specular reflection component depends on (cos φ) ^Kc , only a region having a specific normal direction where cos φ is close to 1 from one viewpoint is compared with the surrounding region. When the image data is divided into small areas of arbitrary size,
It is highly probable that the area where the luminance difference from the adjacent area is large is an area affected by the specular reflection component other than the difference in the diffuse reflection component due to the difference in surface color and material. In step S5, such an area is detected. By setting the size of this area larger than the small area set in step S3, it is possible to cope with the result of corresponding point extraction described later.

In step S6, by integrating the results of steps S3 to S5, an error region having a large specular reflection component and a large amount of change in three-dimensional shape information is extracted, and a flag indicating the error region is set. Will be added. According to the equation (1), since the specular reflection component of an arbitrary region greatly differs depending on the viewpoint, only one of the two image data used in the corresponding point extraction has high brightness for the same small region. . When corresponding points are extracted from such an area, the area becomes an error and an impulse concave or convex shape is shown. Therefore, step S3
In the three-dimensional shape information obtained by the above, in a region where the amount of change in the three-dimensional shape information is large, there is a high possibility that an erroneous response due to specular reflection has occurred. In order to exclude the case where the shape is actually concave or convex, the results of steps S3 to S5 are integrated, and an area where the change in luminance is large and the shape change is large compared to the surroundings is detected.

In step S6, if the angle φ calculated in step S4 is smaller than the set angle, the specular reflection component cannot be ignored in the area, and the area is determined to be an error area. Thereby, the accuracy of the required three-dimensional shape information of the subject 1 can be improved.

After the addition of the flag indicating the error area, in step S7, for the area excluding the area to which the flag has been added in step S6, for each small area of an arbitrary size divided in step S3, equation (2) Then, the diffuse reflection coefficient Ka of the surface is obtained using the angle θ obtained in step S4. In the equation (2), the observed I includes an error such as ambient light, but in a small area, C is treated as having a constant value, so that I0 · Ka in the equation (2) is calculated. You can ask. This process is performed on all the small areas to which no flag is added.

In step S8, processing is performed on the small area to which the flag has been added in step S6. That is, as is apparent from the equation (1), the specular reflection component rapidly increases at a certain viewpoint position, and the area where the flag is added is an area where the influence of the specular reflection component is small and the step S7 There is a region in which the normal direction and the diffuse reflection coefficient are known. Therefore, the diffuse reflection coefficient of the small area is obtained by interpolating the shape of the error area using the result of the surrounding area. Here, the normalized luminance I, the luminance I0 of the light source 2 and the diffuse reflection coefficient Ka are known. In addition to this, the specular reflection coefficient Kb and the constant Kc are obtained by using the angles θ and φ using the normal direction of the surface obtained by interpolation. Assuming that the influence of ambient light is the same in one small area, and that only the angle φ is different between the pixels corresponding to the area, if the difference of equation (1) is obtained for each pixel, By using a method such as the least square method, the specular reflection Kb and the constant Kc in the small area can be obtained. The normal direction and Kb and Kc are determined by the processing for such a region having a large specular reflection component.

In step S9, the three-dimensional shape information obtained by extracting the corresponding points is corrected using the coefficients and the normal direction obtained by the above-described processing. The three-dimensional shape information obtained by the corresponding point extraction in step S3 assumes that the surface of the subject performs only diffuse reflection. The information obtained in consideration of the specular reflection component is (1)
In addition to Ka, Kb, and Kc in the equation, the normal direction of the plane included in the small area is corrected. The distance information of the subject 1 is corrected such that the corrected normal direction matches the distance information of the subject 1 without inconsistency. In addition, the position information and the reflection coefficient of the small area on arbitrary coordinates are added to the point indicating the distance information of the subject so that the reflection coefficient on each surface can be handled together with the distance information.

The processes from step S2 to step S9 are performed on image data taken from two different viewpoints, and one subject information (three-dimensional information) is obtained by this process.

In step S10, the subject information obtained from the image data of an arbitrary combination processed so far is merged with the subject information obtained this time. That is, the subject information of the same region obtained from different combinations and image data does not completely match due to a difference at the time of shooting, a disturbance, or an error in processing. Therefore, the entire subject information is corrected using the difference indicated by the overlapping region. Is performed, and one piece of subject information is created in an arbitrary coordinate system.

When the above-described image processing has been completed for all the image data, the answer to step S1 is affirmative (YES), and the three-dimensional shape information of the subject 1 and the texture information such as the reflectance, etc., constructed by the processing so far. Are integrated in the recording unit 7 in an arbitrary format, and the processing procedure ends.

As described above, according to the present embodiment, based on image data captured from a plurality of viewpoint positions, three-dimensional shape information and texture information such as reflectance on an arbitrary coordinate system can be obtained. 3D information can be obtained with high accuracy.

Further, since the three-dimensional shape information is corrected by the reflectance, an error generated between the three-dimensional shape information of the subject 1 and the reflectance is reduced, and the three-dimensional information can be obtained more accurately.

In this embodiment, the input image data is converted to gray scale. However, after the three-dimensional information is obtained, the correspondence between the three-dimensional shape information and the pixels in the image data is determined. It is also possible to add color information of the subject surface. In this case, the final three-dimensional information is
One point is composed of position information in an arbitrary coordinate system and texture information including the color, pattern, and reflectance at that point. That is, by integrating the three-dimensional shape information of a plurality of regions from each viewpoint and texture information such as color and reflectivity on an arbitrary coordinate system, three-dimensional information in various formats can be efficiently created in a unified format, and , The three-dimensional information can be easily handled.

(Second Embodiment) Next, a second embodiment of the present invention will be described.
The embodiment will be described with reference to FIGS. In the present embodiment, a description will be given of three-dimensional information detection processing when light source information including position information of the light source 2 and the like is unknown.

FIG. 4 is a block diagram showing a schematic configuration of a photographing system employing the stereoscopic information detecting method according to the present embodiment. In the figure, the same components as those of the above-described first embodiment shown in FIG. 1 are denoted by the same reference numerals.

A pad 12 having a known shape is arranged near the subject 1, and a plurality of cameras 3a, 3b,
The positional relationship between 3c, the pad 12, and the subject 1 is known. Also, camera parameters at the time of shooting are known.

An arbitrary pattern is drawn on the pad 12 at a plurality of places, and this pattern is divided into a region having only a diffuse reflection component and a region having a diffuse reflection component and a specular reflection component. The diffuse reflection coefficient and the specular reflection coefficient of each region differ depending on the position on the pad 12, and the relationship between the pattern position and the diffuse reflection coefficient and the specular reflection coefficient is known. The light source 2 is arranged at a position sufficiently distant from the subject 1, and illuminates the subject 1 with uniform light. Under such circumstances, three-dimensional information of the subject 1 is detected from image data taken from a plurality of preset viewpoints.

FIG. 5 is a flowchart showing a procedure for detecting stereoscopic information in the present embodiment.

First, in step S21, it is determined whether or not processing has been completed for all image data.
At first, since this answer is negative (NO), step S2
Proceeding to 2, pad area detection processing for detecting the area occupied in the image data of the pad 12 photographed together with the subject 1 is performed. Although the photographed image data includes the pad 12 in addition to the subject 1, the stereoscopic information detection process of the subject 1 does not require the image data of the pad 12, and the pad 12 is used for processes other than the light source information detection process described later. This is because the image data from which the 12 parts of the image data have been removed is used.
Since the subject position information, the position information of the pad 12 and the viewpoint position of each image data are known, it is possible to calculate the pad position in the image data using these parameters. The pad area can be recognized from the image data by using the pattern of. Specifically, since the relative positional relationship between the pad 12 and the viewpoint position is known, a known pattern obtained by modifying the pattern on the surface of the pad 12 according to the appearance from the viewpoint direction as shown in FIG. By performing the matching process as an image, the area of the pad 12 is recognized.

When the pad area detection process in the image data is completed, in step S23, a matching process is performed using the known pattern on the pad 12 as a template, so that only the diffuse reflection component is detected from the pad region recognized in step S22. The region and the region containing the specular reflection component are recognized. Then, in step S24, the light source characteristics are calculated. Specifically, first, a plurality of regions including only diffuse reflection components are extracted, and a relative luminance difference of reflected light in each region is obtained. Then, assuming that the light source 2 and the ambient light are uniform on the pad 12, a difference between the values of the respective regions calculated using the above-described equation (2) is obtained. The angle cos θ between the normal direction of the light source and the light source direction and the luminance I0 of the light source 2 are calculated.

Further, in order to determine the direction of the light source, information from an area having a specular reflection component on the pad 12 is used to determine the angle φ. This can be obtained from the relative luminance difference between a plurality of regions, as in the case where cos θ is calculated. However, when determining the light source direction using only image data obtained from one viewpoint position, when the influence of the specular reflection component is small, for example, the light source direction and an arbitrary viewpoint position are viewed from the pad 12 as shown in FIG. If they are in the same direction, it is difficult to specify the light source direction. Therefore,
In specifying the light source direction using the pad 12 in step S24, results obtained from a plurality of viewpoint positions are compared, and data at the viewpoint position where the specular reflection component is the largest is adopted.

When the calculation of the light source direction and the brightness of the light source 2 as the characteristics of the light source 2 thus obtained is completed, the process proceeds to step S25. Hereinafter, the processing procedure from step S25 to step S34 is the same as the processing procedure from step S2 to step S11 shown in FIG. 2 of the first embodiment described above.

As described above, according to the present embodiment, a known pattern is drawn, and the characteristics of the light source 2 are not known by using the pad 12 whose optical characteristics are known in advance. However, three-dimensional information including the three-dimensional shape information of the subject 1 and the texture information of the surface of the subject 1 can be obtained.

When the position of the light source 2 is close to the subject 1 and a uniform light beam does not hit the subject, the following measures can be taken. That is, according to the equation (2), when the light source 2 is close, the luminance of the reflected light is different in a plurality of regions having the diffuse reflection component on the pad 12 because the incident angles θ to the pad 12 are different. Therefore, different light source directions θ are calculated in different regions. Therefore, the point at which a plurality of light beams from the light source 2 intersect the pad 12 and the angle of the light beam at that point are known, so that the intersection of the light beams, that is, the direction and position of the light source 2 can be obtained geometrically.

As described above, in this embodiment, the pad 12 having a known surface pattern is used to obtain light source information. It is also possible to substitute a plurality of diffuse reflection surfaces (areas) having a reflection coefficient. In this case, it is necessary to specify an area for acquiring light source information in the image data, but there is an advantage that the degree of freedom in the shooting direction is large.

(Third Embodiment) Next, a third embodiment of the present invention will be described.
The embodiment will be described with reference to FIG. In the present embodiment, a method for detecting stereoscopic information based on a plurality of image data captured from an unspecified viewpoint position will be described.

FIG. 7 is a block diagram showing a schematic configuration of a photographing system employing the stereoscopic information detecting method according to the present embodiment. In the figure, the same components as those of the above-described first embodiment shown in FIG. 1 are denoted by the same reference numerals.

The subject 1 is arranged on a pad 12 whose shape for obtaining the viewpoint position and the light source information is known, and the subject 1 including the pad 12 is photographed by one compound-eye camera 3d. Camera parameters such as the focal length of the compound eye camera 3d, the relative positional relationship between the lens barrels, and the optical axis direction are known. Also, on the surface on the pad 12,
As in the second embodiment described above, an arbitrary pattern whose shape and optical characteristics are known is drawn. The light source 2 is arranged at a position sufficiently distant from the subject 1 and illuminates the subject 1 with uniform light. In this situation, moving the camera 3d allows the subject 1 to be viewed from a plurality of viewpoints.
2 is photographed, and three-dimensional information such as viewpoint position information, three-dimensional shape information, and reflectance is detected from the image data.

In this embodiment, the processing for detecting the three-dimensional shape of the subject 1 and the processing for detecting the light source information are the same as those in the first and second embodiments. The viewpoint position detection procedure will be described.

When the subject 1 is photographed by the camera 3d whose relative positional relationship between the left and right lens barrels is known, two images photographed by the camera 3d are performed using the corresponding point extraction procedure described in the first embodiment. If the correspondence between the pixels of the data can be obtained, the relative positional relationship between the position of the subject corresponding to the pixel and one viewpoint position can be obtained on arbitrary coordinates by using the principle of triangulation. here,
Pad information in the image data is used to associate the relative positional relationship between the two viewpoints. As in the case of obtaining the light source direction in the second embodiment, the known pattern drawn on the pad changes depending on the viewpoint position. By detecting the change in the pattern, the image data is transferred to the pad 12. On the other hand, it is possible to detect from which angle the image data is viewed, that is, the relative positional relationship between the viewpoint position and the subject position.

On the pad 12 used in the present embodiment,
Since there is a region having only the diffuse reflection component, if the corresponding point is extracted and the viewpoint position is detected for this region, a stable viewpoint position detection can be performed irrespective of the position of the light source 2.

Based on the viewpoint position detected in this way, it is possible to obtain three-dimensional information including texture information such as three-dimensional shape information and reflectance information of the subject by the above-described method.

As described above, according to the present embodiment, by using the pad 12 and the compound-eye camera 3d, the subject can be obtained from image data taken from an arbitrary viewpoint without fixing the viewpoint at the time of photographing. 3D information including the 3D shape information and the texture information can be detected.

[0075]

As described above, according to the three-dimensional information detecting method of the first aspect or the three-dimensional information detecting apparatus of the ninth aspect,
The obtained image data is divided into a plurality of small areas, and for each of the plurality of viewpoint positions, light source information regarding the light source and shooting posture information of the imaging device at each of the plurality of viewpoint positions are used. The three-dimensional information of each of the small areas is detected, and the three-dimensional information of each of the plurality of small areas detected for each of the plurality of viewpoint positions is integrated on an arbitrary coordinate system to detect the three-dimensional information of the subject. Therefore, an effect is obtained that stereoscopic information of a subject in an arbitrary coordinate shape can be obtained with high accuracy.

The three-dimensional information detecting method according to claim 2 or claim 1
According to the three-dimensional information detecting device, the three-dimensional information is composed of the three-dimensional shape information of the subject and the texture information of the surface of the subject, and the photographing posture information of the photographing device at the plurality of viewpoint positions and the obtained three-dimensional information are obtained. The three-dimensional shape information of the subject is obtained based on the obtained image data, and the texture information of the surface of the subject is obtained based on the three-dimensional shape information of the subject and the light source information on the light source. Since the obtained texture information is integrated as three-dimensional information in an arbitrary coordinate system, an effect that three-dimensional information including three-dimensional shape information and texture information in an arbitrary coordinate system can be obtained with high accuracy can be obtained. .

The three-dimensional information detecting method according to claim 3 or claim 1
According to the three-dimensional information detecting device, the photographing posture information of the photographing device at each of the viewpoint positions is detected based on the obtained image data. Thus, it is possible to obtain the effect that stereoscopic information including the stereoscopic shape information and the texture information of the subject can be obtained from the image data captured from the viewpoint position.

The three-dimensional information detecting method according to claim 4 or claim 1
According to the three-dimensional information detecting device, the photographing posture information of the photographing device at each viewpoint position is detected by photographing a sample with a known pattern drawn on the surface together with the subject, so that the photographing posture information can be obtained at the time of photographing. The effect is obtained that the stereoscopic information including the stereoscopic shape information and the texture information of the subject can be obtained from the image data photographed from an arbitrary viewpoint position without fixing the starting point position and with a simple configuration.

The three-dimensional information detecting method according to claim 5 or claim 1
According to the three-dimensional information detecting device, the light source information of the light source is detected based on the obtained image data. Therefore, even when the characteristics of the light source are not known, the three-dimensional shape information of the subject and The effect is obtained that three-dimensional information consisting of texture information can be obtained.

The three-dimensional information detection method according to claim 6 or claim 1
According to the three-dimensional information detecting device of No. 4, light source information is detected based on image data obtained by photographing a sample having a plurality of regions with known optical characteristics together with a subject, so that the light source information Even in the case where the characteristics are not known, the effect is obtained that, with a simple configuration, it is possible to obtain the three-dimensional information including the three-dimensional shape information and the texture information of the subject.

The three-dimensional information detecting method according to claim 7 or claim 1
According to the three-dimensional information detecting device of No. 5, since the obtained three-dimensional shape information of the subject is corrected using the reflectance included in the texture information, an error generated between the shape of the subject and the reflectance is reduced. The effect is that the stereoscopic information can be obtained with high accuracy.

The three-dimensional information detecting method according to claim 8 or claim 1
According to the three-dimensional information detecting device of No. 6, since the reflectance is corrected using the corrected three-dimensional shape information, an error occurring between the shape of the subject and the reflectance is reduced, and the three-dimensional information is accurately detected. Can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a schematic configuration of an imaging system that employs a stereoscopic information detection method according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing a stereoscopic information detection processing procedure for detecting stereoscopic information including texture information such as the three-dimensional shape information of the subject and the characteristics of the surface of the subject in the embodiment;

FIG. 3 is an explanatory diagram for explaining a template matching method.

FIG. 4 is a block diagram illustrating a schematic configuration of an imaging system that employs a stereoscopic information detection method according to a second embodiment of the present invention.

FIG. 5 is a flowchart showing a stereoscopic information detection procedure in the embodiment.

FIG. 6 is an explanatory diagram for explaining a method of specifying a light source direction when the influence of a specular reflection component is small.

FIG. 7 is a block diagram illustrating a schematic configuration of an imaging system that employs a three-dimensional information detection method according to a third embodiment of the present invention.

FIG. 8 is an explanatory diagram for describing a conventional stereoscopic information detection method.

FIG. 9 is an explanatory diagram illustrating a reflection state of light rays on a subject surface.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Subject 2 Light source 3 Camera 7 Recording part 8 Three-dimensional information extraction part 12 Pad

Claims (16)

[Claims] 1. A three-dimensional information detection method for detecting three-dimensional information of a subject illuminated by a light source from image data obtained by photographing the subject from a plurality of viewpoint positions with a photographing device, wherein the obtained image is Dividing the data into a plurality of small areas, for each of the plurality of viewpoint positions, using light source information regarding the light source and shooting attitude information of the imaging apparatus at each of the plurality of viewpoint positions, each of the plurality of small areas Detecting a three-dimensional information of the subject by integrating the three-dimensional information of each of the plurality of small areas detected for each of the plurality of viewpoint positions on an arbitrary coordinate system; Information detection method. 2. The three-dimensional information includes three-dimensional shape information of the subject and texture information of a surface of the subject, and the three-dimensional information detecting method includes: acquiring the photographing posture information of the photographing device at the plurality of viewpoint positions; Determining three-dimensional shape information of the subject based on the obtained image data; determining texture information of the surface of the subject based on the three-dimensional shape information of the subject and light source information regarding the light source; The three-dimensional information detecting method according to claim 1, wherein the obtained texture information is integrated as three-dimensional information in an arbitrary coordinate system. 3. The method according to claim 1, further comprising the step of detecting photographing attitude information of the photographing device at each of the viewpoint positions based on the obtained image data.
The stereoscopic information detection method according to 1. 4. The method according to claim 3, wherein the step of detecting photographing posture information of the photographing apparatus at each viewpoint position includes a step of photographing a sample having a known pattern drawn on a surface thereof together with the subject. The stereoscopic information detection method according to 1. 5. The stereoscopic information detecting method according to claim 1, further comprising the step of detecting light source information of the light source based on the obtained image data. 6. In the step of detecting light source information of the light source, the light source information is detected based on image data obtained by photographing a sample having a plurality of regions whose optical characteristics are known together with a subject. The three-dimensional information detection method according to claim 5, wherein 7. The texture information includes a reflectance of a surface of the subject, and the three-dimensional information detecting method includes a step of correcting the determined three-dimensional shape information of the subject using the reflectance. The three-dimensional information detection method according to claim 2, wherein: 8. Using the corrected three-dimensional shape information,
The stereoscopic information detection method according to claim 7, further comprising a step of correcting the reflectance. 9. An input means for inputting image data obtained by photographing a subject illuminated by a light source by photographing devices provided at a plurality of viewpoint positions; Divided into regions, for each of the plurality of viewpoint positions, using light source information relating to the light source and photographing attitude information of the plurality of photographing devices, detecting three-dimensional information of each of the plurality of small regions; Stereoscopic information detecting device, comprising: stereoscopic information detecting means for detecting stereoscopic information of the subject by integrating stereoscopic information of each of the plurality of small areas detected for each position on an arbitrary coordinate system. . 10. The three-dimensional information includes three-dimensional shape information of the subject and texture information of a surface of the subject, and the three-dimensional information detecting means includes: shooting orientation information of the shooting device at the plurality of viewpoint positions; The three-dimensional shape information of the subject is obtained based on the obtained image data, and the texture information of the surface of the subject is obtained based on the three-dimensional shape information of the subject and the light source information on the light source. The three-dimensional information detecting apparatus according to claim 9, wherein the three-dimensional information is detected by integrating the obtained texture information as three-dimensional information in an arbitrary coordinate system. 11. A photographing posture information detecting means for detecting photographing posture information of the photographing device at each of the viewpoint positions based on image data obtained by the photographing device. The three-dimensional information detection device according to the above. 12. The photographing posture information detecting means is configured to detect photographing posture information of the photographing device at each of the viewpoint positions by photographing a sample having a known pattern drawn on a surface thereof together with the subject. The three-dimensional information detection device according to claim 11, wherein 13. The three-dimensional object according to claim 9, further comprising light source information detecting means for detecting light source information of the light source based on image data obtained by the photographing device. Information detection device. 14. The light source information detecting means is configured to detect the light source information based on image data obtained by photographing a sample having a plurality of regions having known optical characteristics together with a subject. 14. The three-dimensional information detection device according to claim 13, wherein: 15. The texture information includes a reflectance of a surface of the subject, and the three-dimensional information detection device corrects the obtained three-dimensional shape information of the subject using the reflectance. The three-dimensional information detection device according to claim 10, further comprising a unit. 16. The three-dimensional information detecting apparatus according to claim 15, further comprising: a reflectance correcting unit configured to correct the reflectance using the corrected three-dimensional shape information.