JP4091455B2 - Three-dimensional shape measuring method, three-dimensional shape measuring apparatus, processing program therefor, and recording medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a three-dimensional shape measuring apparatus for measuring a three-dimensional shape of an object to be measured, and more particularly to a technique effective when applied to a three-dimensional shape measuring apparatus used in the fields of measurement industry, communication industry, video industry and the like. It is.
[0002]
[Prior art]
In recent years, 3D images have been handled by portable devices and the like, and 3D shape measurement devices that are small, low cost, low power consumption, and have no restrictions on measurement environment or measurement range are required as input means for 3D images. ing. However, there is no conventional three-dimensional measuring apparatus that can satisfy all of these conditions.
[0003]
For example, an active type three-dimensional measuring apparatus that measures light by irradiating an object to be measured consumes a large amount of power to drive an illuminating light source, has a limited measurement distance, has limited environmental light, for example, In such a case, it cannot be operated. In many cases, these methods use a laser as the irradiation light, but in this case, they are unsuitable for consumer use from the viewpoint of safety.
[0004]
On the other hand, if there is a passive type three-dimensional measurement apparatus that performs measurement without irradiating the object to be measured with special light, the above problem can be solved. However, the three-dimensional measurement methods belonging to the passive type have difficulty in performance and are particularly unreliable, so most of the methods have not yet reached the stage where a practical measurement device can be made.
[0005]
By the way, there is a passive three-dimensional measurement method called Shape from Focus (see, for example, Non-Patent Document 1). In this method, image information captured by a plurality of imaging units having different focal positions is acquired, and a three-dimensional shape of an imaging target is obtained from the image. The principle of this method is that, in an image taken by a photographing means at a certain focal position, an area close to the focal position is clearly visible, while an area far from the focal position is blurred, and distance measurement is performed. I do. Specifically, the spatial frequency spectrum in the vicinity of each pixel in the image is obtained, and the degree of blurring is obtained from the magnitude of the attenuation rate with respect to the frequency of the spatial frequency spectrum intensity, thereby determining the distance of the subject in the pixel. This can be realized by calculating. In this method, the apparatus can be configured only by a camera that can control the focal position, so that the apparatus can be made extremely small, and is highly advantageous in terms of price and power consumption. On the other hand, this method has low measurement accuracy and resolution, but these performances are often sufficient for consumer use.
[0006]
However, the conventional Shape from Focus has the following drawbacks.
1. An area with little change in shading in an image is difficult to measure, and even if it is forcibly measured, there are many measurement errors, and a meaningless distance may be calculated by complete erroneous measurement.
2.1 Since only one image cannot be distinguished whether it is blurred because it is at a long distance, or whether the original material was originally a short-distance subject that appears blurry, the focal position is different to distinguish this. Requires multiple images. In addition to the purpose of improving the resolution, the necessary number of images may be used in order to reduce the measurement error, and several tens of images may be used.
3. Since the spatial frequency spectrum is obtained for all the pixels of a plurality of photographed images, the calculation cost is very high.
[0007]
As a method for overcoming these drawbacks, there is an active type three-dimensional measurement apparatus that measures the blurring method by irradiating the object to be measured with various patterns of laser light (see, for example, Patent Document 1). This can reliably solve the above problems 1 and 2 because the stable measurement can be performed, but a new problem arises in which all the defects of the active type are held.
[0008]
[Non-Patent Document 1]
A.Pentland "A New Sense for Depth of Field", pp.523-531, VOL.PAMI-9, No.4, JULY 1987
[Patent Document 1]
JP 2002-56348 A
[0009]
[Problems to be solved by the invention]
As described above, in the conventional 3D measurement method belonging to the passive type called Shape from Focus, it is difficult to measure the area where the change in shading is difficult in the image, and several images that require several tens of images with different focal positions are required. There is a problem that the calculation cost is very large because the spatial frequency spectrum is obtained for all the pixels of the photographed image. On the other hand, when an active type three-dimensional measuring device is used to overcome these disadvantages, the power consumption However, there is a problem that the measurement distance is limited, the ambient light is limited, and it is not suitable for consumer use in terms of safety.
[0010]
An object of the present invention is to provide a technique capable of solving the above-described problems and obtaining a simple three-dimensional shape from only one image by the Shape from Focus method. Another object of the present invention is a technology capable of obtaining a three-dimensional shape with higher accuracy and a wider measurement range by using the Shape from Focus method from only two images, greatly reducing the calculation cost and shortening the processing time. Technology that can reduce the cost of computation while minimizing performance degradation, low power by eliminating the need for power, and easy focus operation for fast focus positioning It is to provide a technology that can be changed.
[0011]
[Means for Solving the Problems]
In the present invention, the distance between the area and the imaging means is obtained from the spectral intensity attenuation rate of the spatial frequency spectrum in the captured image captured at a predetermined focal position.
[0012]
In the three-dimensional shape measuring apparatus of the present invention, when photographing is performed by the photographing means at a predetermined focal position, the photographed image photographed by the photographing means is input by the image input means, and the luminance change in the photographed image is a predetermined value. The spatial frequency spectrum of the above region is obtained by the spatial frequency spectrum calculation means.
[0013]
Then, after calculating the spectral intensity attenuation rate in the direction in which the attenuation of the spectral intensity with respect to the spatial frequency of the region is minimized by the attenuation rate calculating means, the distance calculation is performed on the distance from the calculated spectral intensity attenuation rate to the imaging means. Obtain by means. That is, since the spectral intensity attenuation rate reflects the distance information from the focal point to the region, after obtaining the distance from the focal point to the region from the spectral intensity attenuation factor, the distance from the focal point position to the imaging means is obtained at the obtained distance. Is added to obtain the distance between the photographing means and the area. For a region where the luminance change in the photographed image is less than the predetermined value, the distance between the subject and the photographing unit is obtained by interpolation from the region where the distance is obtained.
[0014]
As described above, in the three-dimensional shape measuring apparatus of the present invention, the distance between the region and the imaging means is obtained from the spectral intensity attenuation rate of the spatial frequency spectrum in the captured image captured at a predetermined focal position. It is possible to easily measure the outline of the three-dimensional shape to be imaged.
[0015]
In the three-dimensional shape measuring apparatus of the present invention, the distance between the area and the imaging means may be obtained from the spectral intensity attenuation rate of the spatial frequency spectrum in two captured images captured at different focal positions.
[0016]
That is, in the three-dimensional shape measuring apparatus of the present invention, the first input image captured by the first imaging unit having the focal position at a predetermined position is different from the predetermined position by the image input unit. The second photographed image photographed by the second photographing means is input respectively.
[0017]
Next, the spatial frequency spectrum calculation means calculates the spatial frequency spectrum of the photographic image having the larger luminance change and the other photographic image for the region where at least one of the luminance changes in the first and second photographic images is a predetermined value or more. The spatial frequency spectrum of the image is obtained as a first spatial frequency spectrum and a second spatial frequency spectrum, respectively.
[0018]
Then, the attenuation factor calculation means obtains, as the first attenuation factor, an attenuation factor in the direction in which the attenuation of the spectrum intensity with respect to the spatial frequency is minimized for the first spatial frequency spectrum, and for the second spatial frequency spectrum, After obtaining the attenuation factor of the spectral intensity with respect to the spatial frequency in the same direction as the second attenuation factor, the distance calculation means and the subject in the area and the first or second factor based on the first and second attenuation factors. Find the distance to the imaging means. That is, since the ratio between the first and second attenuation factors reflects the distance information from the focal position to the area, the distance from the focal position to the area is obtained from this ratio, and then the focus is set to the obtained distance. The distance between the photographing means and the area is obtained by adding the position and the distance to the photographing means. For a region where the luminance change in the photographed image is less than the predetermined value, the distance between the subject and the photographing unit is obtained by interpolation from the region where the distance is obtained.
[0019]
As described above, in the three-dimensional shape measurement apparatus of the present invention, the distance between the area and the imaging means is obtained from the spectral intensity attenuation rate of the spatial frequency spectrum in two captured images captured at different focal positions. It is possible to easily measure a three-dimensional shape, and has advantages such as being able to calculate a distance more accurately than when using a single photographed image, and measuring a distant distance.
[0020]
Further, in the three-dimensional shape measuring apparatus of the present invention, the area between the region and the image capturing means is calculated based on the spectral intensity attenuation rate of the spatial frequency spectrum in the captured image captured at a predetermined focal position or two captured images captured at different focal positions. When obtaining the distance, the distance may be obtained from the spectral intensity attenuation rate of the one-dimensional spatial frequency spectrum in the normal direction of the boundary where the luminance changes.
[0021]
That is, in the three-dimensional shape measuring apparatus according to the present invention, the image input means is used to capture a photographed image photographed by a photographing means having a focal position at a predetermined position or two photographed images photographed by photographing means having different focal positions. After input, the spectrum calculation means causes the luminance in the area where the luminance change is greater than or equal to a predetermined value in the captured image captured at the predetermined focal position, or in the captured image where the luminance change captured at a different focal position is larger. For a region where the change is greater than or equal to a predetermined value, the normal direction of the boundary of the luminance change and the one-dimensional spatial frequency spectrum in the normal direction are obtained.
[0022]
Next, an attenuation rate calculating means obtains a spectral intensity attenuation rate with respect to the spatial frequency for the one-dimensional spatial frequency spectrum in the normal direction, and based on the obtained spectral intensity attenuation rate, the subject in the area and the imaging means Find the distance. For a region where the luminance change in the photographed image is less than the predetermined value, the distance between the subject and the photographing unit is obtained by interpolation from the region where the distance is obtained.
[0023]
As described above, in the three-dimensional shape measuring apparatus of the present invention, since the distance is obtained from the spectral intensity attenuation rate of the dimensional spatial frequency spectrum in the normal direction of the boundary where the luminance changes, it is not necessary to obtain the two-dimensional spatial frequency spectrum. As a result, the calculation cost can be greatly reduced and the processing time can be shortened.
[0024]
Further, in the three-dimensional shape measuring apparatus of the present invention, the area between the region and the image capturing means is calculated based on the spectral intensity attenuation rate of the spatial frequency spectrum in the captured image captured at a predetermined focal position or two captured images captured at different focal positions. When obtaining the distance, the distance may be obtained from the spectral intensity attenuation rate of the one-dimensional spatial frequency spectrum in the horizontal direction.
[0025]
That is, in the three-dimensional shape measuring apparatus according to the present invention, the image input means is used to capture a photographed image photographed by a photographing means having a focal position at a predetermined position or two photographed images photographed by photographing means having different focal positions. After input, the spectrum calculation means causes the luminance in the area where the luminance change is greater than or equal to a predetermined value in the captured image captured at the predetermined focal position, or in the captured image where the luminance change captured at a different focal position is larger. A horizontal one-dimensional spatial frequency spectrum is obtained for a region where the change is a predetermined value or more.
[0026]
Next, an attenuation rate calculation means obtains an attenuation rate of the spectral intensity with respect to the spatial frequency for the horizontal one-dimensional spatial frequency spectrum, and based on the obtained spectral intensity attenuation rate, the distance between the subject in the region and the imaging means Ask for. For a region where the luminance change in the photographed image is less than the predetermined value, the distance between the subject and the photographing unit is obtained by interpolation from the region where the distance is obtained.
[0027]
As described above, in the three-dimensional shape measuring apparatus of the present invention, since the distance is obtained from the spectral intensity attenuation rate of the horizontal one-dimensional spatial frequency spectrum, there is no need to obtain the two-dimensional spatial frequency spectrum, and the boundary portion where the luminance changes Therefore, the calculation cost can be greatly reduced and the processing time can be shortened.
In the three-dimensional shape measuring apparatus of the present invention, a variable focal position imaging device using a variable optical system may be used.
[0028]
That is, an optical system whose focal position changes when the movable part is driven, a shutter having a two-step pressing depth, and the movable part of the optical system is driven according to the pressed depth of the shutter. When a variable focal position imaging device including a transmission means is used, when the shutter is pushed down to the first depth, shooting is performed at a predetermined focal position, and when the shutter is pushed down to the second depth. Takes a picture at a focal position different from the predetermined focal position by driving the movable part according to the pressed depth of the shutter by the transmission means to move the imaging element and / or the imaging optical system. .
[0029]
As described above, in the three-dimensional shape measuring apparatus of the present invention, by using the focal position variable imaging apparatus using the variable optical system, a dedicated power for moving the focal position is not necessary when performing three-dimensional shape measurement. There are advantages such that variable focus shooting can be performed naturally and at high speed without forcing the photographer to perform special operations.
[0030]
As described above, according to the three-dimensional shape measuring apparatus of the present invention, the spectral intensity attenuation rate of the spatial frequency spectrum in the photographed image photographed at the predetermined focal position or the two photographed images photographed at different focal positions. Since the distance between the area and the photographing means is obtained, a simple three-dimensional shape can be obtained from only a smaller number of images than in the prior art by the Shape from Focus method.
[0031]
DETAILED DESCRIPTION OF THE INVENTION
(Embodiment 1)
The three-dimensional shape measuring apparatus according to the first embodiment for obtaining the distance between the area and the imaging means from the spectral intensity attenuation rate of the spatial frequency spectrum in the captured image captured at a predetermined focal position will be described below.
[0032]
FIG. 1 is a diagram showing a schematic configuration of a three-dimensional shape measuring apparatus according to the present embodiment. As shown in FIG. 1, the three-dimensional shape measuring apparatus of this embodiment includes an image input unit 102, a spatial frequency spectrum calculation unit 103, a spectrum intensity attenuation rate calculation unit 104, a distance calculation unit 105, and an interpolation calculation unit 106. And have.
[0033]
The image input means 102 is a means for inputting a photographed image photographed by the photographing means 101 having a focal position at a predetermined position. The spatial frequency spectrum calculation means 103 is a means for obtaining a spatial frequency spectrum for a region where the luminance change in the inputted photographed image is a predetermined value or more.
[0034]
The spectral intensity attenuation rate calculating means 104 is a means for obtaining an attenuation rate in a direction in which the attenuation of the spectral intensity with respect to the spatial frequency is minimized with respect to the obtained spatial frequency spectrum. The distance calculation means 105 is a means for obtaining the distance between the subject in the area and the photographing means 101 based on the obtained attenuation rate.
[0035]
Interpolation calculating means 106 obtains the distance between the subject and the photographing means 101 by interpolation from the area where the distance is obtained by the distance calculating means 105 for the area where the luminance change in the inputted photographed image is less than the predetermined value. It is.
[0036]
Further, the three-dimensional shape measuring apparatus of the present embodiment can be realized by a computer and a program, and the program can be recorded on a recording medium or provided through a network.
[0037]
FIG. 2 is a diagram showing an outline of processing of the three-dimensional shape measuring apparatus according to this embodiment. When photographing is performed by the photographing unit 101 having a close focal position, the image input unit 102 of the three-dimensional shape measuring apparatus inputs a photographed image 206 photographed by the photographing unit 101 in step 201. In the acquired captured image 206, a region 207 close to the image capturing unit 101 is clearly visible, whereas a region 208 far away is a blurred image.
[0038]
In step 202, the spatial frequency spectrum calculation means 103 obtains a spatial frequency spectrum of a region where the luminance change in the captured image is equal to or greater than a predetermined value.
[0039]
First, contour area extraction data 209 is obtained by extracting an area where the luminance change is equal to or greater than a predetermined set value. Then, since the brightness change of the contour portion of the object is remarkably large compared to other regions, a region including more contour portions can be efficiently extracted by extracting a region where the brightness change is equal to or greater than a predetermined set value.
[0040]
Next, a spatial frequency spectrum is obtained for each point included in this region. For example, when the spatial frequency spectrum of a certain point 210 on the region is obtained, a two-dimensional spatial frequency spectrum shown in the spatial frequency spectrum 211 is obtained. It is assumed that the vertical axis representing the spectral intensity of the spatial frequency spectrum 211 is logarithmically taken.
[0041]
In step 203, the spectral intensity attenuation rate calculating means 104 obtains an attenuation rate in a direction that minimizes the attenuation of the spectral intensity with respect to the spatial frequency of the region extracted in step 202.
[0042]
For example, the attenuation rate of the spatial frequency spectrum 211 is obtained as an average value of the region 212. Since the obtained spatial frequency spectrum 211 is expressed two-dimensionally, the direction in which the attenuation of the spectrum intensity is the smallest is determined and the attenuation rate for obtaining the average value of the gradient in that direction. If the image is clear, the attenuation factor is small, but a blurred image in an out-of-focus area has a larger attenuation factor.
[0043]
The attenuation rate is determined by selecting the direction in which the spectral intensity attenuation with respect to the spatial frequency is the minimum. The attenuation rate of the spectral intensity in this direction is the most stable measurement and the distance in the depth direction of the subject. This is because the information is reflected.
[0044]
In step 204, the distance calculation means 105 obtains the distance from the region and the imaging means 101 from the spectral intensity attenuation rate obtained in step 203. As the object is farther away from the focal position, the image becomes more blurred. If the attenuation rate is -k [dB / Oct (dB per octave)], k reflects the distance information from the focal point to the region, so it is more focused than k. To the area can be obtained. Further, since the distance between the focal position and the photographing means 101 is known, the distance between the photographing means 101 and the region can be obtained by adding the distance between the focal position and the photographing means 101 to the obtained distance.
[0045]
By the way, k is the focal position of the lens itself, the size of the stop, and the MTF (Modulation Transfer Function) of the lens. It is an abbreviation for the amplitude transfer function. However, it is difficult to analytically remove all these effects. Therefore, paying attention to the fact that these influence factors do not depend on the subject and are determined only by the imaging optical system of the photographing means, using the photographing means in advance, a certain subject (for example, the right The plane of half is black, the left half is white, etc.) is taken at different distances, and the attenuation rate → distance reference table 213 is created in advance by measuring the distance and the spectral intensity attenuation rate each time. . Since this reference table has already incorporated all the effects of the imaging optical system of the imaging means, at the time of subsequent measurement, the distance of the imaging optical system of the imaging means is read by referring to the reference table from the obtained attenuation rate. Only the distance can be obtained by removing the influence.
[0046]
In step 205, the interpolation calculation means 106 generates a distance by interpolating the area for which the distance has not been obtained in the processing up to step 204 from the neighboring area for which the distance calculation means 105 has obtained the distance. For example, the depth distance z of the pixel 214 in the area where the distance cannot be obtained is the distance i to the pixel 214 with respect to the pixel 215 of the i point where the distance in the vicinity is obtained, and the depth distance at that pixel is Zi is obtained by linear interpolation (primary interpolation) according to the following equation (1).
[0047]
[Expression 1]
z = Σ (Zi / li) / Σ (1 / li)
[0048]
It should be noted that the neighborhood region is a region having a predetermined area, a region having a variable area including a predetermined number of pixels whose distances are obtained, and the interpolation method is represented by a high-order interpolation method and various spline functions. There may be various variations such as a rational interpolation method, but the interpolation method of the present invention is not limited to the definition of a specific neighborhood region or the interpolation method.
[0049]
The area for which the distance has not been obtained in the processing up to step 204 is an area where the luminance change is equal to or smaller than a certain set value in step 202. Since these regions often change in luminance continuously, a relatively good approximate value can be obtained even by interpolating from the portion where the distance in the vicinity is obtained.
[0050]
As described above, in the three-dimensional shape measurement method of the present embodiment, by applying these steps, it is possible to easily measure the outline of the three-dimensional shape of the imaging target up to a certain distance.
Next, a more specific configuration and operation of the three-dimensional shape measuring apparatus according to the present embodiment will be described.
[0051]
FIG. 3 is a diagram showing a specific example of the three-dimensional shape measuring apparatus of the present embodiment. 3 is a camera with a close focus position, the image input unit 302 is an input unit for inputting a photographed image taken by the photographing camera 301, an image recording memory 303 is a memory for recording the inputted photographed image, The edge detector 304 detects a region in the image where the change in the luminance value of the pixel is a predetermined set value or more, and the luminance change region recording memory 305 records a region where the change in the luminance value is a predetermined set value or more. A memory / spatial frequency spectrum calculator 306 is a calculator for obtaining a spatial frequency spectrum in a region where the luminance change in the photographed image is a predetermined value or more. A combination of the edge detector 304, the luminance change area recording memory 305, and the spatial frequency spectrum calculator 306 constitutes the spatial frequency spectrum calculator 103 shown in FIG.
[0052]
A spatial frequency spectrum recording memory 307 is a memory for recording the obtained spatial frequency spectrum, a maximum distribution direction detector 308 is a detector for detecting a direction in which the distribution of the obtained spatial frequency spectrum is most widened, and a spectrum. An intensity attenuation factor calculator 309 is a calculator that calculates the attenuation factor of the spectral intensity with respect to the spatial frequency in the direction detected by the maximum distribution direction detector 308, and a distance calculator 310 is an operator that converts the value of the spectrum intensity attenuation factor into a distance. The attenuation rate → distance reference table 325 is a reference table that is referred to when converting the value of the spectral intensity attenuation rate into the distance, and the interpolation calculator 311 is a calculator that interpolates the distance value from the area where the nearby distance is obtained. The dimensional shape recording memory 312 is a memory for storing measured three-dimensional shape data. A shooting object 313 is an object to be shot, and a focal position 314 indicates a position where the shooting camera 301 is in focus. Note that the closest distance side of the measurement range of the present apparatus is up to the focal position 314.
[0053]
The photographed image data 315 represents image data photographed by the photographing camera 301 and recorded in the image recording memory 303. The contour area extraction data 316 represents an area detected by the edge detector 304 and recorded in the brightness change area recording memory 305, where the change in the brightness value of the pixel is greater than or equal to a set value. A spatial frequency spectrum 317 represents a spatial frequency spectrum centered on a certain pixel, which is obtained by the spatial frequency spectrum calculator 306 and recorded in the spatial frequency spectrum recording memory 307.
[0054]
A spatial frequency spectrum centered on a pixel is data of a two-dimensional distribution for each pixel as shown in this figure. In general, the spatial frequency spectrum has the maximum center point (DC component) and attenuates as the spatial frequency increases. Further, the spatial frequency spectrum is distributed up to the highest frequency in the normal direction of the contour portion, and shows a sharp attenuation in the tangential direction.
[0055]
FIG. 4 is a flowchart showing a processing procedure of the three-dimensional shape measuring apparatus of FIG. 3 of the present embodiment. The operation of executing the three-dimensional shape measurement method of this embodiment using the apparatus of FIG. 3 will be described according to the flowchart of FIG.
[0056]
When the photographing object 313 is photographed by the photographing camera 301, the image input unit 302 inputs a photographed image photographed by the photographing camera 301 in step 401 and records the inputted photographed image data 315 in the image recording memory 303. .
[0057]
In step 402, the edge detector 304 reads the captured image data 315 from the image recording memory 303, and extracts all pixels whose difference in luminance value between a certain pixel and a pixel adjacent to the pixel is equal to or greater than a preset value. Thus, the contour region is determined, the pixel information of the contour region is extracted, and written in the luminance change region recording memory 305 as the contour region extraction data 316. At this time, the adjacent condition may be near 4 or near 8.
[0058]
FIG. 5 is a diagram showing an outline of the pixel adjacency condition according to this embodiment. As shown in FIG. 5, the eight neighboring pixels adjacent to the pixel Px, y at the coordinates (x, y) of the captured image data 315 are Px-1, y-1, Px-1, y, Px-1, y + 1, Px, y-1, Px, y + 1, Px + 1, y-1, Px + 1, y, Px + 1, y + 1, and the four neighboring pixels are Px-1, y, Px, y-1, Px, y + 1, and Px + 1, y.
[0059]
In step 403, the spatial frequency spectrum calculator 306 reads out the contour region from the luminance change region recording memory 305, and reads out an image of N × N pixels in the vicinity centering on the pixel from the image recording memory 303 for each pixel. The spatial frequency spectrum 317 is obtained by Fourier transforming the pixels in the neighboring area and written into the spatial frequency spectrum recording memory 307.
[0060]
In the present embodiment, an example using Fourier transform is given, but different orthogonal basis transforms such as cosine / sine transform and WaveLet transform may be used. In addition, since the outline of the spatial frequency spectrum distribution only needs to be approximated, it is possible to use Hadamard transform, Walsh transform, etc. In this embodiment, the type of orthogonal basis transform function used for obtaining the spatial frequency spectrum is determined. It is not limited. Further, a high-speed calculation method such as FFT or WFT may be used, and the present embodiment does not limit the presence or absence of speed-up and the method thereof.
[0061]
In step 404, the maximum distribution direction detector 308 detects the maximum distribution direction 318 in which the obtained spatial frequency spectrum 317 extends to the highest frequency region. Since the spatial frequency spectrum 317 has a distribution shape as shown in the figure, the attenuation rate of the spectrum intensity with respect to the frequency is obtained for each direction from the center, and the direction with the smallest attenuation rate may be set as the maximum distribution direction 318. The maximum distribution direction 318 may be simply detected by connecting the center and the point having the maximum spatial frequency on a concentric circle separated from the center by a certain spatial frequency ω.
[0062]
In step 405, the spectrum intensity attenuation rate calculator 309 obtains an average value of attenuation with respect to the frequency of the detected spatial frequency spectrum intensity in the maximum distribution direction 318, and uses this value as the spectrum intensity attenuation rate.
[0063]
In step 406, the distance calculator 310 refers to the attenuation rate → distance reference table 325 and obtains a distance corresponding to the obtained spectral intensity attenuation rate from the attenuation rate → distance reference table 325. It is assumed that the attenuation rate → distance reference table 325 is created in advance by the following procedure according to the procedure shown in FIG.
[0064]
FIG. 6 is a flowchart showing a processing procedure of attenuation rate → distance reference table creation processing of the present embodiment. When the calibration imaging plate 608 in which the right half is black and the left half is white is placed at the focal position x0 of the imaging camera 301, in step 601, the three-dimensional shape measuring apparatus determines the distance between the calibration imaging plate 608 and the imaging camera 301. Set x to the initial value x0.
[0065]
Next, when the calibration photographing plate 608 is photographed by the photographing camera 301, in step 602, the image input unit 302 inputs the photographed photographed image from the photographing camera 301 and stores it in the image recording memory 303.
[0066]
In step 603, the spatial frequency spectrum calculator 306 reads the stored captured image from the image recording memory 303, obtains a horizontal spatial frequency spectrum at the boundary between white and black from the captured image, and records the spatial frequency spectrum. Store in the memory 307.
[0067]
In step 604, the spectrum intensity attenuation rate calculator 309 reads the stored spatial frequency spectrum from the spatial frequency spectrum recording memory 307, obtains an average value of attenuation with respect to the frequency of the spatial frequency spectrum intensity in the horizontal direction, and obtains this value. It is assumed that the spectral intensity attenuation rate k.
[0068]
In step 605, the three-dimensional shape measurement apparatus writes a pair of the distance x between the calibration imaging plate 608 and the imaging camera 301 and the obtained spectral intensity attenuation rate k to the attenuation rate → distance reference table 325. .
[0069]
Next, when the position of the calibration imaging plate 608 is separated from the imaging camera 301 by δx, the three-dimensional shape measuring apparatus adds δx to the value of the distance x between the calibration imaging plate 608 and the imaging camera 301 in step 606. .
[0070]
In step 607, the value x of the distance to the calibration imaging plate 608 is compared with a preset maximum measurement distance. If the distance x is greater than or equal to the maximum measurement distance, this process ends. If not, the process proceeds to step 602. Return. The attenuation rate → distance reference table 325 is generated by the above steps.
[0071]
In step 407, the interpolation calculator 311 interpolates the distance at that pixel by applying step 408 and step 409 to all the pixels that have not been extracted in step 402, that is, the pixels whose distance has not been determined by the distance calculator 310. Generate.
[0072]
In step 408, the interpolation calculator 311 selects a pixel, and is closest to the pixel in each quadrant from the first quadrant to the fourth quadrant when the pixel is set to the origin 320 as shown in the interpolation schematic diagram 319. By selecting the pixels for which the distance is obtained one by one, a total of four pixels 321 to 324 are obtained, and the distance at the pixel is interpolated and generated according to Equation 1. The reason for selecting the four pixels from the first quadrant to the fourth quadrant is that interpolation with higher accuracy is possible because the interpolated pixels always exist inside a quadrilateral connecting the four pixels. is there. However, since the pixels near the four corners of the image are not necessarily obtained by this selection method, four pixels are interpolated and generated from only the obtained pixels. In this embodiment, one pixel is selected for each quadrant, but this embodiment does not limit the pixel selection method. In step 409, the interpolation calculator 311 writes the distance generated by the interpolation in the three-dimensional shape recording memory 312.
[0073]
As described above, according to the three-dimensional shape measurement apparatus of the present embodiment, the distance between the region and the imaging means is obtained from the spectral intensity attenuation rate of the spatial frequency spectrum in the captured image captured at a predetermined focal position. A simple three-dimensional shape can be obtained from only one image by the Shape from Focus method.
[0074]
(Embodiment 2)
A three-dimensional shape measuring apparatus according to the second embodiment for obtaining the distance between the region and the imaging means from the spectral intensity attenuation rate of the spatial frequency spectrum in two captured images captured at different focal positions will be described below.
[0075]
FIG. 7 is a diagram showing a schematic configuration of the three-dimensional shape measuring apparatus of the present embodiment. As shown in FIG. 7, the three-dimensional shape measuring apparatus of this embodiment includes an image input unit 703, a spatial frequency spectrum calculation unit 704, a spectrum intensity attenuation rate calculation unit 705, a distance calculation unit 706, and an interpolation calculation unit 707. And have.
[0076]
The image input unit 703 is photographed by the first photographing unit 701 having a focal position at a predetermined position and the second photographing unit 702 having a focal position different from the predetermined position. And a second photographed image.
[0077]
The spatial frequency spectrum calculation means 704 has a spatial frequency spectrum of the photographic image having the larger luminance change and the other of the inputted luminance changes in the first and second photographed images in the region where at least one of the luminance changes is a predetermined value or more. It is means for obtaining the spatial frequency spectrum of the photographed image as a first spatial frequency spectrum and a second spatial frequency spectrum, respectively.
[0078]
Spectral intensity attenuation rate calculating means 705 obtains, for the first spatial frequency spectrum, an attenuation rate in a direction that minimizes the attenuation of the spectral intensity with respect to the spatial frequency as the first attenuation rate, and for the second spatial frequency spectrum. This is means for obtaining the attenuation factor of the spectral intensity with respect to the spatial frequency in the same direction as the second attenuation factor.
[0079]
The distance calculation means 706 is a means for obtaining the distance between the subject in the area and the first photographing means 701 or the second photographing means 702 based on the obtained first and second attenuation factors. Interpolation calculation means 707 interpolates the subject from the area where the distance calculation means 706 calculates the distance for the area where the luminance change in the input first and second captured images is less than the predetermined value. Means for obtaining a distance from the first photographing means 701 or the second photographing means 702.
[0080]
The three-dimensional shape measuring apparatus of the present embodiment can be realized by a computer and a program, and the program can be recorded on a recording medium or provided through a network.
[0081]
FIG. 8 is a diagram showing a processing outline of the three-dimensional shape measurement processing of the present embodiment. When photographing is performed by the photographing means 701 having a far focal position, the image input means 703 of the three-dimensional shape measuring apparatus inputs a photographed image 807 photographed by the photographing means 701 in step 801. In the acquired captured image 807, a region 808 far from the image capturing unit 701 is clearly visible, whereas a region 809 near the distance is a blurred image.
[0082]
When photographing is performed by the photographing unit 702 having a close focal position, the image input unit 703 inputs the photographed image 810 photographed by the photographing unit 702 in step 802. In the acquired photographed image 810, a region 811 near the distance from the photographing means 702 is clearly shown, whereas a region 812 far away is a blurred image.
[0083]
Since step 801 and step 802 have no temporal order relationship, they may be performed in the order of step 802 and step 801. Steps 801 and 802 are performed using two photographing means having coaxial photographing optical axes. You may go at the same time.
[0084]
In step 803, the spatial frequency spectrum calculation unit 704 takes the captured image in the region at the position of the contour region extraction data 813 in which the luminance change in at least one of the captured image 807 and the captured image 810 is greater than or equal to a set value. The spatial frequency spectrums of 807 and the captured image 810 are respectively obtained, and the spatial frequency spectrum 814 is the one with the larger luminance change, and the spatial frequency spectrum 815 is the other.
[0085]
For example, when the region 816 has a luminance change equal to or greater than a predetermined setting value in the photographed image 810, the spatial frequency spectrum of the region 816 is obtained as the spatial frequency spectrum 814, and the region in the photographed image 807 is located at the same position as the region 816. On the other hand, a spatial frequency spectrum is obtained to obtain a spatial frequency spectrum 815. In addition, when the brightness | luminance change in either area | region is the same magnitude | size, allocation of the spatial frequency spectrum 814 and the spatial frequency spectrum 815 may be arbitrary.
[0086]
In step 804, the spectral intensity attenuation rate calculating means 705 obtains the attenuation rate 817 in the direction 819 where the attenuation of the spectral intensity with respect to the frequency of the spatial frequency spectrum 814 is the smallest, and obtains the attenuation rate in the same direction from the spatial frequency spectrum 815.
[0087]
In step 805, the distance calculation means 706 obtains the distance between the area and the imaging means from the attenuation rate 817 and the attenuation rate 818. Specifically, the ratio between the attenuation rate 817 and the attenuation rate 818 is obtained.
[0088]
Since the ratio of the attenuation rate reflects the distance information from the focal position to the area, the distance from the focal position to the area can be obtained from this ratio. Further, since the distance between the focal position and the photographing means is known, the distance between the photographing means and the region can be obtained by adding the distance between the focal position and the photographing means to the obtained distance.
[0089]
By the way, the ratio of the attenuation factor fluctuates due to the influence of diffraction caused by the focal position of the lens itself, the size of the diaphragm, the MTF of the lens, the diaphragm, etc., but it is difficult to remove all these effects analytically. is there. Therefore, paying attention to the fact that these influencing factors do not depend on the subject and are determined only by the imaging optical system of the photographing means, using the photographing means in advance, a subject having a region where the luminance changes discontinuously (for example, the right half) The distance reference table 820 is created in advance by measuring the ratio of the distance and the spectral intensity attenuation rate each time. Keep it. Since this reference table has already incorporated all the effects of the imaging optical system of the imaging means, at the time of subsequent measurement, the distance of the imaging optical system of the imaging means is read by referring to the reference table from the obtained attenuation rate ratio. Only the distance can be obtained by removing the influence.
[0090]
In step 806, the interpolation calculation means 707 generates a distance by interpolating the area for which the distance has not been obtained in the processing up to step 805 from the neighboring area for which the distance calculation means 706 has obtained the distance. The area where the distance has not been obtained in the processing up to step 805 is an area where the luminance change is not more than a certain set value in step 803. Since these regions often change in luminance continuously, the distance can be approximated by complementing from the portion where the distance in the vicinity is obtained. This interpolation method may be the method shown in the description of the first embodiment.
[0091]
By applying these steps, the three-dimensional shape of the imaging target can be easily measured. Compared with the method of the first embodiment, the method of the present embodiment has advantages such as being able to calculate the distance more accurately and measuring far distances.
[0092]
In addition, when it is desired to obtain color information of an image at the same time, since the distant view is blurred in the method according to the first embodiment, it is necessary to use a sufficiently high-resolution image capturing unit or use color capturing unit for color information. In the method of the embodiment, it is also possible to easily obtain a color image at the same time by the photographing means by setting the depth of focus of the photographing means having a focal position far away.
Next, a more specific configuration and operation of the three-dimensional shape measuring apparatus according to the present embodiment will be described.
[0093]
FIG. 9 is a diagram showing a specific example of the three-dimensional shape measuring apparatus of the present embodiment. The shooting camera 901 in FIG. 9 is a camera with a far focus position, and the shooting camera 902 is a camera with a close focus position. The half mirror 929 and the mirror 930 enable the photographing camera 901 and the photographing camera 902 to take an image on the same axis. An image input unit 903 is an input unit for inputting a photographic image taken by the photographic camera 901 and the photographic camera 902, an image recording memory 904 is an image recording memory for recording an image of the photographic camera 901, and an image recording memory 905 is an image of the photographing camera 902. An image recording memory for recording an image, and an edge detector 906 detects an area in which a change in luminance value in at least one of a photographed image of the photographing camera 901 or a photographed image of the photographing camera 902 is a predetermined set value or more. A brightness change area recording memory 907 is a memory that records the area where the change in brightness value is not less than a predetermined setting value including information on which image, and the spatial frequency spectrum calculator 908 is a spatial frequency detector for the detected area. This is an arithmetic unit for obtaining a spectrum.
[0094]
A spatial frequency spectrum recording memory 909 is a memory that records a spatial frequency spectrum of a photographed image of the photographing camera 901, a spatial frequency spectrum recording memory 910 is a memory that records a spatial frequency spectrum of a photographed image of the photographing camera 902, and a maximum distribution direction detector 911. Is a detector for detecting the direction in which the attenuation of the spectral intensity with respect to the spatial frequency is the smallest in the image having the larger luminance change, and the spectral intensity attenuation factor calculator 912 is a photographing camera 901 in the direction detected by the maximum distribution direction detector 911. A computing unit for calculating the spectral intensity attenuation rate of the captured image, and a spectral intensity attenuation rate calculating unit 913 for calculating the spectral intensity attenuation rate of the captured image of the imaging camera 902 in the direction detected by the maximum distribution direction detector 911, The distance calculator 914 is a camera 9 An arithmetic unit that converts the ratio of the spectral intensity attenuation rate of the captured image of 1 and the spectral intensity attenuation rate of the captured image of the imaging camera 902 into a distance, the attenuation rate ratio → distance reference table 937 converts the ratio of the spectral intensity attenuation rate into a distance. A reference table to be referred to when performing interpolation, an interpolation calculator 915 is a calculator that interpolates a distance value from an area from which a nearby distance is obtained, a three-dimensional shape recording memory 916 is a memory that stores measured three-dimensional shape data, and imaging An object 917 is an object to be imaged. A focal position 918 indicates a position where the photographing camera 901 is in focus, and a focal position 919 indicates a position where the photographing camera 902 is in focus. Note that the measurement range of the present apparatus is between the focal position 918 and the focal position 919.
[0095]
FIG. 10 is a flowchart showing a processing procedure of the three-dimensional shape measuring apparatus of FIG. 9 of the present embodiment. The operation of executing the three-dimensional shape measurement method of the present embodiment using the apparatus of FIG. 9 will be described according to the flowchart of FIG.
[0096]
When the photographing object 917 is photographed by the photographing camera 901, in step 1001, the image input unit 903 inputs a photographed image photographed by the photographing camera 901 and records the input image data 920 in the image recording memory 904.
[0097]
When the photographing object 917 is photographed by the photographing camera 902, the image input unit 903 inputs a photographed image photographed by the photographing camera 902 in step 1002, and records the inputted image data 921 in the image recording memory 905.
[0098]
In step 1003, the edge detector 906 reads the image data 920 and 921 from the image recording memories 904 and 905, and the difference between the luminance values of a certain pixel and a pixel adjacent to the pixel is greater than or equal to a preset value for each image. The contour region is determined by extracting all such pixels, and the region is written in the luminance change region recording memory 907 as the contour region extraction data 922. At this time, information of an image having a larger luminance value difference is also written. For example, a contour region 923 indicated by white represents a contour region obtained from the image data 921, and a contour region 924 indicated by a black line represents a contour region obtained from the image data 920.
[0099]
In step 1004, the spatial frequency spectrum calculator 908 reads out the contour region from the luminance change region recording memory 907, and for each pixel constituting the contour region, an image of N × N pixels in the vicinity centering on the pixel is obtained. A spatial frequency spectrum is obtained by reading from the image recording memories 904 and 905 and Fourier-transformed and written in the spatial frequency spectrum recording memories 909 and 910.
[0100]
In the present embodiment, an example using Fourier transform is given, but different orthogonal basis transforms such as cosine / sine transform and WaveLet transform may be used. In addition, since the outline of the spatial frequency spectrum distribution only needs to be approximated, it is possible to use Hadamard transform, Walsh transform, etc. In this embodiment, the type of orthogonal basis transform function used for obtaining the spatial frequency spectrum is determined. It is not limited. Further, a high-speed calculation method such as FFT or WFT may be used, and the present embodiment does not limit the presence or absence of speed-up and the method thereof.
[0101]
In step 1005, the maximum distribution direction detector 911 selects the spatial frequency spectrum of the captured image having the larger luminance change for each pixel from the spatial frequency spectrum recording memory 909 or the spatial frequency spectrum recording memory 910, and the spatial frequency spectrum. Detects the direction that extends to the highest frequency range. For example, spatial frequency spectra 926 and 927 are obtained by obtaining the spatial frequency spectrum of the pixel 925 from the images of the image data 920 and 921 in the contour region extraction data 922. Since the pixel 925 is a white line pixel, an image having a larger luminance change is the image data 921. Therefore, the direction 928 in which the spatial frequency spectrum 927 extends from the image data 921 to the highest frequency region is detected.
[0102]
In step 1006, the spectral intensity attenuation rate calculators 912 and 913 calculate the spectral intensity attenuation rate with respect to the spatial frequency in the detected direction for each image.
[0103]
In step 1007, the distance calculator 914 takes the ratio of the calculated spectral intensity attenuation factors, refers to the attenuation factor ratio → distance reference table 937, and sets the distance corresponding to the attenuation factor ratio to the attenuation factor ratio → It is obtained from the distance reference table 937. It is assumed that the attenuation ratio → distance reference table 937 is created in advance by the following procedure according to the procedure shown in FIG.
[0104]
FIG. 11 is a flowchart showing a processing procedure of the attenuation rate ratio → distance reference table creation processing of the present embodiment. When the calibration imaging plate 1109 whose right half is black and left half is white is placed at the focal position 919 of the imaging camera 902, in step 1101, the three-dimensional shape measuring apparatus determines the distance x between the calibration imaging plate 1109 and the imaging camera. Is set to the initial value.
[0105]
Next, when the calibration photographing plate 1109 is photographed by the photographing cameras 901 and 902, the image input unit 903 inputs the photographed photographed images from the photographing cameras 901 and 902 in step 1102 and stores them in the image recording memories 904 and 905. Store.
[0106]
In step 1103, the spatial frequency spectrum calculator 908 reads the stored captured images from the image recording memories 904 and 905, and obtains horizontal spatial frequency spectra at the boundary between white and black from the captured images, respectively. They are stored in the spatial frequency spectrum recording memories 909 and 910, respectively.
[0107]
In step 1104, the spectrum intensity attenuation rate calculators 912 and 913 read the stored spatial frequency spectrum from the spatial frequency spectrum recording memories 909 and 910, respectively, and then determine the attenuation rate of the spectrum intensity with respect to the frequency from each spatial frequency spectrum. .
[0108]
In step 1105, the three-dimensional shape measuring apparatus takes a ratio kr of the obtained spectral intensity attenuation rate, and in step 1106, the distance x between the calibration imaging plate 1109 and the imaging camera and the obtained spectral intensity attenuation rate. The ratio kr pair is added to the attenuation rate ratio → distance reference table 937.
[0109]
Next, when the position of the calibration imaging plate 1109 is separated from the imaging camera by δx, in step 1107, the three-dimensional shape measuring apparatus adds δx to the value of the distance x between the calibration imaging plate 1109 and the imaging camera.
[0110]
In step 1108, the distance x to the calibration imaging plate 1109 is compared with the distance to the far focus position 918. If the distance x is far from the far focus position 918, the process is terminated. Return to. By executing the above steps, the attenuation ratio → distance reference table 937 is generated.
[0111]
In step 1008, the interpolation calculator 915 interpolates the distance at that pixel by applying step 1009 and step 1010 to all the pixels that were not extracted in step 1003, that is, the pixels whose distance was not obtained by the distance calculator 914. Generate.
[0112]
In step 1009, the interpolation calculator 915 selects a pixel, and is closest to the pixel in each quadrant from the first quadrant to the fourth quadrant when the pixel is set to the origin 932 as shown in the interpolation schematic diagram 931 and By selecting the pixels for which distances have been obtained one by one, a total of four pixels 933 to 936 are obtained, and the distances in the pixels are interpolated and generated according to Equation 1. The reason for selecting the four pixels from the first quadrant to the fourth quadrant is that interpolation with higher accuracy is possible because the interpolated pixels always exist inside a quadrilateral connecting the four pixels. is there. However, since the pixels near the four corners of the image are not necessarily obtained by this selection method, four pixels are interpolated and generated according to Equation 1 from only the obtained pixels. In this embodiment, one pixel is selected for each quadrant, but this embodiment does not limit the pixel selection method. In step 1010, the interpolation calculator 915 writes the distance generated by the interpolation in the three-dimensional shape recording memory 916.
[0113]
As described above, according to the three-dimensional shape measuring apparatus of the present embodiment, the distance between the region and the imaging means is obtained from the spectral intensity attenuation rate of the spatial frequency spectrum in two captured images captured at different focal positions. Therefore, it is possible to obtain a three-dimensional shape with higher accuracy and a wider measurement range by using the Shape from Focus method from only two images.
[0114]
(Embodiment 3)
A three-dimensional shape measuring apparatus according to the third embodiment that obtains the distance from the spectral intensity attenuation rate of the one-dimensional spatial frequency spectrum in the normal direction of the boundary where the luminance changes will be described below.
[0115]
FIG. 12 is a diagram showing a schematic configuration of the three-dimensional shape measuring apparatus of the present embodiment. As shown in FIG. 12, the three-dimensional shape measurement apparatus of this embodiment includes an image input unit 1203, a normal direction one-dimensional spatial frequency spectrum calculation unit 1204, a spectral intensity attenuation rate calculation unit 1205, and a distance calculation unit 1206. Interpolation calculation means 1207 is provided.
[0116]
The image input unit 1203 is photographed by the first photographing unit 1202 having a focal position at a predetermined position and the second photographing unit 1202 having a focal position different from the predetermined position. And a second photographed image.
[0117]
The normal direction one-dimensional spatial frequency spectrum calculating means 1204 is configured to detect the image of the captured image having the larger luminance change in the region where at least one of the luminance changes in the input first and second captured images is a predetermined value or more. The normal direction of the boundary portion of the luminance change is obtained, and the one-dimensional spatial frequency spectrum in the normal direction of the photographed image having the larger luminance change and the one-dimensional spatial frequency spectrum in the normal direction of the other photographed image are obtained. Means for obtaining the first spatial frequency spectrum and the second spatial frequency spectrum, respectively.
[0118]
The spectral intensity attenuation rate calculating means 1205 is a means for obtaining the first and second attenuation rates of the spectral intensity with respect to the spatial frequency for the first and second spatial frequency spectra. The distance calculation means 1206 is a means for obtaining the distance between the subject in the area and the first photographing means 1201 or the second photographing means 1202 based on the obtained first and second attenuation factors.
[0119]
The interpolation calculation means 1207 performs interpolation on the subject by interpolating from the area for which the distance is calculated by the distance calculation means 1206 for the area where the luminance change in the input first and second captured images is less than the predetermined value. And a distance between the first photographing unit 1201 and the second photographing unit 1202.
[0120]
FIG. 12 shows a configuration in a case where the distance is obtained from the spectral intensity attenuation rate of the one-dimensional spatial frequency spectrum in the normal direction of the boundary portion where the luminance changes in the apparatus as in the second embodiment. It is good also as what calculates | requires distance similarly with such an apparatus.
[0121]
In other words, the image input unit 1203 inputs a photographed image photographed by the photographing unit 1201 or the photographing unit 1202 having a focal position at a predetermined position, and the normal direction one-dimensional spatial frequency spectrum calculation unit 1204 For a region where the luminance change is equal to or greater than a predetermined value, the normal direction of the boundary portion of the luminance change and the one-dimensional spatial frequency spectrum in the normal direction are obtained, and the spectral intensity attenuation rate calculation means 1205 calculates 1 in the normal direction. With respect to the three-dimensional spatial frequency spectrum, the attenuation rate of the spectral intensity with respect to the spatial frequency is obtained, the distance computing unit 1206 obtains the distance between the subject in the region and the imaging unit based on the attenuation rate, and the interpolation computing unit 1207 performs the imaging. The distance is calculated by the distance calculation means 1206 for an area where the luminance change in the image is less than the predetermined value. By interpolation from the region are or as obtaining the distance between the imaging means and the object.
[0122]
Further, the three-dimensional shape measuring apparatus of the present embodiment can be realized by a computer and a program, and the program can be recorded on a recording medium or provided through a network.
[0123]
FIG. 13 is a diagram showing the relationship between the normal direction of the boundary portion where the luminance changes and the spatial frequency spectrum of this embodiment. A photographed image 1301 in FIG. 13 is an image obtained by photographing a boundary portion where the luminance changes, and a luminance change boundary portion region 1302 is a boundary portion region where the luminance change is equal to or greater than a set value. The luminance change boundary normal direction 1303 indicates the normal direction of the boundary where the luminance changes by an arrow. When the spatial frequency spectrum of the luminance change boundary region 1302 is obtained, a spatial frequency spectrum 1304 is obtained.
[0124]
In the luminance change boundary normal direction 1303, the attenuation rate in the direction where the attenuation of the spectral intensity with respect to the spatial frequency is the smallest is obtained. This direction is indicated by an arrow 1305. Then, it can be seen that the luminance change boundary normal direction 1303 and the arrow 1305 are substantially in the same direction. Therefore, if the normal direction of the boundary where the luminance changes is detected in the process of detecting a region where the luminance change is equal to or greater than a predetermined set value, a one-dimensional spatial frequency spectrum in that direction is obtained, and then the spectral intensity is attenuated. The same result can be obtained by determining the rate. The process for doing this is shown in the flowchart of FIG.
[0125]
FIG. 14 is a diagram showing a processing outline of the three-dimensional shape measurement processing of the present embodiment. When photographing is performed by a photographing unit having a focal position at a predetermined position, in step 1401, the image input unit 1203 inputs a photographed image photographed by the photographing unit. In the case of the configuration similar to that of the second embodiment as shown in FIG. 12, two images are taken using the photographing means 1201 having a far focal position and the photographing means 1202 having a close focal position, and the same as in the first embodiment. In the case of the configuration, an image is captured by the imaging unit 1202 having a close focal position.
[0126]
In step 1402, the normal direction one-dimensional spatial frequency spectrum calculation means 1204 detects a region where the luminance change is equal to or greater than a predetermined set value. When two images are captured as described above, an area in which at least one of the luminance changes in the captured images is a predetermined value or more is detected.
[0127]
In step 1403, the normal direction one-dimensional spatial frequency spectrum calculation means 1204 detects the boundary portion where the luminance changes included in the region detected in step 1402 and obtains the normal direction of the boundary portion.
[0128]
In step 1404, the normal direction one-dimensional spatial frequency spectrum calculation means 1204 obtains the detected normal direction one-dimensional spatial frequency spectrum, and the spectrum intensity attenuation rate calculation means 1205 obtains the attenuation rate of the spectrum intensity with respect to the frequency. When two images are captured as described above, the one-dimensional spatial frequency spectrum in the normal direction of the captured image with the larger luminance change and the one-dimensional spatial frequency in the normal direction of the other captured image. The spectrum is obtained as a first spatial frequency spectrum and a second spatial frequency spectrum, respectively, and the spectral intensity attenuation rate calculating means 1205 determines the attenuation rate of the spectral intensity with respect to the spatial frequency for each of the first and second spatial frequency spectra. It calculates | requires as a 1st and 2nd attenuation factor.
[0129]
In step 1405, the distance calculation means 1206 obtains the distance using the attenuation rate obtained in the previous step. The specific method of determination is based on the method described in the operation of the first or second embodiment. Further, the interpolation calculation means 1207 interpolates the subject and the subject by interpolation from the area for which the distance is calculated by the distance calculation means 1206 for the area where the luminance change in both the first and second captured images is less than the predetermined value. A distance from the first photographing unit 1201 or the second photographing unit 1202 is obtained.
[0130]
By using the three-dimensional shape measurement method of this embodiment, it is not necessary to obtain a two-dimensional spatial frequency spectrum, so that the calculation cost can be greatly reduced and the processing time can be shortened.
Next, a more specific configuration and operation of the three-dimensional shape measuring apparatus according to the present embodiment will be described.
[0131]
FIG. 15 is a diagram showing a specific example of the three-dimensional shape measuring apparatus of the present embodiment. The three-dimensional shape measuring apparatus according to the present embodiment has a configuration for obtaining the distance from the spectral intensity attenuation rate of the one-dimensional spatial frequency spectrum in the normal direction of the boundary where the luminance changes, as shown in FIG. It is built in. Therefore, the same constituent means as in the apparatus of FIG. 3 of the first embodiment is given the same number as in FIG.
[0132]
An edge vector detector 1501 is a detector that detects an area in which a change in luminance value is equal to or greater than a predetermined set value in the image recorded in the image recording memory 303 and a normal vector of the area, and a luminance change area recording memory 1502 A memory for recording a region where a change in luminance value is a predetermined value or more and a normal vector thereof, a spatial frequency spectrum calculator 1503 is a calculator for calculating a one-dimensional spatial frequency spectrum in the direction of the normal vector, and a spatial frequency spectrum recording A memory 1504 is a memory for recording a one-dimensional spatial frequency spectrum.
[0133]
FIG. 16 is a flowchart showing the processing procedure of the three-dimensional shape measurement processing of the present embodiment. The operation of executing the three-dimensional shape measurement method of the present embodiment using the apparatus of FIG. 15 will be described according to the flowchart of FIG.
[0134]
When the photographing object 313 is photographed by the photographing camera 301, the image input unit 302 inputs a photographed image photographed by the photographing camera 301 in step 1601 and records the inputted photographed image data 315 in the image recording memory 303. .
[0135]
In step 1602, the edge vector detector 1501 reads the captured image data 315 from the image recording memory 303 and extracts all pixels whose luminance value difference between a certain pixel and a pixel adjacent to the pixel is equal to or larger than a preset value. To determine the contour region. Next, a normal vector orthogonal to the contour region at that position is obtained for each pixel constituting the contour region, and the region and normal vector are written into the luminance change region recording memory 1502. The contour area extraction data 1505 schematically represents the data, and it is assumed that a normal vector 1506 having a unit length is also recorded for each pixel.
[0136]
In step 1603, the spatial frequency spectrum calculator 1503 reads the contour region and the normal vector from the luminance change region recording memory 1502, and images N pixels in the vicinity of the normal vector direction centered on the pixel for each pixel. A one-dimensional spatial frequency spectrum 1507 is obtained by reading from the recording memory 303 and Fourier-transforming the pixels in the neighboring region, and written in the spatial frequency spectrum recording memory 1504.
[0137]
In step 1604, the spectrum intensity attenuation rate calculator 309 obtains the spectrum intensity attenuation rate from the one-dimensional spatial frequency spectrum.
[0138]
In step 1605, the distance calculator 310 refers to the attenuation rate → distance reference table 325 and obtains a distance corresponding to the obtained spectral intensity attenuation rate from the attenuation rate → distance reference table 325. It is assumed that the attenuation rate → distance reference table 325 is created in advance according to the procedure shown in FIG.
[0139]
In step 1606, the interpolation calculator 311 interpolates the distance at that pixel by applying step 1607 and step 1608 to all the pixels that have not been extracted in step 1602, ie, the pixels whose distance has not been determined by the distance calculator 310. Generate.
[0140]
In step 1607, the interpolation computing unit 311 selects a pixel, and sets the pixel closest to the pixel and the distance obtained for each quadrant from the first quadrant to the fourth quadrant when the pixel is set as the origin. By selecting one by one, a total of four pixels are obtained, and the distances at the pixels are interpolated and generated according to Equation 1. In this embodiment, one pixel is selected for each quadrant, but this embodiment does not limit the pixel selection method. In step 1608, the interpolation calculator 311 writes the distance generated by the interpolation in the three-dimensional shape recording memory 312.
[0141]
As described above, according to the three-dimensional shape measurement apparatus of this embodiment, the distance is obtained from the spectral intensity attenuation rate of the one-dimensional spatial frequency spectrum in the normal direction of the boundary where the luminance changes, so that the calculation cost is greatly reduced. However, the processing time can be shortened.
[0142]
(Embodiment 4)
Hereinafter, a three-dimensional shape measuring apparatus according to Embodiment 4 for obtaining a distance from a spectral intensity attenuation rate of a one-dimensional spatial frequency spectrum in the horizontal direction will be described.
[0143]
FIG. 17 is a diagram showing a schematic configuration of the three-dimensional shape measuring apparatus of the present embodiment. As shown in FIG. 17, the three-dimensional shape measurement apparatus of this embodiment includes an image input unit 1702, a horizontal one-dimensional spatial frequency spectrum calculation unit 1703, a spectrum intensity attenuation rate calculation unit 1704, a distance calculation unit 1705, And an interpolation calculation means 1706.
[0144]
The image input unit 1702 is a unit that inputs a photographed image photographed by the photographing unit 1701 having a focal position at a predetermined position. A horizontal one-dimensional spatial frequency spectrum calculating means 1703 is a means for obtaining a horizontal one-dimensional spatial frequency spectrum for a region where the luminance change in the input photographed image is a predetermined value or more.
[0145]
The spectrum intensity attenuation rate calculating means 1704 is a means for obtaining an attenuation rate of the spectrum intensity with respect to the spatial frequency for the one-dimensional spatial frequency spectrum in the horizontal direction. The distance calculation means 1705 is a means for obtaining the distance between the subject in the area and the photographing means 1701 based on the obtained attenuation rate.
[0146]
The interpolation calculation means 1706 calculates the distance between the subject and the photographing means 1701 by interpolation from the area where the distance is calculated by the distance calculation means 1705 for the area where the luminance change in the inputted photographed image is less than the predetermined value. It is a means to seek.
[0147]
FIG. 17 shows a configuration in the case where the distance is obtained from the spectral intensity attenuation rate of the one-dimensional spatial frequency spectrum in the horizontal direction using the apparatus as in the first embodiment. It is good also as what asks for.
[0148]
In other words, the image input unit 1702 has a first photographed image photographed by the first photographing unit 1701 having a focal position at a predetermined position and a second photographing unit having a focal position different from the predetermined position (FIG. And a second one-shot spatial frequency spectrum calculating means 1703, in which at least one of the luminance changes in the first and second shot images is greater than or equal to a predetermined value. The horizontal one-dimensional spatial frequency spectrum of the captured image with the larger luminance change and the horizontal one-dimensional spatial frequency spectrum of the other captured image are respectively the first spatial frequency spectrum and the second spatial frequency. The spectrum intensity attenuation rate calculating means 1704 obtains the spectrum as a spatial frequency with respect to the first and second spatial frequency spectra. The first and second attenuation factors are obtained as the first and second attenuation factors, respectively, and the distance calculation means 1705 determines the subject in the area and the first or second imaging means based on the first and second attenuation factors. In the interpolation calculation means 1706, the area where the luminance change in the first and second captured images is less than the predetermined value from the area where the distance is calculated by the distance calculation means 1705 is calculated. The distance between the subject and the first or second photographing unit may be obtained by interpolation.
[0149]
Further, the three-dimensional shape measuring apparatus of the present embodiment can be realized by a computer and a program, and the program can be recorded on a recording medium or provided through a network.
[0150]
FIG. 18 is a diagram showing the relationship between the direction of luminance change and the one-dimensional spatial frequency spectrum in the horizontal direction according to this embodiment. A captured image 1801 in FIG. 18 is an image obtained by capturing a boundary portion where the luminance changes in the horizontal direction, and a luminance change boundary portion region 1802 is a region of the boundary portion where the luminance changes in the horizontal direction. When a one-dimensional spatial frequency spectrum in the horizontal direction of the luminance change boundary region 1802 is obtained, a spatial frequency spectrum 1803 is obtained. By obtaining the attenuation factor 1804 from the spatial frequency spectrum 1803, the attenuation factor at the boundary where the luminance changes in the horizontal direction can be obtained efficiently.
[0151]
On the other hand, the photographed image 1805 is an image obtained by photographing the boundary portion where the luminance changes in the vertical direction. When a one-dimensional spatial frequency spectrum in the horizontal direction of the luminance change boundary area 1806 of the photographed image 1805 is obtained, a spatial frequency spectrum 1807 is obtained. Since the spatial frequency spectrum 1807 contains almost no high frequency component, the attenuation rate cannot be obtained.
[0152]
Therefore, in the method of the present embodiment, the boundary portion where the luminance changes in the horizontal direction can be correctly processed, but the error increases as the boundary portion tilts, and the boundary portion where the luminance changes in the vertical direction cannot be measured. However, when used for 3D image generation using the parallax effect, it is often sufficient to capture only the boundary portion where the luminance changes mainly in the horizontal direction, so this method is useful for this type of application. . Furthermore, the method according to the present embodiment does not require detection of the direction of the boundary portion as compared with the method according to the third embodiment, so that it is possible to further reduce the calculation cost and improve the processing speed.
[0153]
FIG. 19 is a diagram showing a process overview of the three-dimensional shape measurement process of the present embodiment. When photographing is performed by a photographing unit having a focal position at a predetermined position, in step 1901, the image input unit 1702 inputs a photographed image photographed by the photographing unit. In the case of the configuration similar to that of the first embodiment as shown in FIG. 17, an image is captured by the imaging unit 1701 having a close focal position, and in the case of the configuration similar to that of the second embodiment, the imaging unit having a far focal position. Two images are photographed using photographing means having a close focal position.
[0154]
In step 1902, the horizontal one-dimensional spatial frequency spectrum calculation means 1703 detects a region where the luminance change is equal to or greater than a predetermined set value.
[0155]
In step 1903, the horizontal one-dimensional spatial frequency spectrum calculation means 1703 obtains a horizontal one-dimensional spatial frequency spectrum for the region detected in step 1902, and the spectral intensity attenuation rate calculation means 1704 attenuates the spectral intensity with respect to the spatial frequency. Find the rate.
[0156]
In step 1904, the distance calculation means 1705 obtains the distance using the attenuation rate obtained in the previous step. The specific method of determination is based on the method described in the operation of the first or second embodiment. The interpolation calculation unit 1706 obtains the distance between the subject and the photographing unit 1701 by interpolation from the region where the distance is obtained by the distance calculation unit 1705 for the region where the luminance change in the photographed image is less than the predetermined value. .
[0157]
By using the three-dimensional shape measurement method of this embodiment, it is not necessary to obtain a two-dimensional spatial frequency spectrum, and it is not necessary to obtain the direction of a boundary portion where the luminance changes, so that the calculation cost can be greatly reduced. Thus, the processing time can be shortened.
[0158]
On the other hand, since the method of the present embodiment is only processing in the horizontal direction, it is not possible to measure a region where the distance changes in the vertical direction. However, when generating a parallax image or the like, it is often sufficient to be able to measure a region whose distance changes only in the horizontal direction, and there is no problem when used for such applications.
Next, a more specific configuration and operation of the three-dimensional shape measuring apparatus according to the present embodiment will be described.
[0159]
FIG. 20 is a diagram showing a specific example of the three-dimensional shape measuring apparatus of the present embodiment. The three-dimensional shape measuring apparatus according to the present embodiment incorporates a configuration for obtaining a distance from the spectral intensity attenuation rate of a horizontal one-dimensional spatial frequency spectrum in the apparatus according to the first embodiment shown in FIG. Therefore, the same constituent means as in the apparatus of FIG. 3 of the first embodiment is given the same number as in FIG.
[0160]
A horizontal one-dimensional spatial frequency spectrum calculator 2001 is a calculator that calculates a horizontal one-dimensional spatial frequency spectrum. A spatial frequency spectrum recording memory 2002 is a memory for recording a horizontal one-dimensional spatial frequency spectrum. The spectral intensity attenuation factor calculator 2003 is an arithmetic unit that calculates the spectral intensity attenuation factor with respect to the frequency of the horizontal one-dimensional spatial frequency spectrum.
[0161]
FIG. 21 is a flowchart showing the processing procedure of the three-dimensional shape measurement processing of the present embodiment. The operation of executing the three-dimensional shape measurement method of the present embodiment using the apparatus of FIG. 20 will be described according to the flowchart of FIG.
[0162]
When the photographing object 313 is photographed by the photographing camera 301, the image input unit 302 inputs a photographed image photographed by the photographing camera 301 in step 2101 and records the inputted photographed image data 315 in the image recording memory 303. .
[0163]
In step 2102, the edge detector 304 reads the photographed image data 315 from the image recording memory 303, and extracts all pixels whose luminance value difference between a certain pixel and a pixel adjacent to the pixel is equal to or larger than a preset value. Thus, the contour region is determined, and the region is written in the luminance change region recording memory 305.
[0164]
In step 2103, the horizontal one-dimensional spatial frequency spectrum calculator 2001 reads the contour region from the luminance change region recording memory 305, and records an image of N pixels in the horizontal direction centering on the pixel for each pixel. A one-dimensional spatial frequency spectrum 2004 is obtained by reading from the memory 303 and Fourier transforming the pixels in the neighboring region, and written in the spatial frequency spectrum recording memory 2002.
[0165]
In step 2104, the spectral intensity attenuation factor calculator 2003 obtains a horizontal spectral intensity attenuation factor from the one-dimensional spatial frequency spectrum.
[0166]
In step 2105, the distance calculator 310 refers to the attenuation rate → distance reference table 325 and obtains a distance corresponding to the obtained spectral intensity attenuation rate from the attenuation rate → distance reference table 325. It is assumed that the attenuation rate → distance reference table 325 is created in advance according to the procedure shown in FIG.
[0167]
In step 2106, the interpolation calculation unit 311 and the pixel not extracted in step 2102, that is, the pixel whose distance is not obtained by the distance calculation unit 310 are applied to step 2107 and step 2108, and the distance in the pixel is generated by interpolation. To do.
[0168]
In step 2107, the interpolation calculator 311 selects a pixel, and sets the pixel closest to the pixel and the distance obtained for each quadrant from the first quadrant to the fourth quadrant when the pixel is set as the origin. By selecting one by one, a total of four pixels are obtained, and the distances at the pixels are interpolated and generated according to Equation 1. In this embodiment, one pixel is selected for each quadrant, but this embodiment does not limit the pixel selection method. In step 2108, the interpolation calculator 311 writes the distance generated by the interpolation in the three-dimensional shape recording memory 312.
[0169]
As described above, according to the three-dimensional shape measuring apparatus of the present embodiment, the distance is obtained from the spectral intensity attenuation rate of the one-dimensional spatial frequency spectrum in the horizontal direction, so that the calculation cost can be efficiently calculated while minimizing performance degradation. Can be reduced.
[0170]
(Embodiment 5)
The three-dimensional shape measuring apparatus according to the fifth embodiment using the variable focal position imaging apparatus using the variable optical system will be described below.
[0171]
FIG. 22 is a diagram showing a schematic configuration of a focal position variable photographing apparatus using the variable optical system of the present embodiment. As shown in FIG. 22, the variable focal position imaging device of this embodiment includes an imaging optical system 2201, an image sensor 2202, shutter means 2203, and transmission means 2204.
[0172]
The imaging optical system 2201 is an optical system whose focal position changes when the movable part is driven. The image sensor 2202 is an element that detects an image to be captured. The shutter unit 2203 is a shutter having a two-step pressing depth. The transmission unit 2204 is a unit that drives the movable part of the imaging optical system 2201 according to the pressed depth of the shutter of the shutter unit 2203.
[0173]
(1) in FIG. 22 shows a state before the shutter is pressed, and the imaging optical system 2201 and the image sensor 2202 are combined to constitute one photographing unit using a variable optical system for the focal position.
[0174]
The operation of this apparatus will be described. When the shutter unit 2203 is pressed, the configuration shown in (2) of FIG. By performing the first photographing in this state, the processing in step 801 of the second embodiment in which the photographed image 807 photographed by the photographing means 701 with a far focal position is input can be performed.
[0175]
When the shutter unit 2203 is further pushed in, the image sensor 2202 is moved further away from the imaging optical system 2201 by the transmission unit 2204. The form at this time is shown in FIG. As a result, the imaging optical system 2201 and the image sensor 2202 are combined to form a photographing optical system with a close focal position. Therefore, the second photographing is performed in this state, and the photographing means 702 with a close focal position is used for photographing. The process of step 802 of the second embodiment for inputting the captured image 810 can be performed.
[0176]
By using the variable focal position imaging device of this embodiment, no dedicated power is required to move the focal position, variable focal imaging can be performed naturally and at high speed without forcing the photographer to perform special operations, etc. Produces benefits.
Next, a more specific configuration and operation of the focal position variable photographing apparatus of the present embodiment will be described.
[0177]
FIG. 23 is a diagram showing a specific example of the variable focal position imaging device using the variable optical system of the present embodiment. A shutter button 2301 is a button for operating the shutter of the variable focus position photographing apparatus of the present embodiment. A signal transmitter 2303 is a transmitter that transmits a signal when the shutter button 2301 reaches the depth 2302 of the first stage. A signal transmitter 2305 is a transmitter that transmits a signal when the shutter button 2301 reaches the depth 2304 of the second stage. Note that the signal transmitter 2303 and the signal transmitter 2305 are not active elements, and may be switches that simply perform electrical ON / OFF. The link mechanism 2306 is a mechanism that transmits the amount of movement to the imaging lens system 2307 when the shutter button 2301 is pushed from the depth 2302 to the depth 2304. The imaging lens system 2307 has a function of forming a photographed image on the image sensor 2308.
[0178]
Note that the configuration of the link system is not limited to this embodiment, and the relative distance between the imaging lens system 2307 and the image sensor 2308 is changed physically or optically by transmitting the movement amount of the shutter button. If it can be configured, it can be regarded as a transmission means for driving the variable optical system of the focal position of this embodiment. Needless to say, the focal position can also be changed by moving the position of the image sensor 2308 rather than the imaging lens system 2307.
[0179]
Next, the operation of this apparatus will be described. FIG. 23 (1) shows a state where the shutter button 2301 is not pressed. At this time, the imaging lens system 2307 is at a position 2309, and in this state, the focal position of the lens is far away, so that a far object is in focus.
[0180]
(2) in FIG. 23 shows a state in which the shutter button 2301 is pushed to the first depth 2302. An imaging signal is transmitted by the signal transmitter 2303 so that imaging can be performed. At this time, since the relative positions of the imaging lens system 2307 and the image sensor 2308 are not changed from (1), shooting is performed in a state where a distant object is in focus.
[0181]
(3) in FIG. 23 shows a state in which the shutter button 2301 is pressed to the second depth 2304. When the shutter button 2301 is pushed down to the depth 2304, the movement is transmitted to the imaging lens system 2307 through the link mechanism 2306, and the imaging lens system 2307 is moved forward to the position 2310. In this state, the focal position of the lens is close, and a nearby object is in focus. In this state, when a photographing signal is transmitted by the signal transmitter 2305 and photographing is performed, photographing is performed in a state where a nearby object is in focus.
[0182]
As described above, in the variable focal position imaging device of the present embodiment, it is possible to perform imaging at two different focal positions with one imaging means without requiring special power, and it is normal without forcing the photographer to perform special operations. It is possible to provide the convenience that two images with different focal positions can be continuously captured by simply pressing the shutter button.
[0183]
As described above, according to the three-dimensional shape measurement apparatus of this embodiment, the focal position is changed by driving the movable part of the optical system in accordance with the pressed depth of the shutter. It is possible to change the focal position at high speed by simple operation.
[0184]
【The invention's effect】
By the method and apparatus of the present invention, it is possible to provide a small-sized, low-cost, low-power-consumption, three-dimensional shape measuring apparatus with no limitation on measurement environment and measurement range.
By using the method and apparatus of the present invention, a region that could not be measured stably by the conventional method can be obtained stably by generating an interpolation from the measured region.
With the method and apparatus of the present invention, it is possible to provide a measuring apparatus with lower calculation cost and quick calculation time, particularly when only distance information that changes in the horizontal direction is required.
With the apparatus of the present invention, it is possible to provide an apparatus that can continuously capture two images with different focal positions by simply pressing the shutter by the photographer.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a schematic configuration of a three-dimensional shape measuring apparatus according to a first embodiment.
FIG. 2 is a diagram illustrating a processing outline of the three-dimensional shape measurement apparatus according to the first embodiment.
FIG. 3 is a diagram illustrating a specific example of the three-dimensional shape measurement apparatus according to the first embodiment.
4 is a flowchart showing a processing procedure of the three-dimensional shape measuring apparatus shown in FIG. 3 according to the first embodiment.
FIG. 5 is a diagram illustrating an outline of a pixel adjacency condition according to the first embodiment.
FIG. 6 is a flowchart illustrating a processing procedure of attenuation rate → distance reference table creation processing according to the first exemplary embodiment.
7 is a diagram showing a schematic configuration of a three-dimensional shape measuring apparatus according to Embodiment 2. FIG.
FIG. 8 is a diagram illustrating a processing outline of a three-dimensional shape measurement process according to the second embodiment.
FIG. 9 is a diagram illustrating a specific example of the three-dimensional shape measurement apparatus according to the second embodiment.
10 is a flowchart showing a processing procedure of the three-dimensional shape measuring apparatus shown in FIG. 9 according to the second embodiment.
FIG. 11 is a flowchart illustrating a processing procedure of attenuation rate ratio → distance reference table creation processing according to the second embodiment;
FIG. 12 is a diagram illustrating a schematic configuration of a three-dimensional shape measurement apparatus according to a third embodiment.
FIG. 13 is a diagram illustrating a relationship between a normal direction of a boundary portion where the luminance changes and a spatial frequency spectrum according to the third embodiment.
FIG. 14 is a diagram illustrating a processing outline of a three-dimensional shape measurement process according to the third embodiment.
FIG. 15 is a diagram illustrating a specific example of the three-dimensional shape measurement apparatus according to the third embodiment.
FIG. 16 is a flowchart illustrating a processing procedure of a three-dimensional shape measurement process according to the third embodiment.
FIG. 17 is a diagram illustrating a schematic configuration of a three-dimensional shape measuring apparatus according to a fourth embodiment.
FIG. 18 is a diagram illustrating a relationship between a luminance changing direction and a horizontal one-dimensional spatial frequency spectrum according to the fourth embodiment.
FIG. 19 is a diagram illustrating a processing outline of a three-dimensional shape measurement process according to the fourth embodiment.
FIG. 20 is a diagram illustrating a specific example of the three-dimensional shape measurement apparatus according to the fourth embodiment.
FIG. 21 is a flowchart illustrating a processing procedure of a three-dimensional shape measurement process according to the fourth embodiment.
FIG. 22 is a diagram illustrating a schematic configuration of a focal position variable imaging apparatus using a variable optical system according to a fifth embodiment.
FIG. 23 is a diagram illustrating a specific example of a focal position variable imaging apparatus using a variable optical system according to a fifth embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 101 ... Imaging | photography means, 102 ... Image input means, 103 ... Spatial frequency spectrum calculation means, 104 ... Spectral intensity attenuation rate calculation means, 105 ... Distance calculation means, 106 ... Interpolation calculation means, 206 ... Captured image, 207 ... Close distance Area 208: Distant area 209 ... Contour area extraction data 210 ... Points 211 ... Spatial frequency spectrum 212 ... Area 213 ... Attenuation rate-> Distance reference table 214 and 215 ... Pixel, 301 ... Shooting camera, 302 ... Image input unit, 303 ... Image recording memory, 304 ... Edge detector, 305 ... Luminance change area recording memory, 306 ... Spatial frequency spectrum calculator, 307 ... Spatial frequency spectrum recording memory, 308 ... Maximum distribution direction detector, 309 ... Spectral intensity attenuation rate calculator, 310 ... Distance calculator, 311 ... Interpolator, DESCRIPTION OF SYMBOLS 12 ... Three-dimensional shape recording memory, 313 ... Shooting object, 314 ... Focus position, 315 ... Shooting image data, 316 ... Contour region extraction data, 317 ... Spatial frequency spectrum, 318 ... Maximum distribution direction, 319 ... Interpolation schematic diagram, 320 ... Origin, 321-324 ... Pixel, 325 ... Attenuation rate → Distance reference table, 608 ... Calibration imaging plate, 701 and 702 ... Imaging means, 703 ... Image input means, 704 ... Spatial frequency spectrum calculation means, 705 ... Spectral intensity Attenuation rate calculation means, 706 ... distance calculation means, 707 ... interpolation calculation means, 807 ... photographed image, 808 ... area far away, 809 ... area near distance, 810 ... photographed image, 811 ... area near distance, 812 ... Distant area, 813 ... contour area extraction data, 814 and 815 ... spatial frequency spectrum, 816 ... area, 17 and 818 ... attenuation rate, 819 ... direction, 820 ... attenuation rate ratio → distance reference table, 901 and 902 ... photographing camera, 903 ... image input unit, 904 and 905 ... image recording memory, 906 ... edge detector, 907 ... Luminance change area recording memory, 908... Spatial frequency spectrum calculator, 909 and 910... Spatial frequency spectrum recording memory, 911... Maximum distribution direction detector, 912 and 913 ... Spectral intensity attenuation rate calculator, 914. ... Interpolation calculator, 916 ... Three-dimensional shape recording memory, 917 ... Shooting object, 918 and 919 ... Focus position, 920 and 921 ... Image data, 922 ... Contour area extraction data, 923 and 924 ... Contour area, 925 ... Pixel, 926 and 927 ... spatial frequency spectrum, 928 ... direction, 929 ... half mirror, 9 DESCRIPTION OF SYMBOLS 30 ... Mirror, 931 ... Interpolation schematic diagram, 932 ... Origin, 933-936 ... Pixel, 937 ... Attenuation rate ratio → Distance reference table, 1109 ... Calibration plate, 1201 and 1202 ... Shooting means, 1203 ... Image input means, 1204 ... Normal direction one-dimensional spatial frequency spectrum calculating means 1205 ... Spectral intensity attenuation rate calculating means 1206 ... Distance calculating means 1207 ... Interpolating calculating means 1301 ... Photographed image 1302 ... Luminance change boundary region 1303 ... Luminance Normal direction of change boundary portion 1304 ... Spatial frequency spectrum, 1305 ... Arrow, 1501 ... Edge vector detector, 1502 ... Luminance change region recording memory, 1503 ... Spatial frequency spectrum calculator, 1504 ... Spatial frequency spectrum recording memory, 1505 ... Outline region extraction data, 1506... Normal vector, 150 ... Spatial frequency spectrum, 1701 ... Imaging means, 1702 ... Image input means, 1703 ... Horizontal one-dimensional spatial frequency spectrum calculation means, 1704 ... Spectral intensity attenuation rate calculation means, 1705 ... Distance calculation means, 1706 ... Interpolation calculation means, 1801 ... taken image, 1802 ... luminance change boundary region, 1803 ... spatial frequency spectrum, 1804 ... attenuation factor, 1805 ... taken image, 1806 ... luminance change boundary region, 1807 ... spatial frequency spectrum, 2001 ... horizontal one-dimensional spatial frequency Spectrum calculator 2002 ... Spatial frequency spectrum recording memory 2003 ... Spectral intensity attenuation factor calculator 2004 ... Spatial frequency spectrum 2201 ... Imaging optical system 2202 ... Imaging element 2203 ... Shutter means 2204 ... Transmission means 2301 ... Shutter button , 2302 ... depth, 2303 ... signal transmitter, 2304 ... depth, 2305 ... signal transmitter, 2306 ... link mechanism 2307 ... imaging lens system, 2308 ... imaging element, 2309 and 2310 ... position.

Claims (14)

A first step of inputting image captured by the shadow unit shooting,
A second step of obtaining a two-dimensional spatial frequency spectrum distribution of luminance in a neighboring region centered on a pixel in the region for a region where the luminance change in the captured image is equal to or greater than a predetermined value ;
For the two-dimensional spatial frequency spectrum distribution, and detects the direction from the center of the two-dimensional spatial frequency spectrum distribution attenuation factor is minimized to a high frequency region from the DC component of the spectrum intensity with respect to the spatial frequency, the attenuation rate in the direction A third step for determining
A fourth method for obtaining the distance between the subject in the area and the photographing unit based on the attenuation rate obtained in the third step and a table in which the correspondence between the distance between the subject and the photographing unit and the attenuation factor is recorded in advance . And the steps
For an area where the luminance change in the captured image is less than the predetermined value , the distance between the subject in the area and the imaging means by interpolation from the area in which the distance between the subject and the imaging means is obtained in the fourth step. And a fifth step for obtaining the three-dimensional shape measurement method. A first photographed image photographed by the first photographing means and a second photographed image photographed by the second photographing means having a focal position different from the focal position of the first photographing means are respectively input. A first step;
For a region where at least one of the luminance changes in the first and second photographed images is greater than or equal to a predetermined value , the two-dimensional luminance of the neighboring region centered on the pixel in the region of the photographed image having the larger luminance change spatial frequency spectrum distribution and the other captured each of the first two-dimensional spatial frequency spectrum distribution of the luminance of the neighboring region centering pixels in the area of the image of the two-dimensional spatial frequency spectrum distribution and the second two-dimensional spatial frequency spectrum A second step for obtaining a distribution ;
For the first two-dimensional spatial frequency spectrum distribution , a direction from the center of the two-dimensional spatial frequency spectrum distribution where the attenuation rate from the direct current component of the spectral intensity with respect to the spatial frequency to the high frequency region is minimized is detected in the direction. It obtains the attenuation factor as a first attenuation factor, for the second two-dimensional spatial frequency spectrum distribution, a third step of obtaining the attenuation factor in the same direction as the direction as the second attenuation factor,
The ratio between the first attenuation factor and the second attenuation factor obtained in the third step is obtained as an attenuation factor ratio, the attenuation factor ratio, the distance between the subject and the first or second photographing means, and the A fourth step of obtaining a distance between the subject in the area and the first or second imaging means based on a table in which the correspondence relationship with the attenuation rate ratio is recorded in advance ;
From the area where the distance between the subject and the first or second photographing means is obtained in the fourth step for the area where the luminance change in the first and second photographed images is both less than the predetermined value. And a fifth step of obtaining a distance between the subject in the region and the first or second photographing means by interpolation of the three-dimensional shape measurement method. A first step of inputting image captured by the shadow unit shooting,
A second step of obtaining a normal direction of a boundary portion of the luminance change and a one-dimensional spatial frequency spectrum distribution of luminance in the normal direction for a region where the luminance change in the captured image is equal to or greater than a predetermined value;
For the previous SL one-dimensional spatial frequency spectrum distribution, a third step of obtaining the attenuation factor of the DC component of the spectral intensity with respect to the spatial frequency to a high frequency region,
A fourth method for obtaining the distance between the subject in the area and the photographing unit based on the attenuation rate obtained in the third step and a table in which the correspondence between the distance between the subject and the photographing unit and the attenuation factor is recorded in advance . And the steps
For an area where the luminance change in the captured image is less than the predetermined value , the distance between the subject in the area and the imaging means by interpolation from the area in which the distance between the subject and the imaging means is obtained in the fourth step. And a fifth step for obtaining the three-dimensional shape measurement method. A first photographed image photographed by the first photographing means and a second photographed image photographed by the second photographing means having a focal position different from the focal position of the first photographing means are respectively input. A first step;
For a region where at least one of the luminance changes in the first and second captured images is equal to or greater than a predetermined value, the normal direction of the boundary portion of the luminance change of the captured image having the larger luminance change is obtained, and the luminance change one-dimensional spatial frequency spectrum distribution and the other of the normal line direction of the luminance of the captured image the one-dimensional spatial frequency spectrum distribution of each of the first one-dimensional spatial frequency spectrum of but luminance normal line direction of the larger side of the captured image A second step for obtaining a distribution and a second one-dimensional spatial frequency spectrum distribution ;
A third step for obtaining the first and second attenuation factors for the first and second one-dimensional spatial frequency spectrum distributions , respectively, from the direct current component of the spectral intensity with respect to the spatial frequency to the high frequency region ;
The ratio between the first attenuation factor and the second attenuation factor obtained in the third step is obtained as an attenuation factor ratio, the attenuation factor ratio, the distance between the subject and the first or second photographing means, and the A fourth step of obtaining a distance between the subject in the area and the first or second imaging means based on a table in which the correspondence relationship with the attenuation rate ratio is recorded in advance ;
From the area where the distance between the subject and the first or second photographing means is obtained in the fourth step for the area where the luminance change in the first and second photographed images is both less than the predetermined value. And a fifth step of obtaining a distance between the subject in the region and the first or second photographing means by interpolation of the three-dimensional shape measurement method. A first step of inputting image captured by the shadow unit shooting,
A second step of obtaining a one-dimensional spatial frequency spectrum distribution of horizontal luminance in a region where the luminance change in the captured image is equal to or greater than a predetermined value;
For the previous SL one-dimensional spatial frequency spectrum distribution, a third step of obtaining the attenuation factor of the DC component of the spectral intensity with respect to the spatial frequency to a high frequency region,
A fourth method for obtaining the distance between the subject in the area and the photographing unit based on the attenuation rate obtained in the third step and a table in which the correspondence between the distance between the subject and the photographing unit and the attenuation factor is recorded in advance . And the steps
For an area where the luminance change in the captured image is less than the predetermined value , the distance between the subject in the area and the imaging means by interpolation from the area in which the distance between the subject and the imaging means is obtained in the fourth step. And a fifth step for obtaining the three-dimensional shape measurement method. A first photographed image photographed by the first photographing means and a second photographed image photographed by the second photographing means having a focal position different from the focal position of the first photographing means are respectively input. A first step;
In a region where at least one of the luminance changes in the first and second photographed images is equal to or greater than a predetermined value , the one-dimensional spatial frequency spectrum distribution of the horizontal luminance of the photographed image having the larger luminance change and the other photographed image a second step of determining a horizontal one-dimensional spatial frequency spectrum distribution and the second one-dimensional spatial frequency spectrum distribution first respective one-dimensional spatial frequency spectrum distribution of the luminance of,
A third step for obtaining the first and second attenuation factors for the first and second one-dimensional spatial frequency spectrum distributions , respectively, from the direct current component of the spectral intensity with respect to the spatial frequency to the high frequency region ;
The ratio between the first attenuation factor and the second attenuation factor obtained in the third step is obtained as an attenuation factor ratio, the attenuation factor ratio, the distance between the subject and the first or second photographing means, and the A fourth step of obtaining a distance between the subject in the area and the first or second imaging means based on a table in which the correspondence relationship with the attenuation rate ratio is recorded in advance ;
From the area where the distance between the subject and the first or second photographing means is obtained in the fourth step for the area where the luminance change in the first and second photographed images is both less than the predetermined value. And a fifth step of obtaining a distance between the subject in the region and the first or second photographing means by interpolation of the three-dimensional shape measurement method. Image input means for inputting the image captured by the shadow unit shooting,
The region change in luminance than the predetermined value in the captured image, a spectrum calculation means for obtaining a two-dimensional spatial frequency spectrum distribution of the luminance of the neighboring region centering pixels within the region,
For the two-dimensional spatial frequency spectrum distribution , a direction from the center of the two-dimensional spatial frequency spectrum distribution where the attenuation rate from the direct current component of the spectral intensity with respect to the spatial frequency to the high frequency region is minimized is detected, and the attenuation rate in the direction is detected. A decay rate calculating means for obtaining
Distance calculation for determining the attenuation rate obtained by the attenuation rate calculating means and the distance between the subject and the photographing means in the area based on a table in which the correspondence between the distance between the subject and the photographing means and the attenuation ratio is recorded in advance. Means,
For region luminance change is smaller than the predetermined value in the captured image, the distance between the imaging means and the object of the area by interpolation from the region where the distance is determined between the imaging means and the object in the distance calculating means A three-dimensional shape measuring apparatus, comprising: a required interpolation calculation means. A first photographed image photographed by the first photographing means and a second photographed image photographed by the second photographing means having a focal position different from the focal position of the first photographing means are respectively input. Image input means;
For a region where at least one of the luminance changes in the first and second photographed images is greater than or equal to a predetermined value , the two-dimensional luminance of the neighboring region centered on the pixel in the region of the photographed image having the larger luminance change spatial frequency spectrum distribution and the other captured each of the first two-dimensional spatial frequency spectrum distribution of the luminance of the neighboring region centering pixels in the area of the image of the two-dimensional spatial frequency spectrum distribution and the second two-dimensional spatial frequency spectrum Spectrum calculation means to obtain as distribution ;
For the first two-dimensional spatial frequency spectrum distribution , a direction from the center of the two-dimensional spatial frequency spectrum distribution where the attenuation rate from the direct current component of the spectral intensity with respect to the spatial frequency to the high frequency region is minimized is detected in the direction. obtains the attenuation factor as a first attenuation factor, for the second two-dimensional spatial frequency spectrum distribution, the attenuation factor calculating means for obtaining the attenuation factor in the same direction as the direction as the second attenuation factor,
The ratio of the first attenuation rate and the second attenuation rate obtained by the attenuation rate calculating means is obtained as an attenuation rate ratio, and the attenuation rate ratio, the distance between the subject and the first or second imaging means, and the Distance calculating means for obtaining a distance between the subject in the area and the first or second imaging means based on a table in which the correspondence relationship with the attenuation rate ratio is recorded in advance ;
For areas where the luminance changes in the first and second photographed images are both less than the predetermined value, from the area where the distance between the subject and the first or second photographing means is obtained by the distance calculating means . 3. A three-dimensional shape measuring apparatus comprising: an interpolation calculation unit that obtains a distance between a subject in the region and the first or second imaging unit by interpolation. Image input means for inputting the image captured by the shadow unit shooting,
For region luminance change is equal to or more than a predetermined value in the captured image, a spectrum calculation means for obtaining a one-dimensional spatial frequency spectrum distribution of brightness in the normal direction and normal line direction of the boundary of the luminance change,
For the previous SL one-dimensional spatial frequency spectrum distribution, the attenuation factor calculating means for obtaining the attenuation factor of the DC component of the spectral intensity with respect to the spatial frequency to a high frequency region,
Distance calculation for determining the attenuation rate obtained by the attenuation rate calculating means, and the distance between the subject and the photographing means in the area based on a table in which the correspondence between the distance between the subject and the photographing means and the attenuation ratio is recorded in advance. Means,
For region luminance change is smaller than the predetermined value in the captured image, the distance between the imaging means and the object of the area by interpolation from the region where the distance is determined between the imaging means and the object in the distance calculating means A three-dimensional shape measuring apparatus, comprising: a required interpolation calculation means. A first photographed image photographed by the first photographing means and a second photographed image photographed by the second photographing means having a focal position different from the focal position of the first photographing means are respectively input. Image input means;
In a region where at least one of the luminance changes in the first and second photographed images is equal to or greater than a predetermined value, the normal direction of the boundary of the luminance change of the photographed image having the larger luminance change is obtained, and the luminance change one-dimensional spatial frequency spectrum distribution and the other of the normal line direction of the luminance of the captured image the one-dimensional spatial frequency spectrum distribution of each of the first one-dimensional spatial frequency spectrum of but luminance normal line direction of the larger side of the captured image Spectrum calculating means for obtaining the distribution and the second one-dimensional spatial frequency spectrum distribution ;
Attenuation rate calculating means for obtaining the first and second attenuation factors as the first and second attenuation factors for the first and second one-dimensional spatial frequency spectrum distributions , respectively, from the direct current component of the spectral intensity to the spatial frequency to the high frequency region ;
The ratio of the first attenuation rate and the second attenuation rate obtained by the attenuation rate calculating means is obtained as an attenuation rate ratio, and the attenuation rate ratio, the distance between the subject and the first or second imaging means, and the Distance calculating means for obtaining a distance between the subject in the area and the first or second imaging means based on a table in which the correspondence relationship with the attenuation rate ratio is recorded in advance ;
For areas where the luminance changes in the first and second photographed images are both less than the predetermined value, from the area where the distance between the subject and the first or second photographing means is obtained by the distance calculating means . 3. A three-dimensional shape measuring apparatus comprising: an interpolation calculation unit that obtains a distance between a subject in the region and the first or second imaging unit by interpolation. Image input means for inputting the image captured by the shadow unit shooting,
Change in luminance for an area equal to or greater than a predetermined value in said captured image, a spectrum calculation means for obtaining the one-dimensional spatial frequency spectrum distribution in the horizontal direction of the luminance,
For the previous SL one-dimensional spatial frequency spectrum distribution, the attenuation factor calculating means for obtaining the attenuation factor of the DC component of the spectral intensity with respect to the spatial frequency to a high frequency region,
Distance calculation for determining the attenuation rate obtained by the attenuation rate calculating means and the distance between the subject and the photographing means in the area based on a table in which the correspondence between the distance between the subject and the photographing means and the attenuation ratio is recorded in advance. Means,
For region luminance variation is smaller than the predetermined value in the captured image, the distance between the imaging means and the object of the area by interpolation from the region where the distance is determined between the imaging means and the object in the distance calculating means A three-dimensional shape measuring apparatus having an interpolation calculation means to be obtained. A first photographed image photographed by the first photographing means and a second photographed image photographed by the second photographing means having a focal position different from the focal position of the first photographing means are respectively input. Image input means;
In a region where at least one of the luminance changes in the first and second photographed images is equal to or greater than a predetermined value , the one-dimensional spatial frequency spectrum distribution of the luminance in the horizontal direction of the photographed image having the larger luminance change and the other photographed image a spectrum calculating means for obtaining a one-dimensional spatial frequency spectrum distribution in the horizontal direction of the luminance as a first one-dimensional spatial frequency spectrum distribution and the second one-dimensional spatial frequency spectrum distribution each,
Attenuation rate calculating means for obtaining the first and second attenuation factors as the first and second attenuation factors for the first and second one-dimensional spatial frequency spectrum distributions , respectively, from the direct current component of the spectral intensity to the spatial frequency to the high frequency region ;
The ratio of the first attenuation rate and the second attenuation rate obtained by the attenuation rate calculating means is obtained as an attenuation rate ratio, and the attenuation rate ratio, the distance between the subject and the first or second imaging means, and the Distance calculating means for obtaining a distance between the subject in the area and the first or second imaging means based on a table in which the correspondence relationship with the attenuation rate ratio is recorded in advance ;
For areas where the luminance changes in the first and second photographed images are both less than the predetermined value, from the area where the distance between the subject and the first or second photographing means is obtained by the distance calculating means . 3. A three-dimensional shape measuring apparatus comprising: an interpolation calculation unit that obtains a distance between a subject in the region and the first or second imaging unit by interpolation.   A program for causing a computer to execute the steps according to any one of claims 1 to 6. A computer-readable recording medium characterized by recording a program according to claim 1 3.

Images