JP2006023133A - Instrument and method for measuring three-dimensional shape - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To restrain an edge extraction error in a photographed image by a first camera for conducting observation from the direction same to an optical axis of a projector. <P>SOLUTION: A stripe pattern of the projector 10 is read out from a frame memory 110 to be written in a reference stripe pattern memory 111, and the written stripe pattern is corrected by recoding. Stripes are extracted from the pattern image of the first camera 20, and each of the stripe is divided in a longitudinal directional in every stripe width to generate cells. Intensities among the corresponding respective cells are compared sequentially from the center of the image, a new code is generated and allocated when the pattern is varied by a factor such as a reflectance of an object to bring the intensity among the cells into a threshold value ore more, so as to correct the stripe pattern of the reference stripe pattern memory 111. Since an intrinsic edge is set in the stripe pattern, and since it is not varied by a surface shape or texture of the object, accurate edge coordinates are acquired to be free from an edge detection error. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a three-dimensional shape measurement technique for acquiring distance information to a target object using a pattern projection method.

  As a method for measuring the shape of an object, a method called a pattern projection method is used in which a reference pattern is projected onto an object, and photographing is performed with a CCD camera or the like from a direction different from the direction in which the reference pattern light is projected. There is. The captured pattern is deformed depending on the shape of the object. By associating the photographed deformation pattern with the projected pattern (or an equivalent photography pattern), three-dimensional measurement of the object can be performed. The problem with the pattern projection method is how to make the correspondence between the deformed pattern and the projected pattern simple and easy to avoid. Various pattern projection methods have been proposed in the past.

For example, a technique disclosed in Patent Document 1 (Japanese Patent No. 3482990) includes a projector that projects a coded pattern, a first camera that captures a projection pattern from the optical axis direction of the projector, and an optical axis of the projector. A second camera that captures a projection pattern from a direction different from the direction, and assigns a new code to an area where the amount of change in the captured pattern by the first camera relative to the projection pattern is a predetermined value or more, and uses the allocated code A three-dimensional image photographing apparatus configured to generate first distance information from a photographing pattern by a second camera and obtain a three-dimensional image based on the first distance information and luminance information obtained from the first camera. is there. Three-dimensional measurement can be performed with high accuracy by performing re-encoding using an imaging pattern obtained by imaging a projection pattern with a first camera placed on the same optical axis as the projector.
Japanese Patent No. 3482990

  By the way, in the method using such a projection pattern, the coordinates of the stripe boundary are obtained by the principle of triangulation from the positional relationship between the camera and the projector. Specifically, the edge of the recoded image obtained from the first camera and the edge of the measurement image obtained from the second camera are extracted, and pixel information (luminance value / hue) on both sides of the edge is extracted. The distance is calculated from the principle of triangulation by acquiring each pixel and assuming that the pixel having the same pixel information on the epipolar line is a corresponding point.

  However, in the case of the above method, edge extraction is performed from the camera image, but since edge extraction is specified from the luminance value and hue difference on both sides of the edge, the luminance value on both sides of the edge is determined from the texture information of the subject. In the case of an image observed with a small hue difference, edge extraction becomes difficult. In addition, a camera generally uses a solid-state image sensor such as a CCD. Since a CCD or the like is configured with a certain resolution, an observed image is quantized, and it is difficult to extract an edge with accuracy in units of pixels or less. For this reason, the edge position of each of the first and second camera images is shifted due to subject information and CCD sub-pixel noise, so that there is a problem that the distance calculation accuracy is lowered.

  The present invention has been made to solve such a problem, and in a three-dimensional shape measurement method having a first camera for re-encoding from the same optical axis using a projection pattern of a plurality of levels, An object is to reduce edge extraction noise and improve distance calculation accuracy.

  In the present invention, in order to solve the above problem, the edge extraction accuracy of the image observed by the first camera is improved, and the distance calculation accuracy is improved.

  That is, according to the principle configuration example of the present invention, a plurality of levels of stripe pattern light are projected onto a subject (object) by a projection device (also referred to as a projector or a projector). The first camera captures a projected image from the same direction as the optical axis of the projector. The second camera captures a projected image from a direction deviating from the optical axis of the projection apparatus. Since the first camera is arranged in the same optical axis direction as the projection device (at the same principal point), the stripe pattern shape of the image observed by the first camera is in principle the same as the stripe pattern projected from the projector. Only the luminance and hue values are changed by the object surface information. The stripe pattern shape itself is the same or corresponds between the image projected from the light projecting device and the image captured by the first camera, so that the light projecting device does not re-code the captured image of the first camera. If the stripe pattern is recoded, the problem of the edge extraction error of the image captured by the first camera is eliminated. In other words, the image of the stripe pattern projected from the projection apparatus is corrected by adding the change in luminance and hue value observed by the first camera while maintaining the position of the stripe edge. The distance is measured by triangulation from the positional relationship between the corrected stripe pattern and the image observed by the second camera.

  In this configuration, since the stripe pattern projected from the projection device is used as the reference stripe pattern, the edge position is known in advance and is not affected by the shape, texture, etc. of the subject. Problems caused by edge extraction errors in the captured image are solved.

  According to another exemplary configuration of the principle of the present invention, calibration is performed between the projection pattern of the projection apparatus and the image observed by the first camera, and the stripe of the image observed by the first camera is obtained. Edges are extracted and stored as templates. During distance measurement, the brightness / hue value of the projection pattern changes depending on the subject information, etc., and an error occurs in edge extraction from the observed image of the first camera. Using only the hue change actually observed by the first camera, the distance is measured by triangulation from the positional relationship between the image synthesized in this way and the image observed by the second camera.

  In this configuration, the edge extraction error is suppressed during calibration to extract the edge of the photographed image of the first camera, and this edge is also used for distance measurement. Therefore, the edge extraction error of the photographed image of the first camera is used. It is possible to suppress problems caused by the problem.

  Further, the present invention will be described.

  According to the present invention, a stripe pattern coded with a plurality of levels of brightness set in advance within a predetermined brightness range is projected onto a subject by a projector, and the same direction as or the same as the optical axis of the projector is projected by a first camera. A projected image is taken from the principal point, the projected image is taken from a direction deviated from the optical axis of the projector by a second camera, and each region (stripe is divided in the length direction) of the taken image of the first camera The recaptured partial area (also referred to as a cell) based on the change amount of the stripe pattern with respect to the stripe pattern. The point information on the edge of the adjacent stripe of the image captured by the first camera and the edge of the adjacent stripe of the image captured by the second camera are associated with the code information of the image captured by the first camera. Triangular in surveying three-dimensional shape measurement device which performs computing the distance to the subject, and it is assumed that predetermined each of said first camera edge coordinates of adjacent stripes of captured images with the.

  The predetermined luminance range is, for example, a range from all black to all white. The plurality of luminance levels are preferably set in consideration of gradation reproduction characteristics of the projector and the camera, and are set at substantially equal intervals.

  In this configuration, the edge information of the image captured by the first camera is accurately obtained separately from the image obtained by capturing the stripe pattern image of a plurality of levels. It is possible to avoid the deterioration of the distance calculation accuracy due to the edge extraction error.

  In this configuration, the coordinates of the edge are fixedly associated with the physical positions of the components of the first camera, for example. For example, it can be a predetermined line between cells of the CCD element of the first camera.

  Further, when projecting a stripe pattern for calibration onto a reference subject by the projector, capturing a projected image with the second camera, detecting each edge coordinate of an adjacent stripe from the captured image, and calculating distance information The edge coordinates of the adjacent stripe of the image captured by the first camera may be used.

In this case, it is preferable that the reference subject has a plane having the same luminance.
The calibration stripe pattern includes a line segment drawn at a position corresponding to the edge of the stripe pattern at the time of measurement, and two levels of stripes alternately corresponding to each stripe of the stripe pattern at the time of measurement. It is an arrangement.

  The present invention can be realized not only as an apparatus or a system but also as a method. Of course, a part of the invention can be configured as software. Of course, software products used for causing a computer to execute such software are also included in the technical scope of the present invention.

  These and other aspects of the invention are set forth in the appended claims and will be described in detail below with reference to examples.

  According to the present invention, the edge information of the stripe pattern used as a reference is known in advance or is extracted while suppressing errors during calibration. Therefore, if the edge is extracted from the captured image of the first camera during distance measurement, Thus, it is possible to avoid deterioration in distance calculation accuracy due to edge extraction errors that occur.

  An embodiment of a three-dimensional image measuring apparatus according to the present invention will be described.

  FIG. 1 is a block diagram of a three-dimensional image measuring apparatus according to an embodiment of the present invention. In FIG. 1, the three-dimensional image measuring apparatus projects a pattern for three-dimensional measurement (for example, a liquid crystal projector) 10. An imaging device (for example, a CCD camera, also referred to as a first camera) 20, a triangulation imaging device (for example, a CCD camera, also referred to as a second camera) 30, other calculation processing devices (not shown), etc. Composed. Reference numeral 40 denotes a half mirror, and reference numeral 50 denotes an object. The basic configuration of this three-dimensional measuring apparatus is the same as the configuration disclosed in Patent Document 1 (Japanese Patent No. 3482990). The pattern projection apparatus 10 uses a liquid crystal projector, a DLP (trademark) projector, or a slide projector. The projection pattern input to the pattern projection apparatus 10, for example, a liquid crystal projector, uses a stripe pattern with shading as shown in FIG. 2, and projects the pattern onto an object (object) shown on the right side of FIG. When using a slide projector, the projection pattern is formed on a slide film, or a metal film or the like is vapor-deposited on a glass pattern, and the transmittance is controlled by the film thickness, the retinal point pattern, or the like.

  FIG. 3 shows a luminance profile in the horizontal direction of pattern data input to a liquid crystal projector, for example. The projection pattern (pattern data) is a combination of seven types of luminance stripes in which 256 gradations are divided into six levels, and the adjacent stripes are arranged so as not to have a one-level level difference. This is equivalent to dividing 256 gradations into three levels (four levels of 0, 85, 170, and 255) as far as the level difference between adjacent stripes is concerned. In the case of repeating a normal four-level stripe into one set, the limit is to set one set to 12 stripes so that each combination of adjacent stripes does not appear in one set. Repeatedly, a stripe pattern including a large number of stripes is formed. If the same combination of adjacent stripes overlaps in the vicinity, it causes a code search error. On the other hand, when 7 levels of stripes are used and the adjacent stripes are arranged so as not to have a one-level level difference as in this embodiment, the same combination of adjacent stripes appears in one set. In order to avoid this, the number of stripes in one set can be about 28. In the range of at least 28 stripes, combinations of the same adjacent stripes do not overlap, and the probability of code search errors also decreases. Of course, when the code search error does not become a problem, the level difference between adjacent stripes may be made one step.

  FIG. 4 shows a pattern projection pattern. An imaging device (first camera 20) and a projection device (pattern projection device 10) are arranged on the same optical axis by a half mirror 40 or the like, an imaging device is prepared for triangulation measurement, and a stripe pattern as shown in FIG. 2 is projected. To do. Re-encoding is performed from an image (first camera image) observed by the image sensor (first camera 20) of the same optical axis, and an image (second image) observed by the measurement image sensor (second camera 30). A three-dimensional distance image (distance) is calculated using the camera image.

  FIG. 5 shows a configuration example for calculating a distance image. In this figure, the pattern projection apparatus 10 projects a coded pattern onto the object 50. This pattern is stored in the frame memory 110. A projection pattern on the object 50 is imaged by the first camera 20 for monitoring and the second camera 30 for triangulation, and stored in the pattern image memories 120 and 150, respectively.

  The area dividing unit 130 divides the pattern image stored in the pattern image memory 120 into an area where the projection pattern (light) from the pattern projection apparatus 10 has sufficiently arrived (also referred to as area 2) and an area where it has not reached (also referred to as area 1). To divide. For example, for an area where the intensity difference between adjacent stripes is less than or equal to the threshold, it is determined that the projection pattern has not reached sufficiently, and the projection pattern has sufficiently reached an area where the intensity difference between stripes is greater than or equal to the threshold. Determined as an area. As described below, an edge pixel serving as a boundary line is calculated and a distance calculation is performed on an area where the projection pattern has sufficiently reached. For areas where the projection pattern does not reach sufficiently, distance calculation based on parallax is performed separately. Although not specifically described here, refer to Patent Document 1 for details.

  The recoding unit 160 extracts stripes for the extracted region 2, divides each stripe in the vertical direction for each stripe width, generates a square cell, and recodes the cell. This will be described in detail later.

  The code decoding unit 170 determines the code of each cell (edge) of the pattern image from the second camera 30 for triangulation stored in the pattern image memory 150 using the code from the recoding unit 160. . Thereby, the x coordinate of the pixel at the measurement point p (edge) in the pattern image of the pattern image memory 150 and the irradiation direction (slit angle) θ from the light source are determined, and the distance Z is measured by the equation (1) described later. (See FIG. 14). The three-dimensional image memory 180 stores the distance and the luminance value of the object acquired from the first camera 20 (stored in the luminance value memory 140) as three-dimensional image data.

  Details of the calculation of the three-dimensional shape in this configuration example will be further described.

  First, re-encoding is performed on a reference stripe pattern (a stripe pattern in which triangulation is performed in association with an image captured by the second camera 30). In this configuration example, the reference stripe pattern is a stripe pattern stored in the frame memory 110, and recoding is performed on the stripe pattern stored in the frame memory 110. For example, a stripe pattern stored in the frame memory 110 is written into the reference stripe pattern memory 111, and the written stripe pattern is corrected by recoding. A unique edge is set in the stripe pattern, and it does not change depending on the surface shape or texture of the object, so that accurate edge coordinates can be obtained and there is no problem of edge detection error. Instead of storing the stripe pattern in the reference stripe pattern memory 111, the edge information may be stored.

  The recoding is performed according to the flowchart shown in FIG. 7 using the pattern image photographed by the first camera 20 for monitoring on the same optical axis and the pattern image used for light projection. First, the region of the pattern image photographed by the first camera 20 is divided. A region where the intensity difference between adjacent stripes is less than or equal to the threshold is extracted as region 1 where the projection pattern from the pattern projection apparatus 10 has not reached, and a region where the intensity difference between stripes is greater than or equal to the threshold is extracted as region 2. (S10), the edge pixel that becomes the boundary line for the region 2 is calculated.

  Stripes are extracted from the extracted region 2, and each stripe is divided in the vertical direction for each stripe width to generate a square cell. The average value of the intensity is taken for each generated cell, and the average value is set as the intensity of each cell (S11). Compare the intensities between the corresponding cells in order from the center of the image, and if the inter-cell intensity differs by more than a threshold because the pattern has changed due to factors such as the reflectance of the object, the distance to the object, etc. Code generation and allocation are performed (S12 to S16).

  FIG. 8 is a simplified example for simplicity, but the left side of FIG. 8 is a light projection pattern coded by the arrangement of stripes. The intensity is 3 (strong), 2 (medium), 1 (weak) is assigned. The right side of FIG. 8 is obtained by extracting the stripes photographed by the first coaxial camera 20 in the direction perpendicular to the stripes by the cell width. In the example on the upper right in FIG. 8, since the strength changes in the third cell from the left and a new code appears, a new code of 0 is assigned. In the lower right example of FIG. 8, since the existing code appears in the third cell from the left and the second cell from the top, as a new code (232, 131) from the cell sequence (vertical) The code is expressed by (line, horizontal line). This re-encoding is equivalent to projecting a complex pattern such as a two-dimensional pattern at a site where the shape of the object is rich in change, and projecting a simple pattern at a site where there is little change. This process is repeated and recoding is performed by assigning unique codes to all cells.

  As an example, FIGS. 11 and 12 show simplified images obtained by the first camera 20 and the second camera 30 when the pattern of FIG. 10 is projected onto the object of FIG. In this example, FIG. 13 is obtained as a new coded pattern on the surface of the plate.

  Next, stripes are extracted from the stripe image obtained by the second camera 30 and divided into cells as before. For each cell, the code of each cell is detected using the recoded code, and the irradiation direction θ from the light source is calculated based on the detected code. The distance Z is calculated by Equation (1) using θ of the cell to which each pixel belongs, the x coordinate on the image taken by the camera 2, the focal length F and the base line length L which are camera parameters. FIG. 14 shows the relationship among the measurement point p, the irradiation direction θ from the light source, the x coordinate on the image captured by the second camera 30, the focal length F that is a camera parameter, and the baseline length L. .

  Z = FL / (x + Ftanθ) ---- Equation (1)

  This calculation is actually performed using the x-coordinate of the cell boundary. At this time, the x-coordinate is calculated in a unit smaller than the pixel resolution of the camera, thereby improving the measurement accuracy. The x-coordinate value is obtained from the brightness average values d1 and d2 of appropriate pixels of the cells on both sides of the edge pixel calculated in advance and the brightness de of the edge pixel. A pixel position de ′ (indicated by x in the drawing for convenience) corresponding to the luminance de is obtained from a straight line obtained by linear interpolation from the pixel positions p1 and p2 adjacent to the edge pixel and the luminance average values d1 and d2, and this is the x coordinate value. . (See Figure 15)

  Here, the projection pattern is a combination of luminance stripes for each of 42 gradations or 43 gradations obtained by dividing 256 gradations into 6 stages, but the adjacent stripes are arranged so as not to have a level difference of 1 stage. Therefore, the intensity difference between stripes is large compared to a pattern in which luminance stripes simply divided into six levels are combined, and it is easy to calculate the x coordinate value. FIG. 16 shows the luminance distribution of the pixel unit of the camera when the intensity difference between stripes is large and small. The average luminance value of several pixels on both sides of the edge pixel can take a range indicated by the thickness of the gray horizontal line due to the influence of noise of each pixel. As shown on the left in FIG. 16, when the intensity difference is large, the influence of noise on the luminance average value of several pixels on both sides of the edge pixel is expressed by the thickness of each gray line when the calculation method of FIG. 15 is applied. As a result, the x coordinate value is determined within the range of a. However, when the intensity difference is small, the ratio of the noise component of the luminance average value of several pixels on both sides of the edge pixel becomes relatively large with respect to the intensity difference, which causes ambiguity in the calculation of the x coordinate value. The range becomes the range of b, and as a result, the measurement accuracy is lowered. For this reason, it is desirable that the difference in intensity between stripes is large.

  FIG. 6 shows a configuration example for obtaining the x coordinate. In FIG. 6, the edge right neighboring pixel position input unit 210, the edge right cell luminance average value input unit 220, the edge left neighboring pixel position input unit 230, the edge left cell luminance average value input unit 240, and the edge luminance input unit 250 respectively. d1, p1, d2, p2, and de are supplied to the interpolation calculation unit 200 to calculate the x coordinate as described above.

  In this embodiment, since the pattern is constrained so that the intensity difference between stripes is 2 levels or more, the x-coordinate value can be measured with high accuracy.

  Next, correction of gamma characteristics of the projector and camera will be described.

  When a liquid crystal projector is used as the pattern projection device (projector) 10, an output image projected on an input image of the liquid crystal projector generally has a gamma characteristic. Therefore, even if 256 gradations are divided into six equal parts in the input image, the brightness of the projected pattern is not equal to seven levels. Therefore, the gamma characteristic must be taken into account when creating the input image. In the projector having the gamma characteristic of the output level on the vertical axis with respect to the input level on the horizontal axis as shown in FIG. 17, the input image shows each luminance gradation value of 0, 148, 185, 203, 222, 237, and 255 from the graph. It is desirable to have a stripe pattern. The same applies to the case where the pattern projector (projector) 10 is a DLP projector. When the pattern projector (projector) 10 is a slide projector, it is necessary to prepare data in consideration of the gamma characteristics of the slide film at the time of film creation.

  Similarly, a gamma characteristic also exists in a CCD camera. Therefore, when it is desired to obtain a stripe pattern having a uniform luminance interval on a camera image, an input image that further considers the gamma characteristic of the CCD camera is required.

  In this embodiment, a stripe pattern which is a reference for distance calculation by the triangulation method is used as a stripe pattern projected by the pattern projection apparatus 10, and recoding is performed on the reference stripe pattern using an image captured by the first camera 20. Since the correction is performed and the edges are originally determined and known and do not vary depending on the shape and texture of the subject, there is no problem of edge detection errors.

  This point will be further described.

  In Patent Document 1 (Japanese Patent No. 3482990), the captured image of the first camera 20 arranged on the same optical axis as the pattern projection apparatus 10 is re-encoded, and the re-encoded captured image of the first camera and the second In the case where the distance calculation is performed by associating the captured images of the camera 30 with each other, an edge extraction error occurs in both the captured image of the first camera 10 and the captured image of the second camera 30, and the distance measurement accuracy is increased. May deteriorate.

  That is, as described above, since the edge coordinates of the captured image of the first camera 20 are also interpolated at the sub-pixel level as shown in FIG. 14, the number of both sides of the edge pixel is represented by the variation in the average luminance value. An error occurs in the range shown in FIG. In particular, the error increases when the luminance difference between adjacent stripes is small.

  In reality, when a pattern having an intensity difference (luminance difference) is projected, there are a pattern having a large intensity difference and a pattern having a small intensity difference. When creating a stripe pattern, it is conceivable to make the level difference between adjacent stripes 2 or more, but there is still a magnitude difference in intensity.

  For example, consider a case where a stripe pattern composed of a plurality of levels as shown in FIG. 18 is projected. Although the re-encoding process as shown in FIG. 8 is performed by the first camera 20 having the same optical axis, the edge position shifts depending on the subject information and the adjacent intensity of the stripe pattern. As shown in FIG. 18B, edge extraction is performed at a position different from the original edge position (dotted line portion), resulting in a decrease in accuracy.

  In this embodiment, in order to solve such a problem, a stripe pattern that is a reference in triangulation is a stripe pattern projected from the projection device 10.

  That is, as shown in FIG. 11, the camera image of the projection device 10 and the first camera 20 arranged in the same optical axis direction as the projection device 10 shows subject information without changing the stripe shape of the stripe pattern. Since the luminance and hue values only change due to the above, it is possible to define the edge position of the stripe. Therefore, using the image of FIG. 18 (a) as a template, the brightness / hue change of the image of FIG. 18 (b) observed by the camera is detected, and the edge position of FIG. 18 (a) is held as it is. Only the luminance / hue change of (b) is superimposed on the image of FIG. 18 (a) to obtain the image of FIG. 18 (c). With this method, the projected stripe pattern can correspond to an image observed by the second camera 30 having a different optical axis even if the level of each stripe changes depending on the subject information. In addition, the edge position on the projection side does not shift due to the influence of subject information and camera images.

  The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the invention. For example, it goes without saying that the present invention is effective for projection light having an invisible wavelength such as near infrared in addition to the visible region wavelength.

  In addition, the reference stripe pattern for distance calculation by the triangulation method may be a captured image captured by the first camera, instead of the stripe pattern projected from the projection apparatus as described above. The captured image of the first camera is corrected by recoding as shown in FIG. Since the captured image of the first camera is arranged in the same optical axis direction as the stripe pattern of the projector, the edge position of the stripe can be uniquely determined. In the arrangement as shown in FIG. 19, the reference board 51 is arranged in advance, the stripe pattern is projected and the stripe edge position is specified from the observed position. The edge extraction deviation due to the change in luminance and hue value depending on the subject information does not occur. It is also possible to project only a pattern that specifies only the edge position as shown in the lower part of FIG. 20 and extract only the edge. In this case, since it is possible to extract a black and white image regardless of the magnitude difference, the edge extraction accuracy is increased. A stripe pattern in which two types of stripes of white and black are alternately arranged may be used.

  As described above, the edge position detected by using the calibration stripe pattern and the stripe code on both sides of the edge can be stored and managed in various modes. For example, as shown in FIG. 21, the edge position information extracted by the edge extraction unit 112 is stored in the extraction edge memory 113, and each stripe image of the photographed image of the first camera 20 is obtained from the edge position information of the extraction edge memory 112. The distance may be calculated by extracting and associating with each stripe of the photographed image of the second camera 30, or the stripe of the photographed image of the first camera 20 and the stripe of the photographed image of the second camera 30 at the time of measurement may be used. After that, the edge position information may be extracted from the extracted edge storage unit 113 as the stripe edge of the captured image of the first camera 20 and the distance may be calculated.

  In the above-described embodiment, the projection device 10 and the monitor imaging device 20 are arranged on the same optical axis (same principal point) using the half mirror 20, but as shown in FIG. The imaging devices 20 may be arranged so as to be negligible in the direction of the stripe (edge) of the pattern, and may be arranged on substantially the same optical axis (same principal point). In this case, it is possible to avoid power reduction and variation due to pattern light loss and distribution by the half mirror.

It is a figure which shows the apparatus structure of the Example of this invention. It is a figure which shows an example of the pattern and subject for demonstrating the said Example. It is a figure which shows the example of a stripe pattern of the above-mentioned Example. It is a schematic diagram explaining operation | movement of the said Example. It is a block diagram explaining the structural example of the said Example. It is a figure explaining the circuit structural example which performs the interpolation calculation of the coordinate x on the image of the 2nd camera of the measurement point of the said structural example. It is a flowchart explaining operation | movement of the said Example. It is a figure explaining coding of the above-mentioned example. It is an arrangement diagram of a camera and a subject for explaining the coding of the embodiment. It is a pattern diagram for demonstrating the encoding of the said Example. It is a figure which shows the example of the monitor image of the 1st camera of the above-mentioned Example. It is a figure which shows the example of the monitor image of the 2nd camera 2 of the above-mentioned Example. It is a figure explaining the part to which the brightness | luminance changed in the to-be-photographed object in the above-mentioned Example. It is a figure explaining distance calculation in the above-mentioned example. It is calculation explanatory drawing of the edge coordinate of the above-mentioned Example. It is a figure explaining the difference of the edge coordinate calculation by the gradation difference in the said Example. It is a gamma characteristic and pattern input value calculation explanatory drawing of the light projector (projector) of the said Example. It is a figure explaining the example which recodes the slit pattern of the said Example, hold | maintaining edge information. It is a figure explaining the edge extraction by the reference board used in the modification of the above-mentioned Example. It is a figure explaining the other projection pattern used by the edge extraction of FIG. It is a figure explaining the extraction edge memory | storage part used by the modification of the above-mentioned Example. It is a figure explaining the other modification of the above-mentioned Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Pattern projection apparatus 20 1st camera 30 for a monitor 2nd camera 40 for a triangulation 50 Half mirror 50 Object 110 Frame memory 111 Reference stripe pattern storage part 112 Edge extraction part 113 Extraction edge storage part 120 Pattern image memory 130 Area division Unit 140 luminance value memory 150 pattern image memory 160 recoding unit 170 code decoding unit 180 three-dimensional image memory 200 interpolation calculation unit 210 edge right neighboring pixel position input unit 220 edge right cell luminance average value input unit 230 edge left neighboring pixel position Input unit 240 Edge left cell luminance average value input unit 250 Edge luminance input unit

Claims (8)

A projector that projects a stripe pattern encoded with a plurality of levels of brightness set in advance in a predetermined brightness range onto a subject;
A first camera that captures a projected image from the same direction as the optical axis of the projector;
A second camera that captures the projected image from a direction deviated from the optical axis of the projector;
The reference stripe pattern is re-encoded based on the amount of change in each region of the image captured by the first camera with respect to the stripe pattern, and the re-encoded reference stripe pattern and the image captured by the second camera are obtained. A distance calculation for calculating a distance to the subject by performing triangulation between a point on the edge of the adjacent stripe of the reference stripe pattern and a point on the edge of the adjacent stripe of the image taken by the second camera in association with each other. Means,
3. A three-dimensional shape measuring apparatus according to claim 1, wherein each of the coordinates of edges of adjacent stripes of the reference stripe pattern is predetermined.   The reference stripe pattern is a stripe pattern projected from the projector, and the stripe pattern projected from the projector is re-encoded based on a change amount of each region of the image captured by the first camera with respect to the stripe pattern. A point on the edge of an adjacent stripe of the stripe pattern projected from the projector by associating the re-coded stripe pattern projected from the projector and the captured image of the second camera, and the second camera The three-dimensional shape measuring apparatus according to claim 1, wherein the distance information of the subject is calculated by performing a triangulation with a point on an edge of an adjacent stripe of a photographed image.   The three-dimensional shape measuring apparatus according to claim 2, wherein each region of the stripe pattern projected from the projector is fixedly associated with each region of a captured image of the first camera.   The reference stripe pattern is a photographed image of the first camera, and the photographed image of the first camera is re-encoded based on the amount of change with respect to the stripe pattern in each region of the photographed image of the first camera. At the same time, a stripe pattern for calibration is projected on the reference subject in advance by the projector, the projected image is captured by the second camera, the edge of the adjacent stripe is extracted from the captured image, and the edge is recoded. The reference image of the adjacent stripe of the image captured by the first camera is used as a reference edge, and the re-encoded image captured by the first camera and the image captured by the second camera are associated with each other. Triangulation is performed between a point on the reference edge of the adjacent stripe and a point on the edge of the adjacent stripe of the image taken by the second camera. 3-dimensional shape measuring apparatus according to claim 1, wherein calculating the distance.   The three-dimensional shape measuring apparatus according to claim 4, wherein the reference subject has a plane having the same luminance.   6. The three-dimensional shape measuring apparatus according to claim 4, wherein the calibration stripe pattern is drawn with a line segment in the background at a position corresponding to an edge of the stripe pattern at the time of measurement.   6. The three-dimensional shape measuring apparatus according to claim 4, wherein the calibration stripe pattern is obtained by alternately arranging two levels of stripes corresponding to each stripe of the stripe pattern at the time of measurement.   A stripe pattern encoded with a plurality of levels of brightness set in advance within a predetermined brightness range is projected onto a subject by a projector, and a projected image is captured from the same direction as the optical axis of the projector by a first camera, The projected image is captured from a direction deviated from the optical axis of the projector with the two cameras, and a reference stripe pattern is re-encoded based on the amount of change with respect to the stripe pattern in each region of the captured image of the first camera. The re-encoded reference stripe pattern and the captured image of the second camera are associated with each other, the point on the edge of the adjacent stripe of the reference stripe pattern and the adjacent stripe of the captured image of the second camera. In the three-dimensional shape measuring method for calculating the distance to the subject by performing triangulation with a point on the edge, the reference stripe pattern 3-dimensional shape measuring method characterized in that it is assumed that predetermined each coordinates of the edges of adjacent stripes over emissions.