JP2016121916A - Shape measurement device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To simplify a process for measuring the shape of a target by a pattern mirrored in the target.SOLUTION: A first image-capturing device 10 and a second image-capturing device 12 capture an image of each pattern mirrored in a target 20 from a display 16. A control/arithmetic unit 18 finds, on the basis of an image of each mirrored pattern, a physical relationship vector m1 indicating physical relationship between a center point C1 of image capture and a display point A1 on a display surface, and finds a physical relationship vector m2 indicating physical relationship between a center point C2 of image capture and a display point A2 on a display surface. The control/arithmetic unit 18 finds the position of each measurement point on the target 20 using the physical relationship vectors m1 and m2 on the basis of a geometric optical relation that holds true with regard to an optical path from the display point through measurement points on the target 20 to the first center point C1 of image capture and a geometric optical relation that holds true with regard to an optical path from the display point through measurement points on the target 20 to the second center point C2 of image capture.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a shape measuring apparatus, and more particularly to an apparatus for measuring the shape of a target by using a pattern reflected on the target.

  Research and development has been extensively conducted on devices for measuring the surface shape of objects. Such a shape measuring apparatus includes a surface shape measuring apparatus that measures a three-dimensional shape of a target without contact. In particular, as an apparatus for measuring the surface shape of a target with high specularity, there is one that employs a method of reflecting a pattern displayed on a display on a target. In this method, a pattern reflected on a target is photographed by an imager. Then, a correspondence relationship between the position on the pattern reflected on the target and the position of the display point on the display surface is obtained based on the data captured by the imager. Furthermore, based on this correspondence relationship, a normal vector at each measurement point on the target object and a geometric optical analysis on the optical path from the display point to the image sensor through the measurement point are performed, and each measurement point on the target object is analyzed. To determine the shape of the target.

  Patent Document 1 and Non-Patent Document 1 describe a technique for measuring the shape of a target with high specularity. These documents describe the phase shift method. In the phase shift method, as a pattern to be displayed on the display, a pattern in which the luminance changes in a waveform in the vertical direction and a pattern in which the luminance changes in a waveform in the horizontal direction are used. A wave-like light / dark pattern (light / dark pattern) is moved by a predetermined phase in a direction perpendicular to the longitudinal direction of the stripe, and the change of the stripe pattern reflected on the target is analyzed.

  Non-Patent Document 2 describes a technique for representing an image pickup device having a lens and a light receiving surface with a pinhole camera, and a technique for calibrating the posture of each equipment used for measurement.

JP 2007-322162 A

M. Sea. 2 outside MC Knauer, “Phase Measuring Deflectometry: a new approach to measure specular free-form surfaces”, Proceedings of SPIE 2004, 5457, p. 366-377 Zet. Z. Zhang “A Flexible New Technique for Camera Calibration”, IEEE Bulletin, Pattern Analysis and Machine Intelligence, IEEE Transactions, Volume 22, p. 1300-1334

  In general, in the method of measuring the shape of a target by a light / dark pattern reflected on the target, the distance between the stripes of the light / dark pattern, that is, the position of the display point on the display surface becomes shorter as the wavelength of the light / dark change is shorter. The measurement resolution when obtaining is increased. This is because the shorter the wavelength, the greater the change in brightness between adjacent pixels on the display surface, making it less susceptible to noise. However, if the wavelength is shortened, it is necessary to perform processing for obtaining an absolute phase value in each period of the change in brightness reflected in the target in consideration of the phase of the adjacent period. Here, the absolute phase value means a value obtained by adding 2πκ to the phase value in the phase range of −π to + π. κ is an integer that increases by 1 for each period along the light-dark change direction, and corresponds to each period of the light-dark pattern displayed on the display surface. Such processing for connecting phases between adjacent periods may involve complicated processing using a captured image of a pattern reflected in a target.

  An object of this invention is to simplify the process which measures the shape of a target object by the pattern reflected on the target object.

  The present invention provides a pattern display unit that displays a plurality of types of display patterns one by one on a target, and a display position based on the plurality of types of display patterns with respect to display points on a pattern display surface of the pattern display unit. A display position code associating unit for associating codes, an image capturing unit for capturing each pattern reflected in the target, and a point on the reflected image captured by the image capturing unit for associating a measurement point position code Based on the measurement point position code association unit, the display position code, and the measurement point position code, the position of the imaging unit and the display point on the pattern display surface corresponding to the measurement point on the target A positional relationship calculation unit for obtaining a positional relationship amount representing a relationship, and a geometric optical relationship established for an optical path from a display point on the pattern display surface through a measurement point on the target to the imaging unit. , Based on the geometrical optics relationships including the position relationship represented by the position related quantity, characterized in that it comprises a measurement point calculation unit for determining the position of the measurement point on the target.

  Preferably, the plurality of types of display patterns are binary in luminance, and the display position code association unit has the same position of the display points constituting the pattern display surface as the number of the plurality of types of display patterns. The measurement point position code association unit encodes the display position code represented by a binary code of the number of digits, and configures the reflected image based on each reflected image captured by the imaging unit. Each pixel is encoded as the measurement point position code represented by a binary code having the same number of digits as the number of the plurality of types of display patterns.

  Preferably, the imaging unit captures each pattern captured on the target from a first position and a second position, and generates a first image and a second image, respectively, and the positional relationship calculation unit includes: Based on the first image and the second image, a first positional relationship amount and a second positional relationship amount are obtained as the positional relationship amounts for which a stereo matching condition is satisfied for the measurement points on the target, respectively, The measurement point calculation unit obtains the position of the measurement point on the target based on the first positional relationship amount and the second positional relationship amount that satisfy the stereo matching condition.

  Preferably, the measurement point calculation unit solves the equation based on the geometric optical relationship for each of a plurality of measurement points on the target and the continuity of planar elements at adjacent measurement points, thereby An equation calculation unit for obtaining each position of a plurality of measurement points on the target is provided, and the equation calculation unit is a representative point for which the position is determined based on a stereo matching condition among the plurality of measurement points on the target. The position is used to solve the equation.

  In addition, the present invention provides a pattern display unit that displays a display pattern on a target, an imaging unit that captures each pattern reflected on the target, and a positional relationship and an attitude relationship between the pattern display unit and the imaging unit. And a calculation unit for obtaining the position of the measurement point on the target based on the reflected image captured by the imaging unit, and the positional relationship and the posture relationship between the pattern display unit and the imaging unit At the time of calibration, the pattern display unit, the mirror, and the imaging unit are arranged so that an optical path from the pattern display unit to the mirror and from the mirror to the imaging unit is formed. Based on the shape of the mirror imaged by the imaging unit and the mirror image of the calibration pattern imaged by the imaging unit and imaged by the pattern display unit. And obtaining the positional relationship and orientation relationship between the pattern display unit and the imaging unit.

  Preferably, at the time of the calibration, the calculation unit is configured to have a geometric internal structure of the pinhole camera when the imaging unit is replaced with a pinhole camera based on a mirror image of the calibration pattern captured by the imaging unit. Find the parameters.

  Preferably, at the time of the calibration, the calculation unit obtains the positional relationship and the posture relationship between the imaging unit and the mirror based on the shape of the mirror photographed by the imaging unit, and the calculated positional relationship and the posture relationship are obtained. Based on this, the positional relationship and the posture relationship between the pattern display unit and the imaging unit are obtained.

  According to the present invention, it is possible to simplify the process of measuring the shape of a target by using a pattern reflected on the target.

It is a figure which shows the structure of a shape measuring apparatus. It is a figure which shows the monochrome pattern displayed on a display. It is a figure which shows each reflection pattern which appears on a target object. It is a figure which shows multiple types of random monochrome patterns. It is a figure which shows the positional relationship of a display, a 1st image pick-up device, and a target surface. It is a figure which shows the model for demonstrating a stereo matching. It is a figure which shows the convergence result of the solution of a shape measurement equation. It is a figure which shows the measurement result of the shape of a target object. It is a figure which shows arrangement | positioning of each apparatus at the time of calibration. It is a figure which shows the example of the pattern for a calibration displayed on a display. It is a figure which shows the example of the image of the mirror image | photographed with the 1st image pick-up device. It is a figure which shows the relationship between the shape of the image | photographed mirror, and the xyz coordinate defined on the imaging surface of a 1st imaging device.

(1) Outline of Shape Measurement FIG. 1 shows the configuration of a shape measuring apparatus according to an embodiment of the present invention. The shape measuring device measures the shape of the target 20 by measuring the position of each measurement point on the surface of the target 20. The shape measuring apparatus includes an imaging unit 14, a display 16, and a control / calculation unit 18. The display 16 causes each of the plural types of display patterns to be reflected on the target 20 one by one under the control of the control / calculation unit 18. In the present embodiment, the display pattern on the display 16 is a monochrome pattern. Instead of such a display 16, a general display device that displays a pattern such as a black and white pattern may be used. The black and white pattern may be a pattern whose luminance is represented by a binary value.

  The imaging unit 14 includes a first imaging device 10 and a second imaging device 12 installed at different positions. Under the control of the control / arithmetic unit 18, the imaging unit 14 causes each of the plurality of types of patterns reflected on the target 20 from the display 16 by the first imaging device 10 and the second imaging device 12 from two different positions. Take a picture.

  The control / arithmetic unit 18 is configured by an information processing apparatus such as a computer. The control / arithmetic unit 18 solves the shape measurement equation based on the geometrical optical relationship established for each measurement point on the target 20 using each image of a plurality of types of reflection patterns photographed from two positions. To find the position of each measurement point. This geometrical optical relationship is such that the incident optical path 22 from the display point A1 on the display surface to the measurement point q on the target 20 and the reflected optical path 24 from the measurement point q to the imaging center point C1 of the first imager 10. And the incident optical path 23 from the display point A2 on the display surface to the measurement point q on the target 20, and the reflected optical path 25 from the measurement point q to the imaging center point C2 of the second imager 12. This is the relationship that holds. Here, the first imaging center point C1 and the second imaging center point C2 are imaging center points when the first imaging device 10 and the second imaging device 12 are replaced with pinhole camera models, respectively. This is the reference point.

  The geometric optical relationship established for the incident optical path 22 and the reflected optical path 24 includes the positional relationship between the imaging center point C1 and the display point A1. The control / arithmetic unit 18 obtains a positional relationship vector m1 as a positional relationship amount using each image of a plurality of types of reflection patterns photographed by the first imager 10, and uses it for calculation to solve the shape measurement equation. The positional relationship vector m1 is a vector having an imaging center point C1 as a start point and a display point A1 as an end point.

  Similarly, the geometric optical relationship established for the incident optical path 23 and the reflected optical path 25 includes the positional relationship between the imaging center point C2 and the display point A2. The control / arithmetic unit 18 obtains a positional relationship vector m2 as a positional relationship amount using each image of a plurality of types of reflection patterns photographed by the second imager 12, and uses it for calculation to solve the shape measurement equation. The positional relationship vector m2 is a vector having an imaging center point C2 as a start point and a display point A2 as an end point.

  As will be described later, with respect to an image photographed by the first imager 10, in addition to the geometric optical relationship established for each of a plurality of measurement points, the condition that the planar elements at adjacent measurement points are continuous is a shape measurement. Reflected in the equation. For the image photographed by the second imager 12, the geometrical optical relationship established for the representative point that is one of the plurality of measurement points is reflected in the shape measurement equation. The control / arithmetic unit 18 obtains the direction and distance from the imaging center point C1 to each measurement point by solving the shape measurement equation, and obtains the position of each measurement point based on the imaging center point C1.

(2) Spatial coding method (processing for obtaining positional relationship vectors m1 and m2)
Specific processing executed by the shape measuring apparatus will be described. Here, the relationship between the distance between the display 16, the first imager 10 and the second imager 12, and the orientation of the display 16, the first imager 10 and the second imager 12 is obtained in advance. To do.

  A display coordinate system U0 is defined on the display display surface as three-dimensional coordinates with a point U on the display display surface as an origin. In the first imager 10, a first imaging coordinate system C10 is defined as three-dimensional coordinates with the imaging center point C1 as the origin. In the second imager 12, a second imaging coordinate system C2 is defined as three-dimensional coordinates with the imaging center point C2 as the origin. The display coordinate system U0, the first imaging coordinate system C10, and the second imaging coordinate system C20 can be coordinate-converted with each other.

  In order to solve the shape measurement equation, the control / arithmetic unit 18 obtains a positional relationship vector m1 for each measurement point photographed by the first imager 10 by a spatial coding method described below. Similarly, the control / calculation unit 18 obtains a positional relationship vector m2 for each measurement point imaged by the second imager 12.

  The display 16 displays a plurality of types of black and white patterns one by one and causes each black and white pattern to be reflected on the target 20 one by one. The first imager 10 captures each pattern captured on the target 20 one by one, and outputs each captured image data to the control / arithmetic unit 18. Similarly, the second imager 12 captures each pattern captured one by one on the target 20 and outputs each captured image data to the control / calculation unit 18.

  FIG. 2 shows a monochrome pattern displayed on the display 16. In the black and white pattern shown in FIGS. 2A-1 to 2A-n, 1/2, 1/3, 1/4... 1 / (n + 1) in order from the top by vertical lines. ) Is divided into display areas. In the monochrome pattern in which the display display surface is divided into ½, a white area is arranged on the left side and a black area is arranged on the right side. In the monochrome pattern in which the display display surface is divided into 1/3, 1/4... 1 / (n + 1), white areas and black areas are alternately arranged from the left to the right. In the monochrome patterns shown in FIGS. 2B-1 to 2B-N, 1/2, 1/3, 1/4... 1 / (N + 1) in order from the top by horizontal lines. ) Is divided into display areas. In the monochrome pattern in which the display display surface is divided into ½, the white area is arranged on the upper side and the black area is arranged on the lower side. In the monochrome pattern in which the display display surface is divided into 1/3, 1/4... 1 / (N + 1), white areas and black areas are alternately arranged from the top to the bottom.

  FIG. 3 shows each reflection pattern appearing on the target when each monochrome pattern shown in FIG. 2 is reflected on the spherical target. 3 (a-1) to (an) correspond to FIGS. 2 (a-1) to (an), respectively, and FIGS. 3 (b-1) to (bN) respectively correspond to FIGS. 2 corresponds to FIGS. 2B-1 to 2B-N. As shown in FIG. 3, the shape of the pattern reflected on the target is distorted according to the surface shape of the target.

  In the shape measuring apparatus, the code “1” is assigned to the white area of the black and white pattern and the reflection pattern, and the code “0” is assigned to the black area. The assignment of 1 and 0 for each region may be reversed. In addition to such binary codes “1” and “0”, other binary codes may be used. The horizontal position of the display point A in FIG. 2 is specified by the n-digit code “101... 1” by the n types of black and white patterns shown in FIGS. Is done. The vertical position of the display point A in FIG. 2 is an N-digit code “001... 1” according to the N types of black and white patterns shown in FIGS. Specified by Therefore, the position of the display point A on the display surface is specified in the display coordinate system by an n + N-digit display position code “101... 1001.

  For each measurement point on the target surface, n + N types of measurement point position codes are obtained by photographing n + N types of reflection patterns obtained by n + N types of black and white patterns. That is, when n + N types of black and white patterns are projected onto the target sequentially from the display 16, the code “1” or “0” is associated with the black and white observed in order at each measurement point. , N + N-digit measurement point position codes are obtained for each measurement point.

  A display point on the display surface having the same display position code as the measurement point position code corresponds to the measurement point on the target surface. For example, when “101... 1001... 1” is obtained as a measurement point position code for a certain measurement point, the display point A shown in FIG. It is assumed that an image of display point A is reflected at the measurement point.

  Returning to FIG. 1, the control / arithmetic unit 18 specifies a display point corresponding to each measurement point based on the reflected image data obtained from the first imager 10 for each reflected pattern. Then, a positional relationship vector m1 having the imaging center point C1 as a start point and the display point as an end point is obtained for each measurement point. The positional relationship vector m1 is obtained by converting the coordinate value of the display point in the display coordinate system U0 into the coordinate value in the first imaging coordinate system C10.

  Similarly, the control / calculation unit 18 specifies a display point corresponding to each measurement point based on the reflected image data obtained from the second image pickup device 12 for each reflected pattern. Then, a positional relationship vector m2 having the imaging center point C2 of the second imager 12 as a start point and the display point as an end point is obtained for each measurement point. The positional relationship vector m2 is obtained by converting the coordinate value of the display point in the display coordinate system U0 into the coordinate value in the second imaging coordinate system C20.

  In the above description, as shown in FIG. 2, an example using a monochrome pattern in which a display display surface is equally divided into a white area and a black area has been described. The display display surface does not necessarily need to be equally divided, and the area ratio of the white area and the black area may be adjusted according to the tendency of the shape of the target. For example, when the approximate shape of the target is known, the black and white pattern may be made finer in a region where the angle of the target surface with respect to the display surface is close to 90 °. Further, as shown in FIG. 2, the plurality of black and white patterns may not include a set of patterns in which black and white are divided in the vertical direction and a set of patterns in which black and white are divided in the horizontal direction. . For example, a plurality of types of checkered black and white patterns with different density may be used. A plurality of types of black and white patterns in which a plurality of black polygonal regions having different sizes and a plurality of white polygonal regions having different sizes are randomly scattered on the display display surface may be used. Furthermore, as shown in FIGS. 4 (c-1) to (cn), a plurality of types in which patterns such as particles, fluff, and scratches having random sizes are randomly scattered on the display surface. Alternatively, a random black and white pattern may be used.

  In addition, here, a black and white pattern using white and black has been described as a display pattern on the display 16, but other colors may be used. Alternatively, the number of colors may be increased to form a multi-tone pattern, and a multi-digit code may be assigned to each color to increase the number of digits of the display position code and the measurement point position code.

  Note that the position resolution of the measurement point is determined according to the size of the pixels or the number of pixels (pixel resolution) constituting the image captured by the first imager 10 and the second imager 12. On the other hand, the fineness of the black-and-white pattern is determined as the fineness to which the measurement point position code and the display position code are assigned to each pixel constituting the image photographed by each image pickup device.

(3) Measurement of the position of each measurement point (3-1) Measurement system After obtaining the positional relationship vector m1 and the positional relationship vector m2 for each measurement point, the control / calculation unit 18 solves the shape measurement equation, The position of each measurement point with respect to the position of the imaging center point C1 is obtained.

  FIG. 5 conceptually shows the positional relationship among the display 16, the first imager 10, and the target 20. The first imager 10 is represented by a pinhole camera model including an imaging center point C1 and an imaging surface P1. A technique for replacing an image pickup device with a pinhole camera model is described in Non-Patent Document 2. The pinhole camera model is geometrically represented by internal parameters. The internal parameters include the coordinate value of the imaging center point C1, the focal length F between the imaging center point C1 and the imaging surface P1, the central coordinate value of the imaging surface P1, the parameter representing the shape of the imaging surface P1, and the imaging surface P1. Including the coordinate value of each pixel to be obtained and obtained by calibration described later. Here, the coordinate value of the imaging center point C1 and the coordinate value of each pixel included in the imaging surface P1 are, for example, coordinate values in a local two-dimensional coordinate system defined on the imaging surface P1.

  The absolute coordinate system in the present embodiment is the first imaging coordinate system. That is, in the absolute coordinate system, the imaging center point C1 is the origin, the right direction from the origin C1 toward the imaging plane P1 is the x-axis positive direction, the downward direction is the y-axis positive direction, and the direction from the origin C1 to the imaging plane P1. Is the z-axis positive direction. Instead of using the first imaging coordinate system as the absolute coordinate system, the second imaging coordinate system, the display coordinate system, or another three-dimensional coordinate system may be used as the absolute coordinate system.

(3-2) Processing for Obtaining Measurement Point Pixel Vector I The first imager 10 captures the target 20 and generates target image data. The target image data is for recognizing the target 20 on the imaging plane P1. Therefore, any of the plurality of reflected image data obtained by the spatial coding method may be used as the target image data.

The control / arithmetic unit 18 recognizes the target image Tg based on the target image data obtained by the first imager 10. The target image Tg is a set of pixels PQ representing the target 20, and each pixel PQ corresponds to each measurement point on the target 20. Control / calculation unit 18, based on the internal parameters and the target image Tg, measured point pixel vector _{I =} toward the direction of the measurement point q from the image center point _{C1 (e x, e y,} F) , and the target image Tg It calculates | requires about each pixel PQ to represent. The measurement point pixel vector I is a vector having the imaging center point C1 as a start point and a pixel PQ corresponding to the measurement point q as an end point.

Here, the process for obtaining the measurement point pixel vector I = (e _x , e _y , F) for the first imager 10 has been described. However, the measurement point pixel vector I = ( e _x , e _y , F) are determined.

(3-3) Shape Measurement Equation The shape measurement equation will be described with reference to FIG. The plane element Q of the measurement point q corresponding to the pixel PQ is expressed by the plane equation shown in (Equation 1).

Here, the distance variable k included in (Expression 1) is an unknown scalar variable that represents the distance from the imaging center point C1 to the measurement point together with the measurement point pixel vector I as k · | I |. By obtaining the distance variable k for each pixel PQ constituting the target image Tg, the direction and distance to each measurement point q on the target 20 is obtained. F _x and f _y included in (Equation 1) are expressed by (Equation 2) and (Equation 3), respectively. Vector _{n = (f x, f y} , 1) is the normal vector of the planar element Q.

  Further, E and D included in (Equation 2) and (Equation 3) are expressed by (Equation 4) and (Equation 5), respectively.

The measurement point pixel vector I = (e _x , e _y , F) and the positional relationship vector m 1 = (m _x , m _y , m _z ) are obtained in advance. Therefore, is assigned to E and D (Equation 2) and (Equation 3), further, it is substituted into equation (2) and _{f x} and _{f y} is expressed by the equation (3) (Equation 1) , The coefficient in (Equation 1) is obtained.

  The pixels on the imaging surface P1 are two-dimensionally arranged in the vertical direction and the horizontal direction. A pixel code (i, j) is assigned to each pixel by representing the array position in the x-axis direction by a positive integer i and the array position in the y-axis direction by a positive integer j. By adding a subscript “i, j” indicating that the pixel code (i, j) is an expression corresponding to the assigned pixel, (Equation 1) is expressed as (Equation 6).

Next, consider the condition that adjacent planar elements are continuous. The planar element Q _{i, j} corresponding to the pixel to which the pixel code (i, j) is assigned is adjacent to the planar element Q _{i + 1, j} corresponding to the pixel to which the pixel code (i + 1, j) is assigned. The expression of the planar element Q _{i + 1, j} corresponding to the pixel to which the pixel code (i + 1, j) is assigned is expressed as (Equation 7).

  The midpoint coordinate α of the measurement point on each planar element represented by (Expression 6) and (Expression 7) is expressed as (Expression 8).

  If the z-coordinate values of (Equation 6) and (Equation 7) are equal at this midpoint α, (Equation 9) is obtained from (Equation 6) to (Equation 8).

  It is assumed that X pixels are arranged in the horizontal direction of the imaging surface P1, and Y pixels are arranged in the vertical direction. Pixel code (i, j) = (1,1), (1,2),... (1, Y), (2,1), (2,2),. Y), (3, 1), (3, 2)... (X, Y), when (Equation 9) is made simultaneous, (Equation 10) to (Equation 13) The equation represented is obtained.

(Equation 10) is a simultaneous equation including X × Y unknown distance variables k _{i, j} . The coefficient matrix G included in (Equation 10) is a matrix of X × Y rows and X × Y columns _, and other than the matrix components G _{i, j} and G _{i + 1, j} represented by (Equation 12) and (Equation 13). The component of is zero. The distance variable vector k expressed by (Equation 11) includes X × Y unknown distance variables as vector components. The vector 0 is a zero vector including X × Y vector components having a value of 0.

  The equations represented by (Equation 10) to (Equation 13) are equations relating to each pixel of the pixel code (i, j) and the pixel code (i + 1, j). Similarly, equations relating to each pixel of pixel code (i, j) and pixel code (i-1, j) can be created. Furthermore, it is possible to create equations for each pixel of pixel code (i, j) and pixel code (i, j + 1), and in addition, pixel code (i, j) and pixel code (i, j−1) ) For each pixel. By expanding these equations to the equations represented by (Equation 10) to (Equation 13), the coefficient matrix G is expanded to a matrix of (4 × X × Y) rows and (X × Y) columns. The equation is obtained.

A Lagrange condition is that the position of the representative point q _{s, t} , which is one of a plurality of measurement points, is known, and the representative value k _{s, t} is known as a distance variable for the representative point q _{s, t.} The shape measurement equation represented by (Equation 14) is obtained by reflecting it in the expanded simultaneous equations by the undetermined multiplier method.

Here, the representative point vector h included in the shape measurement equation is a column vector in which elements other than the known representative values k _{s and t} among the elements of the distance variable vector k are 0, as shown in (Expression 15). It is expressed in The representative value k _{s, t} is a distance variable obtained in advance for the representative point q _{s, t} , and is obtained, for example, by stereo matching described later. The representative value k _{s, t} may be obtained by measuring the distance from the first imaging center point C1 to the representative point q _{s, t} by a method different from stereo matching. In this case, the second imager 12 is not necessary.

  The mediation unit vector M is a row vector in which the element corresponding to the representative point vector h is 1 and the other elements are 0, and is expressed as (Equation 16).

  When the difference equation is derived from the shape measurement equation of (Equation 14), (Equation 17) is obtained.

The subscript “p” of the distance variable vector k _p in (Expression 17) indicates a value in the p-th calculation. An initial vector k ₀ of the distance variable vector k _p is given, p is incremented by 1, and the error Err obtained by (Equation 18) is less than a predetermined value, for example, Err is less than or equal to 1 × 10 ⁻¹⁶ The solution of the distance variable vector k is obtained by performing iterative calculation until. As the initial vector k ₀ , for example, a vector having all vector components as representative values k _{s, t} is used.

(3-4) Arithmetic processing based on shape measurement equation The control / calculation unit 18 solves the difference measurement shape measurement equation (difference / shape measurement equation) shown in (Equation 17) to obtain the distance variable vector k. Ask. The position of the measurement point corresponding to the pixel code (i, j) is obtained from the elements k _{i, j} of the distance variable vector k. That is, the position of the measurement point corresponding to the pixel code (i, j) is represented by the vector q _{i, j} in (Equation 19) with the imaging center point C1 as a reference.

(3-5) Stereo Matching The control / calculation unit 18 obtains representative values k _{s, t} to be included in the initial vector k ₀ and the representative point vector h before solving the difference / shape measurement equation. The representative value k _{s, t} is a distance variable for the representative point q _{s, t} . The control / calculation unit 18 obtains the representative values k _{s, t} by stereo matching described below.

  FIG. 6 conceptually shows a model for explaining stereo matching. Among the components shown in FIG. 6, the same components as those shown in FIG. 1 are denoted by the same reference numerals and description thereof is omitted.

Of the pixels on the imaging surface P1, the pixel to which the pixel code (s, t) is assigned is shown as the representative point pixel PQ _{s, t} . The representative point pixel PQ _{s, t} may be determined by a user operation or may be determined according to a pixel code (s, t) stored in advance.

Representative point pixels _{PQ s,} for the _t, the representative point on the target object 20 _{q s, t} and, previously obtained positional relationship vector _{m1 s, t} is associated. However, as long as the value obtained by photographing with the first imager 10 is present, the representative point q _{s, t} may be at any position on the line of sight V1 passing through the imaging center point C1 and the representative point pixel PQ _{s, t.} I only understand. That is, the position relation vector m1 s to start the imaging center point _C1, display point A _s in the end of _t, although the position of _t can be specified, any incident light path extending toward the display point A _s, a _t the line of sight V1 It cannot be specified which of the intersection points CRS between β1 and the line of sight V1 is the representative point q _{s, t} . In FIG. 5, a plurality of intersection points CRS that can be the measurement points q _{s, t} are shown on the line of sight V1.

Therefore, the control / arithmetic unit 18 specifies the representative points q _{s and t} using values obtained by imaging by the second imaging device 12 in addition to values obtained by imaging by the first imaging device 10.

The control / arithmetic unit 18 performs each process on the imaging plane P2 by the process described in the above “(3-1) Measurement System” and the second imager 12 by the same process as the process for the first imager 10. A positional relationship vector m2 _{i, j} corresponding to the pixel PQ2 _{i, j} is obtained. Further, each pixel _{PQ2 i,} for _j, imaging center points C2 and pixel _{PQ2 i,} gaze through _{j V2 i, j} and the display point _{A2 i,} gaze from _{j V2 i,} extending toward the _j incident light path .beta.2 _{i , J.} Here, i is an integer from 1 to X, j is an integer from 1 to Y, and for each of X × Y pixels PQ2 _{i, j} , the line of sight V2 _{i, j} and the positional relationship vector m2 _{i, j} , And the incident optical path β2 _{i, j} .

The control / arithmetic unit 18 searches for pixels PQ2 _{i, j} on the imaging surface P2 that have a geometrical optical relationship that is not physically inconsistent with the incident optical path β1 and the line of sight V1. That is, a search is made for a pixel PQ2 _{i, j} on the imaging surface P2 such that an angle θ1 formed by the incident light path β2 _{i, j} and the incident light path β1 is equal to an angle θ2 formed by the line of sight V2 _{j, i} and the line of sight V1. To do.

The control / arithmetic unit 18 determines, as representative points q _{s, t} , among the intersection points CRS between the incident optical path β1 and the line of sight V1 as representative points q _{s, t} , and represents the representative point q from the imaging center point C1. _s, determine the distance to the _t representative value _{k s,} as _t.

The control / arithmetic unit 18 solves the shape measurement equation of the differential expression shown in (Equation 17) using the representative values k _{s, t} to obtain the distance variable vector k. Then, based on the distance variable vector k, the position of each measurement point with respect to the position of the imaging center point C1 is obtained.

(4) Measurement Result FIG. 7 shows the convergence result of the solution of the shape measurement equation represented by (Equation 17). The target is a convex mirror with a curvature radius of 103.35 mm. The horizontal axis represents the number of iterations p, and the vertical axis represents the error Err defined by (Equation 18). The error Err was 1 × 10 ⁻¹⁶ or less at the number of iterations p = 16. FIG. 8 shows the measurement result of the shape of the target. In this figure, a set of measurement points is indicated by a plane in the first imaging coordinate system C10. The error with respect to an ideal spherical surface having a curvature radius of 103.35 mm was 3σ = 46.7 μm (σ is a standard deviation).

(5) Modification In the above, the representative value k _{s, t} as the distance variable is obtained by stereo matching for the representative point q _{s, t} among the plurality of measurement points, and the difference / shape is obtained using the representative value k _{s, t.} The process of solving the measurement equation has been described. In addition to such processing, distance variables may be obtained based on stereo matching for all measurement points. In this case, on each line of sight extending from each pixel PQ on the imaging plane P1 representing the target image Tg, the second imaging device 12 is used to search for a measurement point that satisfies the above stereo matching condition, and each measured measurement point is searched. Find the distance variable for. In this case, the position of each measurement point in the first imaging coordinate system C10 can be obtained without solving the difference / shape measurement equation.

  In the above description, the configuration in which the imaging unit 14 includes the first imaging device 10 and the second imaging device 12 has been described. In addition to such a configuration, the imaging unit 14 may include only the first imaging device 10 and the second imaging device 12 may not be used. In this case, the shape measuring apparatus is provided with a moving mechanism that moves the first imager 10. Photographing is performed before and after the first imager 10 is moved, and the first imager 10 captures images from two different positions, whereby the same processing as in the case of providing two imagers is performed.

  In the above description, a measurement point is determined for each pixel on the imaging surface P1 of the first imager 10. The measurement point may be determined for each of a plurality of pixels on the imaging surface P1. For example, one measurement point may be determined for one pixel and eight pixels surrounding the pixel, and further, 16 pixels (a total of 25 pixels) surrounding the nine pixels may be defined. On the other hand, one measurement point may be determined.

(6) Effect by Shape Measuring Device The shape measuring device according to the present embodiment displays a display 16 as a pattern display unit that displays a plurality of types of display patterns one by one on the target 20 and one type on the target 20. An image capturing unit 14 that captures the displayed display pattern and a control / calculation unit 18 that obtains the position of the measurement point on the target 20 are provided. The control / arithmetic unit 18 includes the following components. That is, a display position code associating unit that associates display position codes (display position codes) based on a plurality of types of display patterns with display points on the display display surface as a pattern display surface is included. Further, a measurement point position code associating unit that associates a measurement point position code with each point on each reflected image captured by the imaging unit 14 is included. Further, based on the display position code and the measurement point position code, a positional relation for obtaining a positional relation amount representing the positional relation between the imaging unit 14 and the display point on the pattern display surface corresponding to the measurement point on the target 20 A calculation unit is included. In addition, the measurement point calculation for obtaining the position of the measurement point on the target 20 based on the geometric optical relationship established for the optical path from the display point on the pattern display surface to the imaging unit through the measurement point on the target 20 Part is included. The measurement point calculation unit solves the geometric equation based on the geometric optical relationship established for each of the plurality of measurement points on the target 20 and the continuity of the planar elements at the adjacent measurement points, thereby obtaining the target An equation calculation unit for obtaining each position of a plurality of measurement points on 20 is included.

  As described above, in the shape measuring apparatus, the positional relationship amount representing the positional relationship between the imaging unit 14 and the display point corresponding to the measurement point is obtained based on the display position code and the measurement point position code. Therefore, it is not necessary to obtain the absolute phase value of the pattern reflected on the target, and the process for measuring the shape of the target can be simplified.

(7) Calibration of Measurement System In the above, the relationship between the distance between the display 16, the first imager 10, and the second imager 12 and the posture of the display 16, the first imager 10, and the second imager 12. Is known. That is, the relationship between the position and orientation of the display coordinate system U0, the first imaging coordinate system C10, and the second imaging coordinate system C20 shown in FIG. 1 is known. The mutual relationship between these coordinate systems is obtained by calibration before measuring the shape.

  FIG. 9 shows the arrangement of each equipment when calibration is performed. For the calibration, a mirror 26 having a circular mirror surface is used. The first arrangement position (i) and the second arrangement position (ii) indicate that the same mirror 26 is arranged at different positions or different postures. The mirror 26 at the first arrangement position is drawn with a solid line, and the mirror 26 at the second arrangement position is drawn with a two-dot chain line. In the calibration, in addition to the first arrangement position and the second arrangement position, the mirror 26 is arranged at a plurality of different positions or postures such as a third arrangement position, a fourth arrangement position,.

  The display 16, the mirror 26, and the first imager 10 are arranged so that an incident light path 32 from the display display surface to the mirror and a reflected light path 34 from the mirror 26 to the first imager 10 are formed. A line of sight V-1 extending from the imaging center point C1 toward the mirror 26 at the first arrangement position and extending through the mirror 26 is symmetrical with respect to the mirror surface of the mirror 26 at the first arrangement position. A mirror image 16-1 appears. A line of sight V-2 extending from the imaging center point C1 toward the mirror 26 at the second arrangement position and extending through the mirror 26 is symmetrical with respect to the mirror surface of the mirror 26 at the second arrangement position. A mirror image 16-2 appears. In FIG. 9, mirror images of the origin of the display coordinate system are shown as point U-1 and point U-2.

  FIG. 10 shows an example of the calibration pattern 28 displayed on the display 16. The calibration pattern 28 is a pattern in which dots 30 (spots) are arranged in the vertical direction and the horizontal direction at predetermined intervals. That is, in the calibration pattern 28, a plurality of dots 30 are arranged in the vertical direction, and a plurality of rows of dots 30 are arranged in the horizontal direction.

  Returning to FIG. 9, the first imager 10 is calibrated each time the mirror 26 is arranged at a plurality of arrangement positions such as the first arrangement position, the second arrangement position, the third arrangement position,. Take a mirror image of the pattern. Then, each image data corresponding to each arrangement position is output to the control / arithmetic unit 18.

  The control / arithmetic unit 18 obtains internal parameters when the first image pickup device 10 is replaced with a pinhole camera model by analyzing calibration patterns photographed at a plurality of arrangement positions. Specifically, the control / arithmetic unit 18 analyzes the calibration pattern a plurality of times while changing the temporary internal parameter to obtain an appropriate internal parameter. In the pinhole camera model based on ideal internal parameters, the straight line connecting the mirror image of the dot of the calibration pattern and the dot projected on the imaging surface of the first imager 10 is the position where all the mirrors 26 are arranged. And passes through the imaging center point C1. The control / arithmetic unit 18 obtains, as an appropriate internal parameter, a parameter that is the same as the ideal internal parameter or approximated to the ideal internal parameter.

  FIG. 11 shows an example of an image of the mirror 26 photographed by the first imager 10. The shape of the mirror 26 in which the calibration image is reflected is an ellipse. The mirror 26 on the calibration image shows a plurality of dots 30 whose arrangement positions are distorted according to the actual position and orientation of the mirror.

Returning to FIG. 9, the control / calculation unit 18 translates the internal parameters of the first imager 10 and the mirror image position and orientation of the display coordinate system based on the mirror image position of each dot appearing in the calibration image. A vector T _QL and a rotation matrix R _QL are obtained for each arrangement position. The subscript L indicates a value for the Lth arrangement position. The translation vector T _QL is a vector having the imaging center point C1 as a start point and a mirror image UL at the origin U of the display coordinate system as an end point, and includes three vector components. The rotation matrix R _QL is a matrix of 3 rows and 3 columns representing the attitude of the mirror image of the display coordinate system with reference to the first imaging coordinate system. The rotation matrix R _QL is obtained by, for example, the three-axis pitch angle and roll angle of the mirror image of the display coordinate system with respect to the three-axis directions of the first imaging coordinate system.

The control / arithmetic unit 18 arranges a translation vector T _mL and a rotation matrix R _mL representing the position and orientation of the mirror 26 based on the internal parameters and the shape (projection shape) of the mirror 26 appearing in the calibration image. Ask for position. The mirror 26 defines a mirror coordinate system having the mirror center point mr as the origin. The translation vector T _mL is a vector having an imaging center point C1 as a start point and an origin mr in the mirror coordinate system as an end point, and includes three vector components. The rotation matrix R _mL is a 3 × 3 matrix representing the attitude of the mirror coordinate system with reference to the first imaging coordinate system. The rotation matrix R _mL is represented by, for example, the triaxial pitch angle and roll angle of the mirror coordinate system with respect to the triaxial direction of the first imaging coordinate system.

FIG. 12 conceptually shows the relationship between the projection shape of the mirror 26 and the xyz coordinates defined on the imaging surface P1. In this embodiment, the diameter of the mirror 26 is known. Therefore, the direction and distance from the imaging center point C1 to the origin mr of the mirror coordinate system, and the mirror coordinates in the first imaging coordinate system, depending on the internal parameters and the length and direction of the major axis γ of the ellipse that is the projection shape of the mirror 26 The attitude of the system is required. The control / calculation unit 18 determines the internal parameters, the position of the center point ξ of the projection shape of the mirror 26, the length of the major axis γ, the angle φ formed by the major axis γ with respect to the x-axis positive direction, and the known diameter of the mirror 26. A translation vector T _mL and a rotation matrix R _mL are obtained based on

  Regarding the posture of the mirror 26, the mirror surface looks up above the imaging center point C1 with respect to the direction from the origin mr of the mirror coordinate system to the imaging center point C1, and the mirror surface is below the imaging center point C1. There are two solutions to the attitude of looking down. The control / arithmetic unit 18 employs one of the two solutions based on a predetermined positional relationship range for the display 16, the mirror 26, and the first imager 10.

The control / arithmetic unit 18 uses the translation vector T _QL and the rotation matrix R _QL obtained for the mirror image of the display 16 and the translation vector T _mL and the rotation matrix R _mL obtained for the mirror 26. According to (Equation 20), a translation vector T _dL and a rotation matrix R _dL for the display 16 are obtained.

The control / arithmetic unit 18 obtains a translation vector T _dL and a rotation matrix R _dL for each arrangement position (L = 1, 2, 3,... Lmax). Then, the final translation vector T _d and the rotation matrix R _d are obtained by optimization using the translation vector T _dL and the rotation matrix R _dL obtained for each of L = 1 to Lmax. The optimization is performed so that the error regarding the position of each dot 30 of the calibration pattern 28 reproduced by the translation vector T _d and the rotation matrix R _d is smaller than a predetermined value or minimized. The true position of each dot 30 is the position of each dot actually displayed on the display 16. Such processing, for example, a square error of the position of each dot 30 is reproduced by the translation vector T _d and the rotation matrix R _d (total value of the square of errors for all of the dot 30) is minimum A least square method for obtaining the translation vector T _d and the rotation matrix R _d may be used.

The control / arithmetic unit 18 performs the same processing based on the translation vector T _{d —} 2 starting from the origin C2 of the second imaging coordinate system and the origin U of the display coordinate system, and the second imaging coordinate system. A rotation matrix R _{d — 2} representing the orientation of the display coordinate system is obtained. Then, according to (Expression 21), the translation vector R _rel starting from the origin C1 of the first imaging coordinate system and ending at the origin C2 of the second imaging coordinate system, and the second based on the first imaging coordinate system A rotation matrix R _rel representing the orientation of the imaging coordinate system is obtained.

  In this manner, the mutual relationship among the display coordinate system, the first imaging coordinate system, and the second imaging coordinate system is obtained. That is, a translation vector from the origin C1 of the first imaging coordinate system to the origin of the other two coordinate systems and a rotation matrix representing the postures of the other two coordinate systems with respect to the first imaging coordinate system C1 are obtained. . Such mutual relationship between coordinate systems is used for the above-described shape measurement.

In the above description, the mirror shape of the mirror is a circular shape having a known diameter. A mirror having another shape whose mirror surface shape and size are known may be used. For example, a polygonal or elliptical mirror may be used. In this case, the control / calculation unit 18 obtains the translation vector T _mL and the rotation matrix R _mL based on the internal parameters, the position of the projection shape of the mirror, the projection shape, and the known shape and size.

  Thus, in the calibration, the calibration pattern displayed on the display 16 and reflected in the mirror and the projection shape of the mirror are used for the calculation. Since the display 16 used for measuring the shape of the target 20 is also used for displaying a calibration pattern, the configuration for performing calibration is simplified. Furthermore, the mirror 26 provides the first imaging device 10 and the second imaging device 12 with its own projected shape together with a mirror image of the calibration pattern. Since the mirror 26 is used, a plurality of types of parameters such as a calibration pattern and its own projected shape are provided to each image pickup device, so that the configuration for performing calibration becomes simple.

DESCRIPTION OF SYMBOLS 10 1st image pick-up device, 12 2nd image pick-up device, 14 Image pick-up part, 16 Display, 18 Control / calculation part, 20 Target object, 22, 23, 32 Incident light path, 24, 25, 34 Reflected light path, 26 Mirror, 28 Calibration Pattern, 30 dots.

Claims (7)

A pattern display unit that displays a plurality of types of display patterns one by one on the target;
A display position code associating unit for associating display position codes based on the plurality of types of display patterns with respect to display points on the pattern display surface of the pattern display unit;
An imaging unit that captures each pattern reflected on the target;
A measurement point position code association unit that associates a measurement point position code with each point on each reflected image captured by the imaging unit;
A position for obtaining a positional relation amount representing a positional relation between the imaging unit and a display point on the pattern display surface corresponding to a measurement point on the target based on the display position code and the measurement point position code A relational computation unit;
Geometrical optical relationship established for the optical path from the display point on the pattern display surface to the imaging unit via the measurement point on the target, and including the positional relationship represented by the positional relationship amount Based on the relationship, a measurement point calculation unit for obtaining the position of the measurement point on the target,
A shape measuring apparatus comprising: In the shape measuring apparatus according to claim 1,
The plurality of types of display patterns are binary in luminance.
The display position code association unit includes:
The position of the display point constituting the pattern display surface is encoded as the display position code represented by a binary code having the same number of digits as the number of the plurality of types of display patterns,
The measurement point position code association unit
The measurement point position represented by a binary code having the same number of digits as the number of the plurality of types of display patterns, for each pixel constituting the reflected image based on each reflected image captured by the imaging unit A shape measuring apparatus that performs coding as a code. In the shape measuring device according to claim 1 or 2,
The imaging unit captures each pattern reflected on the target from a first position and a second position, and generates a first image and a second image, respectively.
The positional relationship calculation unit, based on the first image and the second image, as the positional relationship amount for which the stereo matching condition is satisfied for the measurement point on the target, respectively, Find the positional relationship quantity,
The shape measuring apparatus, wherein the measurement point calculation unit obtains a position of a measurement point on the target based on the first positional relationship amount and the second positional relationship amount that establish a stereo matching condition. In the shape measuring device according to claim 3,
The measurement point calculator is
Each of the plurality of measurement points on the target is solved by solving an equation based on the geometric optical relationship for each of the plurality of measurement points on the target and the continuity of planar elements at adjacent measurement points. Equipped with an equation calculation unit to find the position,
The equation calculator is
A shape measuring apparatus that solves the equation using a position of a representative point whose position is obtained based on a stereo matching condition among a plurality of measurement points on the target. A pattern display section for reflecting the display pattern on the target;
An imaging unit that captures each pattern reflected on the target;
A calculation unit that obtains the position of the measurement point on the target based on the positional relationship and the posture relationship of the pattern display unit and the imaging unit, and the reflected image captured by the imaging unit;
At the time of calibration for obtaining the positional relationship and the posture relationship between the pattern display unit and the imaging unit, the pattern display unit, the pattern display unit, the optical path from the pattern display unit to the mirror and the imaging unit is formed. A mirror, and the imaging unit,
At the time of the calibration, the calculation unit
The positions of the pattern display unit and the imaging unit based on the shape of the mirror photographed by the imaging unit and the mirror image of the calibration pattern that is copied to the mirror by the pattern display unit and photographed by the imaging unit A shape measuring device for obtaining a relationship and a posture relationship. In the shape measuring apparatus according to claim 5,
At the time of the calibration, the calculation unit
A shape measuring apparatus for obtaining a geometric internal parameter of the pinhole camera when the imaging unit is replaced with a pinhole camera based on a mirror image of the calibration pattern imaged by the imaging unit . In the shape measuring apparatus according to claim 5 or 6,
At the time of the calibration, the calculation unit
Based on the shape of the mirror imaged by the imaging unit, the positional relationship and posture relationship of the imaging unit and the mirror are obtained, and based on the obtained positional relationship and posture relationship, the pattern display unit and the imaging unit A shape measuring apparatus characterized by obtaining a positional relationship and a posture relationship of each other.

Images