JP2004325096A - Image processing method in lattice pattern projective method, image processor and measurement device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically acquire a fringe wave number even for a discontinuous portion of a fringe in a lattice pattern projective method. <P>SOLUTION: This image processing method includes steps for: acquiring a first image of a measured object when projecting a first lattice pattern comprising fringes of light and shade onto the measured object; acquiring a second image of the measured object when projecting a second lattice pattern obtained by reversing the light and shade of the fringes of a part of the first lattice onto the measured object; and specifying a pixel related to the fringe obtained by reversing the light and shade on the basis of the first and second images. Because the light and shade of the specific fringe is intentionally reversed, the fringe wave number of the reversed fringe is self-evident, and the fringe of the pixel is determined when the pixel related to the reversed fringe is specified from a difference between the first image and the second image. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a technique for measuring a three-dimensional shape of a measured object by projecting a grid pattern on the measured object.
[0002]
[Prior art]
The measurement of three-dimensional shapes is being used not only in industry but also in many areas of society such as medicine, biology, archeology, and research and restoration of works of art. In three-dimensional measurement, a non-contact type optical measurement using light is more demanded than a conventional contact type. Optical measurement area measurement is classified into a light cutting method that measures the area by scanning a single linear slit light in a direction orthogonal to the slit direction, and a measurement method that collectively measures each surface. Is done. The latter is more advantageous from the viewpoint of measurement time, and this development is desired.
[0003]
The light-section method is a well-established technique, and various modes have been developed to enable highly reliable measurement. However, "optical three-dimensional measurement" edited by Toru Yoshizawa [New Technology Communications] p. As shown in FIG. 31, a mechanical moving mechanism such as a slit light projecting unit is required, and this becomes a bottleneck, so that the measurement time cannot be shortened and the efficiency is poor. Therefore, there is a problem in application in a field where measurement time needs to be reduced. In addition, there is also a problem that maintenance adjustment is troublesome because of the mechanical moving mechanism.
[0004]
On the other hand, a typical method of processing surfaces in a lump is a moire method of acquiring contour lines by moire fringes (“optical three-dimensional measurement” edited by Toru Yoshizawa [New Technology Communications] p.65), a striped grid There is a grid pattern projection method for observing a grid pattern formed by projecting an image onto an object (“Three-dimensional optical measurement” edited by Toru Yoshizawa [New Technology Communications] p. 83). In particular, the latter is expected at present in view of the measurement accuracy and the price of the device.
[0005]
As shown in FIG. 1, the grid pattern projection apparatus includes a projection unit 1000, a projection system including a projector lens 1002 and a grid 1001, and an imaging unit that captures a deformed grid pattern 1003 projected on a measured object 1004 via a lens 1006. 1005 including two imaging systems. The projection unit 1000 projects a striped grid formed by the grid 1001 onto the measured object 1004 via the lens 1002. The grid pattern generated by the projection is deformed by the unevenness of the measured object. The image capturing unit 1005 captures the deformed grid pattern 1003 from an angle different from the projection direction, converts the image into an electric signal via a light receiving element such as a CCD (Charge Coupled Device), and stores the signal in a storage device. By analyzing this, the three-dimensional shape of the measured object 1004 is measured. Each light receiving element outputs an electric signal such as a voltage corresponding to the illuminance (luminance) at that point. The grating 1001 was previously formed on a glass surface or the like and was fixed. However, recently, a grating made of liquid crystal or the like has been put into practical use, and a computer can turn on or off a striped grating embodied in a liquid crystal device at high speed. Control.
[0006]
Normally, a reference plane orthogonal to the optical axis of the projection unit 1000 is set, and two axes of the stripe direction Y and the stripe orthogonal direction X are set thereon. For details, see Non-Patent Document 1 (H. Lu et al., "Automatic Measurement of 3-D Object Shapes Based on Computer-Generated Reference Surface," Bull. Japan Soc., No. 21, Sec. p251 (1987)), in this method, the grid pattern is expressed as a wave of illuminance I with respect to a position on an axis X orthogonal to the stripe, and the measured object 1004 is set on a reference plane. Phase of the wave Φ (= Φ ₂ ) Or the phase Φ of the wave about the reference plane ₁ And the phase Φ of the wave when the measured object 1004 is set on the reference plane ₂ And the difference ΔΦ (= Φ ₂ −Φ ₁ Analyze using). Thus, the grating pattern projection method is also called “modulation grating phase method”. The reason for using the phase difference ΔΦ of the wave here is to improve the accuracy because the systematic error in the measurement disappears by subtraction. In this method, the fringe wave (pair of bright fringes and dark fringes) numbers can be arbitrarily set as long as they are relative.
[0007]
Since a detailed calculation formula is disclosed in Non-Patent Document 1, detailed description is omitted here. However, as shown in FIG. ₁ Is the distance between the grating 1001 and the lens 1002, the distance between the intersection of the optical axis of the imaging unit 1005 and the optical axis of the projection unit 1000 and the lens 1002, the grid interval, the optical axis of the imaging unit 1005 and the light of the projection unit 1000. It is obtained by calculation from the optical geometrical positional relationship such as the angle of the axis. On the other hand, (Φ ₂ = 2π × fringe wave number + fringe internal phase angle), which is obtained by measurement. As described below, it is particularly important to specify the fringe wave number.
[0008]
In order to obtain the fringe internal phase angle, scanning is performed along the X-axis (see FIG. 1) in the fringe orthogonal direction set on the reference plane. Even if it is a so-called Ronchi grating in which the fringe grating is binarized as well as the sine wave grating, the vicinity of the boundary between the bright and dark fringes is caused by a light diffraction phenomenon and a discretization error due to the light receiving element. The illuminance I of the grid pattern along the X-axis in the direction perpendicular to the stripes changes. The illuminance distribution is recognized as a wave, and the phase angle in the fringe wave starting from the base point of the sine wave is obtained. In addition to the method of setting the peaks and valleys of the wave to π / 2 and 3π / 2 (Non-Patent Document 1 described above), a method of setting the point at the median of the section from the valley to the peak to 0 in the fringe wave phase angle, etc. Exists. Regardless of which method is used, the fringe wave number changes at the point where the fringe internal phase angle is 0, and one fringe wave is classified there.
[0009]
As described above, since the projected grating pattern 1003 is recognized as a wave of illuminance I, its phase Φ is expressed as 2π × striped wave number + striped internal phase angle. Therefore, in the grid pattern projection method or the like, three-dimensional measurement cannot be performed unless the fringe wave number is known. When the fringe wave is continuous, the fringe wave number increases or decreases by one along the X axis, so that the phase difference ΔΦ can be easily calculated. However, as shown in FIG. 2, the stripes may be discontinuous. FIG. 2 shows a scene in which a spherical object is arranged on a reference plane. In a normal order, a dark stripe is generated at a portion where a light stripe should appear, and three dark stripes are continuously formed. , The discontinuities of the stripes are clarified. That is, the line 1100 on the plane and the line 1101 on the sphere are the same line, but the position of the line 1100 on the reference plane and the position of the line 1101 on the sphere are far apart and discontinuous. . When the fringes are discontinuous in this way, the continuity of the fringe wave numbers as described above cannot be used, so that it is not easy to calculate the phase difference. In addition, if the fringe wave number is incorrectly recognized, a measurement error occurs.
[0010]
On the other hand, as a method of automatically acquiring a fringe wave number in response to a discontinuity of a fringe, many studies have been made on a black-and-white fringe lattice as outlined in Non-Patent Document 3 described below. Have been done, but none can be said to be successful. In addition, a method using a color fringe grating is known (Patent Document 1 and Non-Patent Document 2). Theoretically, if the number of colors is increased, it becomes easier to determine which stripe wave number the stripe is, even if the stripes are discontinuous. However, the method using color has a problem that it is not easy to remove the chromatic aberration of the lens, and the lens becomes expensive if the chromatic aberration is removed. Further, there is a problem that the types of colors that can be used are easily affected by the color of the object and are limited, so that it is not possible to cope with a high degree of discontinuity of stripes.
[0011]
In addition to this, there is a method called a spatial coding method (“optical three-dimensional measurement” edited by Toru Yoshizawa [New Technology Communications], p. 79). The space coding method is a method of coding each point in a measurement target space with a binary code and inputting a distance image with a small number of projections. From the projector L, pattern lights whose light and dark pitches double change by the masks A, B and C, respectively, are sequentially projected. Assuming that a light passing portion is 1 and a light blocking portion through which light does not pass is 0, projection light is binary coded pattern light. The measurement target onto which the pattern light is projected is input as a grayscale image by the camera Q. In this method, ∠QPL is obtained using the code of the stripe to which the point P in the space belongs, 、 PQL is obtained from the position of the point P captured by the camera Q, and the position of the point P is measured by triangulation. . As described above, in the spatial coding method, stripe numbers are given to each point on the surface of an object existing in three dimensions, so that "the original purpose is a robot vision, and the accuracy is not sufficient in terms of measurement."("Measurement of surface shape of reflective object using grid pattern projection method", Masayuki Yamamoto et al., Journal of Precision Engineering Society, Vol. 64, No. 8, p. 1171, (1998)). Further, there is no viewpoint of solving the problem of discontinuity of the fringe (phase). Further, there is no viewpoint of determining a shadow.
[0012]
[Patent Document 1]
JP-A-3-192474
[Non-patent document 1]
H. Lu et al. , "Automatic Measurement of 3-D Object Shapes Based on Computer-Generated Reference Surface", Bull. Japan Soc. of Prec. Eng. Vol. 21, no. 4, p251 (1987)
[Non-patent document 2]
W. Liu et al. , Color Coded Projection Grating Method for Shape Measurement with a Single Exposure, Applied Optics Vol. 39, No. 20, p3503 (2000)
[Non-Patent Document 3]
Xiao Yuan He et al. "Proposed Algorithm for Phase Unwrapping," Applied Optics Vol 41, No. 5, pp. 1-64. 35, p. 7422, Dec, 2002
[0013]
[Problems to be solved by the invention]
Accordingly, an object of the present invention is to provide a novel technique for automatically acquiring a fringe wave number even in a discontinuous portion of a fringe in a grid pattern projection method in order to solve the above problems. It is.
[0014]
[Means for Solving the Problems]
An image processing method in a grid pattern projection method according to a first aspect of the present invention is a method for projecting a first grid pattern composed of bright and dark stripes onto a measured object, A first image acquiring step of acquiring a first image, and an object to be measured in a case where a second lattice pattern obtained by inverting the brightness of some of the stripes of the first lattice pattern is projected on the object to be measured. A second image obtaining step of obtaining a second image of the above, and a step of identifying a pixel related to a stripe whose light and darkness has been inverted based on the first and second images.
[0015]
In order to intentionally reverse the brightness of a specific stripe, the stripe wave number of the inverted stripe is self-evident, and if a pixel related to the inverted stripe can be identified from the difference between the first image and the second image. , The stripes of the pixel can be determined. Further, even when a plurality of stripes are inverted, the stripe wave number can be identified by identifying the pixel related to the inverted stripe.
[0016]
Further, in the first aspect of the present invention, the method may further include a step of specifying a stripe wave number of the inverted stripe corresponding to the specified pixel. Since a fringe wave is composed of a set of light and dark fringes, the fringe wave number is specified from the fringe number or the like.
[0017]
Further, in the first aspect of the present invention, for a predetermined pixel in the first or second image, calculating a fringe internal phase angle based on a position in a fringe wave section specified by the fringe wave number of the pixel. May be further included.
[0018]
In the second image acquiring step described above, the second grid pattern is changed a plurality of times so that the brightness of all the stripes is inverted at least once, and the second image corresponding to each second grid pattern is changed. May be obtained. For example, the brightness of the stripes is reversed one by one.
[0019]
Further, the above-described second image acquisition step performs a maximum of 2 based on the imaging stage number n (n is an integer of 1 or more). ⁿ In a case where a grid pattern including light and dark stripes of a book is projected on the measured object, a step of acquiring an image of each photographing stage of the measured object may be included.
[0020]
Also, in the first aspect of the present invention, based on the first image, assigning a first code to a pixel related to a bright stripe, and assigning a second code to a pixel related to a dark stripe, Identifying pixels associated with the non-inverted stripe based on the first and second images; and assigning pixels associated with the non-inverted stripe to corresponding pixels in the first image. Assigning a code different from the code assigned to the corresponding pixel of the first image for the pixel associated with the inverted stripe, and assigning a code different from the code assigned to the corresponding pixel of the first image. Specifying a fringe wave number from a code sequence of pixels.
[0021]
According to the image processing method in the grid pattern projection method according to the second aspect of the present invention, a maximum of 2 ⁿ When a grid pattern including light and dark stripes of a book is projected on the object to be measured, a step of acquiring an image of each imaging stage for the object to be measured; Or a judgment step of judging whether the pixel belongs to a dark stripe or a dark stripe, and, for a predetermined pixel, a light-dark sequence for a predetermined photographing stage or a code sequence based on a code associated with light and a code associated with dark. Identifying step.
[0022]
By determining whether a given pixel belongs to a light or dark stripe at each shooting stage, a light-dark sequence or an equivalent code sequence corresponding to the stripe number of the given pixel can be specified. Become like
[0023]
The above-described method can be implemented by a combination of a program and a computer, and the program is stored in a storage medium or a storage device such as a flexible disk, a CD-ROM, a magneto-optical disk, a semiconductor memory, and a hard disk. . In some cases, it is distributed as a digital signal via a network or the like. The intermediate processing result is temporarily stored in a memory.
[0024]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 3 shows a functional block diagram according to one embodiment of the present invention. In the present embodiment, the main processing in the present embodiment is performed and the image processing apparatus 5 which is a computer and the projection unit 11 such as a lamp connected to the image processing apparatus 5 and forming a grid are formed. A liquid crystal projector 1 having a liquid crystal panel 12 and a lens 13 as described above, and a lens 4, a CCD 42, and a camera 4 having a memory (not shown) constitute a measuring device. Note that the measured object 3 is arranged on the reference plane 2, and a grid pattern is projected from the liquid crystal projector 1. On the reference plane 2, the X-axis is the direction perpendicular to the stripe, the Y-axis is the direction of the stripe, and the optical axis of the camera 4 is provided in the Z-axis direction.
[0025]
The image processing device 5 includes a projection control unit 56, an image acquisition unit 51, a light / dark reversal determination unit 53, a phase angle calculation unit 54, an altitude calculation unit 55, and a grid for storing images captured by the camera 4. A pattern storage unit 57 and a measurement data storage unit 59 for storing calculation results of the phase angle calculation unit 54 and the altitude calculation unit 55 are included. Further, the image processing device 5 secures a work memory area 58 in, for example, a main memory. The projection control unit 56 controls the liquid crystal panel 12 of the liquid crystal projector 1 to project a predetermined lattice pattern. The image acquisition unit 51 reads from the camera 4 an image of the measured object 3 onto which the predetermined grid pattern has been projected, captured by the CCD 42 of the camera 4. Each pixel of the image is represented by RGB, but is processed by an illuminance value (brightness value) in the present embodiment. The light / dark reversal determination unit 53 generates light / dark reversal determination data based on the illuminance values of the two images acquired by the image acquisition unit 51. The phase angle calculation unit 54 includes a stripe wave number determination unit 541 that determines the stripe wave number of each pixel by processing the light-dark inversion determination data generated by the light-dark inversion determination unit 53. By calculating the phase angle, the phase angle of each pixel is calculated by (2π × the fringe wave number + the fringe internal phase angle). The altitude calculation unit 55 calculates the altitude of the measured object 3 from the reference plane 2 from the phase angle calculated for each pixel, as described in Non-Patent Document 1 described above. The processing contents of the altitude calculation unit 55 are not the main part of the present invention, but are described in detail in Non-Patent Document 1 and the like, and will not be described hereafter.
[0026]
[Embodiment 1]
In the present embodiment, as shown in FIG. 2, a grid pattern (referred to as an inverted grid pattern) is used by using a grid in which the light and dark of one stripe are inverted among the grids in which bright stripes and dark stripes repeatedly appear. ) Is projected, and an image of the inverted lattice pattern is taken. A difference image of the illuminance between the image of the inverted grid pattern and the image of the normal grid pattern is generated, and pixels related to the inverted stripe are specified. In this way, the stripe wave number of the inverted stripe can be assigned to the pixel related to the inverted stripe. Then, if the above-described processing is performed by inverting the brightness of all the stripes, the stripe wave numbers are assigned to the pixels on which the stripes are projected. Pixels in which no change is found even after inverting the brightness of all the stripes are shades. That is, as shown in FIG. 4, both a shadow, which is a dark portion formed when an object is irradiated with light, and a shadow, which is obtained by projecting the shadow of the object on a projection surface, are captured in black. Therefore, the height of the measured object 3 from the reference plane 2 cannot be measured for a pixel determined to be “dark” using any of the inverted grid patterns, and thus can be determined to be “shadow”.
[0027]
Hereinafter, the processing of the measuring device according to the present embodiment will be described with reference to FIGS. First, the projection control unit 56 instructs the liquid crystal projector 1 to project a normal lattice pattern, and the liquid crystal panel 12 of the liquid crystal projector 1 forms a normal lattice according to the instruction and projects it on the measured object 3. (Step S1). In the present embodiment, a grating as shown in FIG. That is, it is a grid in which light stripes and dark stripes are repeatedly arranged at predetermined intervals.
[0028]
On the other hand, the camera 4 captures an image of the grid pattern of the measured object 3 on which the above-described grid pattern is projected by the CCD 42 via the lens 41, and stores the image in a memory built in the camera 4 (step S3). ). In the image of the lattice pattern, it is assumed that stripes are arranged in the horizontal direction as shown in FIG. Then, the image acquisition unit 51 of the image processing device 5 reads the captured image of the lattice pattern from the camera 4 and stores the image in the lattice pattern image storage unit 57 of the image processing device 5. In addition, a table as shown in FIG. 7A is registered in the work memory area 58, for example. FIG. 7A is a management table related to the current measurement, and includes a row of pixel numbers, a row of illuminance values (illuminance values of an image for a normal grid pattern), a row of fringe wave numbers +, A row of the fringe internal phase angle is provided, and a reference illuminance value, a fringe wave number +, and a fringe internal phase angle are recorded for each pixel. Here, the fringe wave number + is registered in the form of a fringe wave number such as “1-1” and 1 (dark) or 2 (bright). In addition, the stripe internal phase difference is, for example, “1”, which means “1” π. In step S3, the illuminance value of each pixel is registered. Further, the number n of the stripe to be inverted is initialized to 1 (step S5).
[0029]
Next, the projection control unit 56 instructs the liquid crystal projector 1 to project a grid pattern in which the n-th stripe is inverted, and the liquid crystal panel 12 of the liquid crystal projector 1 instructs the liquid crystal projector 1 to invert the grid in which the n-th stripe is inverted. And project it on the measured object 3 (step S7). Here, since n = 1, a lattice as shown in FIG. 6B is used. That is, it is a lattice in which dark stripes of n = 1 are inverted to light stripes. Hereinafter, in the case of n = 2, a grid in which the second bright stripe is inverted to the dark stripe as shown in FIG. 6C is used, and in the case of n = 3, FIG. A grid is used in which the third dark stripe is inverted to a light stripe as shown.
[0030]
On the other hand, the camera 4 captures an image of the grid pattern of the measured object 3 on which the above-mentioned grid pattern is projected by the CCD 42 via the lens 41, and stores the image in a memory built in the camera 4 (step S9). ). It is assumed that stripes are arranged in the horizontal direction also in the image of the lattice pattern as shown in FIG. Then, the image acquisition unit 51 of the image processing device 5 reads the captured image of the lattice pattern from the camera 4 and stores the image in the lattice pattern image storage unit 57 of the image processing device 5. Further, a table as shown in FIG. 7B is registered in the work memory area 58, for example. FIG. 7B is a table for one inverted grid pattern, which is a row of pixel numbers, a row of illuminance values (illuminance values of the image for the current inverted grid pattern), and an image of a normal grid pattern. Are provided with a row of the difference (absolute value) between the illuminance value of the image and the illuminance value of the image of the inverted grid pattern of this time, a row of the determination result of inversion or non-inversion, and a row of the fringe wave number +. The absolute value of the difference between the illuminance value to be compared with the reference illuminance value for the pixel, the determination result of inversion or non-inversion, and the striped wave number + of the pixel determined to have been inverted are registered. ing. In step S9, the illuminance value of each pixel is registered.
[0031]
Then, the light / dark inversion determination unit 53 and the phase angle calculation unit 54 perform a process of specifying an inversion pixel (step S11). The process of specifying the inverted pixel will be described with reference to FIG. The light-dark reversal determination unit 53 calculates the absolute value of the difference in illuminance between the image of the normal grid pattern and the image of the grid pattern obtained by inverting the brightness of the n-th stripe this time, and stores the calculated absolute value in, for example, the work memory area 58. (Step S31). The absolute value of the difference between the illuminance values is registered, for example, in the table of FIG. Then, the stripe wave number determination unit 541 of the phase angle calculation unit 54 determines that the pixel whose absolute value of the illuminance difference is equal to or more than a predetermined value is an inverted pixel, calculates a stripe wave number + from n, and outputs data indicating “inversion”. And the fringe wave number + are registered in, for example, the tables of FIGS. 7A and 7B (step S33). As described above, the fringe wave number is a number for a set of light and dark fringes, and the fringe wave number + is a fringe wave number with a bright or dark identifier. For example, when the stripe number is n, the integer part of n / 2 becomes the stripe wave number. For example, if there is no remainder, a bright identifier is given, and if there is a remainder, a dark identifier is given. Then, it is determined that a pixel whose absolute value of the difference between the illuminance values is less than a predetermined reference value is not a pixel related to the inverted stripe, and data representing “non-inverted” is, for example, shown in FIG. Register in the table (step S35). Therefore, in FIG. 7B, data indicating “ambiguity” is not registered.
[0032]
There is a case where there is no problem by performing the processing as shown in FIG. 8, but there are also cases where it is not possible to determine the inversion or the non-inversion with only one threshold value. In such a case, a process as shown in FIG. 9 may be performed. The light / dark reversal determination unit 53 calculates the absolute value of the difference between the illuminance values of the image of the normal grid pattern and the image of the grid pattern obtained by inverting the brightness of the n-th stripe this time, and stores the calculated absolute value in, for example, the work memory area 58. (Step S41). The absolute value of the difference between the illuminance values is registered, for example, in the table of FIG. Then, the fringe wave number determining unit 541 of the phase angle calculating unit 54 determines that the pixel whose absolute value of the difference in illuminance value is equal to or more than the first predetermined value is an inverted pixel, calculates the fringe wave number + from n, Data indicating “inversion” and the stripe wave number + are registered in, for example, the tables of FIGS. 7A and 7B (step S43). The fringe wave number + is specified by the calculation as described above. Further, a pixel whose absolute value of the difference between the illuminance values is smaller than the second predetermined value is determined as a non-inverted pixel, and data representing “non-inverted” is registered in, for example, the table of FIG. 7B (step S44). ). The fringe wave number determination unit 541 of the phase angle calculation unit 54 determines that a pixel whose absolute value of the difference between the illuminance values is equal to or larger than the second predetermined value and smaller than the first predetermined value is an “ambiguous” pixel. Is determined, and data representing “ambiguity” is registered, for example, in the table of FIG. 7B corresponding to the pixel (step S45).
[0033]
Returning to FIG. 5, the fringe wave number determining unit 541 of the phase angle calculating unit 54 performs a process of specifying the fringe wave number + of the vague pixel (step S13). When the processing as shown in FIG. 7 is performed, there is no “ambiguous” pixel, so this step is skipped. The process of specifying the stripe wave number + of the ambiguous pixel will be described with reference to FIGS. 10 and 11A and 11B.
[0034]
The fringe wave number determination unit 541 scans FIG. 7B from an image of a normal lattice pattern and, as described in Non-Patent Document 3, scans FIG. A line is extracted (step S51). Then, FIG. 7B is scanned from the extracted lines of the image of the normal lattice pattern to extract the pixels determined to be ambiguous pixels and the pixels determined to be inverted pixels in the same line (step S53). Then, if the pixel having the highest illuminance value among the extracted pixels is an ambiguous pixel, the inverted pixel having the lowest illuminance value is determined if the pixel having the lowest illuminance value is an ambiguous pixel. Sets a fringe section based on the inverted pixel having the highest illuminance value (step S55).
[0035]
The processing in step S55 will be described with reference to FIGS. The repetition of light and dark stripes can be represented as a wave, and when one scan line of an image of a normal lattice pattern is extracted, its illuminance value can be represented by a wave as shown in FIG. In FIG. 11A, white circles are pixels determined to be ambiguous pixels, triangles are pixels determined to be non-inverted pixels, and black circles are pixels determined to be inverted pixels. Here, a case where the bright stripe is inverted is shown, and a pixel having a high illuminance value is determined to be an inverted pixel. On the other hand, pixels determined to be non-inverted pixels are pixels having a low illuminance value and pixels having a high illuminance value in other stripe wave sections. In step S53, the inverted pixel and the ambiguous pixel are extracted. In the example of FIG. 11A, the pixel having the lowest illuminance value among the extracted pixels is the ambiguous pixel (open circle), and thus the illuminance value is the highest. The inversion pixel 1201 is specified, and a stripe wave section is set based on the inversion pixel 1201. The fringe wave section has the width of the light fringe and the dark fringe, and the inverted pixel 1201 having the highest illuminance value is located at a position １ ／ (π in phase angle) of the fringe wave section. , And set the fringe section according to the position. On the other hand, FIG. 11B shows a case where dark stripes are inverted, and a pixel having a low illuminance value is determined to be an inverted pixel. The pixels determined to be non-inverted pixels are pixels having high illuminance values and pixel pixels having low illuminance values in other fringe wave sections. In step S53, the inverted pixel and the ambiguous pixel are extracted. In the example of FIG. 11B, the pixel having the highest illuminance value among the extracted pixels is the ambiguous pixel (open circle), and thus the illuminance value is the lowest. The inversion pixel 1202 is specified, and a fringe wave section is set based on the inversion pixel 1202. In the fringe wave section, the inverted pixel 1202 having the lowest illuminance value is located at a position of 3/4 of this fringe wave section (3π / 2 in phase angle), so the fringe wave section is set according to the position.
[0036]
An ambiguous pixel entering the fringe wave section set in this way is specified, and for an ambiguous pixel entering the first half of the fringe wave section, the fringe wave number is the same as the fringe wave number of the inverted pixel, and the “bright” stripe 7A and FIG. 7A, the stripe wave number + representing the “dark” stripe, which is the same as the stripe wave number of the inverted pixel, and representing the “dark” stripe, 7 (b) (step S57). Then, it is determined whether all the lines have been processed (step S59). If there is an unprocessed line, the process returns to step S51. If all the lines have been processed, the process returns to the original process.
[0037]
Returning to FIG. 5, when the fringe wave number + is specified in this way, it is determined whether n has reached the maximum value (step S15). If n is less than the maximum value, n is incremented by 1 (step S17), and the process returns to step S7. On the other hand, when n is the maximum value, in each line of the image of the normal lattice pattern, the stripe wave section is defined by the pixels for which the stripe wave number + is determined and the illuminance values of which are the maximum and the minimum. Then, the internal phase angle of the fringe is calculated based on the position in the fringe wave section and registered in, for example, the table of FIG. 7A (step S19). As described with reference to FIGS. 11A and 11B, the pixel having the maximum illuminance value indicates a position １ ／ of the fringe wave section, and the pixel having the minimum illuminance value indicates the fringe wave section. Is shown, the phase angle can be calculated based on the pixel position in the fringe wave section.
[0038]
Then, data of “shadow” is registered, for example, in the table of FIG. 7A with respect to the invariant pixels in the images of all the grid patterns (step S21). Since "shadow" is always determined to be "non-inverted" even if the stripes are inverted, the table of FIG. 7B for all the grid patterns is scanned, and any row is set to non-inverted. It can be determined that a certain pixel is a shadow. Therefore, data representing “shadow” is registered in the table of FIG. 7A corresponding to the pixel.
[0039]
In this way, data of the fringe wave number + and the fringe internal phase angle can be obtained for pixels other than the shadow. For example, the data of the table in FIG. 7A is registered in the measurement data storage unit 59. Also, if the fringe wave number and the fringe internal phase angle can be obtained, the overall phase can be calculated for each pixel by 2π × the fringe wave number + the fringe internal phase angle. The altitude of the measured object 3 from the reference plane 2 can be calculated. This data is stored in the measurement data storage unit 59, for example.
[0040]
In the above description, an example in which one stripe is inverted for one grid pattern is shown, but the invention is not necessarily limited to one stripe. That is, it is also possible to invert a plurality of stripes at intervals exceeding the discontinuous portions of the stripes. Even in such a case, if the above-described processing is executed for each inverted stripe portion, the stripe wave number of each pixel can be automatically specified.
[0041]
[Embodiment 2]
Using the method of the first embodiment increases the processing time. In addition, although a method of inverting a plurality of stripes at the same time has been described a little, the processing time can be shortened. There are countless methods for simultaneously inverting a plurality of stripes that can specify the stripe wave number of an arbitrary pixel. The following describes the most efficient binary stripe inversion method that has a simple inversion rule and is easy to develop.
[0042]
Briefly, in the first stage, a pattern of the entire area “white” is projected as shown in FIG. 12A, and in the second stage, half of the entire area is set to “white” as shown in FIG. 12B. A grid pattern in which the other half is “black” is projected, and in the third stage, as shown in FIG. 12C, １ ／ of the entire area is formed by “white”, “black”, “white”, and “black” stripes. The grid pattern to be divided is projected, and in the fourth stage, ８ of the entire area is “white”, “black”, “white”, “black”, “white”, “black”, “white” and “white” as shown in FIG. A grid pattern divided by "black" stripes is projected. In this manner, the processing of dividing the stripes of the lattice pattern of the previous stage into white stripes and black stripes to generate a lattice pattern is performed recursively. The division is repeated until the lattice pattern finally used in the first embodiment is obtained. In the case of a lattice pattern including M stripes, log ₂ Measurement is performed on M stages. On the other hand, in the case of inverting one by one, M stages are required, and the measurement time can be reduced. This lattice pattern group has the same configuration as the binary tree. Although FIGS. 12A to 12D are described assuming a complete binary tree, the present invention is not necessarily limited to the complete binary tree.
[0043]
The processing will be described below with reference to FIGS. First, the number of stages n is initialized to 1 (step S61). Then, the projection control unit 56 instructs the liquid crystal projector 1 to project the grid pattern of the nth stage, and the liquid crystal panel 12 of the liquid crystal projector 1 configures the grid of the nth stage according to the instruction, and (Step S63). In the first stage, a grating as shown in FIG. That is, it is a pattern of the entire area “bright”.
[0044]
On the other hand, the camera 4 captures an image of the grid pattern of the measured object 3 onto which the grid pattern of the n-th stage is projected by the CCD 42 via the lens 41, and stores the captured image in a memory built in the camera 4 (step S65). ). Then, the image acquisition unit 51 of the image processing device 5 reads the captured image of the lattice pattern from the camera 4 and stores the image in the work memory area 58 and the lattice pattern image storage unit 57 of the image processing device 5.
[0045]
Then, the light / dark reversal judgment unit 53 performs a light / dark judgment process (step S67). This light / dark determination processing will be described with reference to FIG. First, the phase angle calculation unit 54 determines whether or not it is the first stage (step S81). If the current stage is the first stage, an image having a predetermined illuminance value or less is detected, and “shadow” is registered in, for example, the table of FIG. 15 (step S109). Then, the processing returns to the original processing.
[0046]
The table of FIG. 15 includes a row of pixel numbers, a row of determination results of light (“0”) or dark (“shading”) in the first stage, and a row of light or dark (hereinafter “1”) in the second stage. )), A row of the light or dark judgment result in the third stage, a row of the light or dark judgment result in the fourth stage,. . . . A row of a fringe wave number and a row of a fringe internal phase angle are provided, and a light / dark sequence (sequence of 01) from the second stage to the final stage, a fringe wave number, and a fringe internal phase angle are registered for each pixel. Is done.
[0047]
On the other hand, when the current stage is not the first stage, the fringe wave number determination unit 541 specifies one line to be processed (step S83), and specifies one pixel to be processed among the lines (step S85). ), The illuminance value of the pixel is read out from the grid pattern image storage unit 57, for example. Then, the fringe wave number determination unit 541 determines whether the read illuminance value is equal to or greater than the first threshold (step S87). If the illuminance value is equal to or larger than the first threshold value, the pixel is determined to be “bright”, and “0” is registered in, for example, the table of FIG. 15 (step S89). Then, control goes to a step S103. If the illuminance value is not equal to or greater than the first threshold value, it is determined whether the illuminance value is equal to or less than the second threshold value (step S91). If the illuminance value is equal to or smaller than the second threshold, the pixel is determined to be “dark”, and “1” is registered in, for example, the table of FIG. 15 (step S93). Then, control goes to a step S103.
[0048]
If it is determined that the illuminance value is equal to or smaller than the second threshold value, the light / dark reversal determination unit 53 calculates (illuminance value of the previous stage−illuminance value of the current stage) for the same pixel, and stores it in the work memory area 58. It is stored (step S95). Then, the fringe wave number determination unit 541 determines whether the absolute value of the calculation result is equal to or greater than the third threshold (Step S97). If the absolute value of the calculation result is equal to or greater than the third threshold, if the value of the calculation result is positive, it is known that the illuminance value of the previous stage is larger, that is, it has been inverted from light to dark. Is registered, and if the value of the calculation result is negative, it is known that the illuminance value of the previous stage is smaller, that is, it has been inverted from dark to bright, so “0” is registered (step S99). The reason why the contrast is determined by referring to the image of the previous stage is as follows. That is, as shown in FIGS. 12A to 12D, when dividing according to a binary tree, the boundary between light and dark in the current stage is often clearly dark or light in the previous stage. For example, the first light-dark boundary from the left in the fourth stage is clearly a bright stripe in the third stage. Therefore, even if the brightness of the pixel is ambiguous in the current stage, if the brightness is clear or dark in the previous stage, the difference between the illuminance values is expected to be equal to or more than the third threshold value if the brightness is switched. You. In this way, light or dark is determined with reference to the previous stage. Even if the previous stage is referred to, for example, at the second boundary in FIG. 12D, since the previous stage is also a light-dark boundary, in some cases, even if the difference is calculated, it may remain unclear. . That is, when it is determined that the calculation result is less than the third threshold value, data representing “ambiguous pixel” is registered in the table of FIG. 14 (step S101). Step S99 may be executed for a further earlier stage. This is because there may be pixels that are clearly “bright” and more clearly “dark” in the stage two or more previous.
[0049]
Then, it is determined whether all pixels have been processed for the line specified in step S83 (step S103). If there is an unprocessed pixel, the process returns to step S85. If all pixels of a specific line have been processed, if there is a pixel determined to be an ambiguous pixel as a result of the processing of the current stage, the fringe wave number determination unit 541 determines, for the ambiguous pixel, A stripe section is set based on the stripe section length of the current stage and the illuminance value of each pixel of the line currently being processed, and it is determined whether the vague pixel belongs to a bright stripe or a dark stripe. Alternatively, “1” is registered (step S105). Then, it is determined whether or not all lines have been processed (step S107). If an unprocessed line exists, the process returns to step S83. On the other hand, when processing has been performed for all lines, the processing returns to the original processing.
[0050]
By performing such processing, it is possible to determine whether each pixel belongs to a bright or dark stripe in the current stage.
[0051]
Returning to the processing of FIG. 13, when the determination of the brightness is completed in step S67, it is determined whether the current stage is the final stage (step S69). If it is not the final stage, n is incremented by 1 (step S71), and the process returns to step S63. On the other hand, if it is the final stage, the stripe wave number is determined from the light-dark sequence from the second stage to the final stage shown in FIG. 15 and registered in the table of FIG. 15 (step S73). For example, when a light / dark sequence “00000” is obtained for a certain pixel, the pixel belongs to the stripe with the stripe number “0”. Therefore, the fringe wave number is “0”. Further, when the light / dark sequence “00001” is obtained, the pixel belongs to the stripe having the stripe number “1”. Therefore, the fringe wave number is “0”. Further, when the light / dark sequence “00010” is obtained, the pixel belongs to the stripe with the stripe number “2”. Therefore, the fringe wave number is “1”. By performing such processing, the fringe wave number is determined.
[0052]
Then, in each line of the image of the lattice pattern of the final stage, a fringe wave section is set by a pixel whose fringe wave number is determined and whose illuminance value is maximum and minimum, and based on a position in the fringe wave section. The internal phase angle of the stripe is calculated and registered in, for example, the table of FIG. 15 (step S75). As described with reference to FIGS. 11A and 11B, the pixel having the maximum illuminance value indicates a position １ ／ of the fringe wave section, and the pixel having the minimum illuminance value indicates the fringe wave section. Is shown, the phase angle can be calculated based on the position in the fringe wave section.
[0053]
In this way, data of the fringe wave number and the fringe internal phase angle can be obtained for pixels other than the shadow. For example, at least a part of the data in the table of FIG. 15 is registered in the measurement data storage unit 59. Also, if the fringe wave number and the fringe internal phase angle can be obtained, the overall phase can be calculated for each pixel by 2π × the fringe wave number + the fringe internal phase angle. The altitude of the measured object 3 from the reference plane 2 can be calculated. This data is stored in the measurement data storage unit 59, for example.
[0054]
By performing the above-described processing, the stripe wave number and the stripe internal phase angle of each pixel can be specified more efficiently.
[0055]
Similar results can be obtained by the following processing. First, the photographing stage number n is initialized to 1 (FIG. 16: step S111). Then, the projection control unit 56 instructs the liquid crystal projector 1 to project the grid pattern of the nth stage, and the liquid crystal panel 12 of the liquid crystal projector 1 configures the grid of the nth stage according to the instruction, and (Step S113). In the first stage, a grating as shown in FIG. That is, it is a pattern of the entire area “bright”. In the second stage, a grating as shown in FIG. Hereinafter, FIG. 12 (c), FIG. 12 (d). . . It becomes.
[0056]
On the other hand, the camera 4 captures an image of the grid pattern of the measured object 3 on which the grid pattern of the n-th stage is projected by the CCD 42 via the lens 41, and stores the captured image in a memory built in the camera 4 (FIG. 16). : Step S115). Then, the image acquisition unit 51 of the image processing device 5 reads the captured image of the lattice pattern from the camera 4 and stores the image in the lattice pattern image storage unit 57 of the image processing device 5. Then, it is determined whether the current projection and imaging are for the final stage (step S117). If it is not the final stage, n is incremented by 1 (step S118), and the process returns to step S113. In the case of the final stage, the light-dark reversal judgment unit 53 and the phase angle calculation unit 54 execute a light-dark judgment process (step S119). The light / dark determination processing will be described with reference to FIG.
[0057]
The phase angle calculation unit 54 specifies one line to be processed (step S131), and specifies one pixel to be processed belonging to the line (step S133). Then, the light-dark inversion determination unit 53 calculates a difference between the illuminance value of the pixel in the first stage and the illuminance value of the pixel in the final stage, and stores the difference in, for example, the work memory area 58 (step S135). Then, the fringe wave number determination unit 541 of the phase angle calculation unit 54 determines whether the absolute value of the difference is equal to or larger than the first threshold (step S137). If the absolute value of the difference is equal to or greater than the first threshold value, the first stage is in the entire area “bright”, so that inversion, that is, inversion from “bright” to “dark” has occurred. "1" is registered in the table shown in FIG. 18A, and "inversion" is registered in the table shown in FIG. 18B, for example (step S139). Then, control goes to a step S147.
[0058]
In the table of FIG. 18A, a row of a pixel number and a row of light and dark for registering whether each pixel of the image of the grid pattern serving as a reference of the final stage is “bright” or “dark”. , The row of the comparison result of the second stage, the row of the comparison result of the third stage, the row of the comparison result of the fourth stage,. . . . A row of the fringe wave number and a row of the fringe internal phase angle are provided, and for each pixel, the result of determining the brightness of the pixel, the code sequence for each stage, the fringe wave number, and the fringe internal phase angle correspond. It is attached. On the other hand, in the table of FIG. 18B, it is registered whether or not the brightness is inverted by comparing the row of the pixel number with the image of the grid pattern in the first stage and the image of the grid pattern serving as the reference in the final stage. A row for registering whether or not the brightness is reversed by comparing the image of the grid pattern in the second stage with the image of the grid pattern serving as a reference for the final stage, and a grid for the third stage. A row for registering whether or not the brightness is inverted by comparing the pattern image and the grid pattern image serving as the reference of the final stage; and a grid pattern image and a reference grid of the final stage in the fourth stage. A line for registering whether the contrast is inverted by comparing the pattern with the image, and. . . . And are included.
[0059]
Returning to the processing of FIG. 17, the fringe wave number determination unit 541 of the phase angle calculation unit 54 determines whether the absolute value of the difference calculated in step S135 is equal to or smaller than the second threshold (step S141). If the absolute value of the difference is equal to or smaller than the second threshold value, the first stage is in the entire area “bright”, and is determined to be non-inverted, that is, to remain “bright”, for example, as shown in FIG. "0" is registered in the table, for example, "non-inverted" is registered in the table shown in FIG. 18B (step S143). Then, control goes to a step S147.
[0060]
If a shadow exists in the image, “0” representing “bright” is registered here, but it is always “non-inverted”. Are treated as "shadow". Note that, in the present embodiment, the bright fringe at the left end is never inverted, so that the fringe cannot be used for measurement. If this part is also used, it is necessary to project a pattern of the entire area "dark" and compare it with an image of the pattern.
[0061]
If the absolute value of the difference calculated in step S135 exceeds the second threshold value and is less than the first threshold value, it is temporarily registered as an ambiguous pixel in, for example, the table of FIG. 18A (step S145).
[0062]
Then, it is determined whether or not the processing has been performed on all the pixels of the specific line (step S147). If there is an unprocessed pixel, the process returns to step S133. On the other hand, when processing is performed on all pixels on a specific line, a stripe section is set for pixels registered as ambiguous pixels based on the stripe section length of the final stage and the illuminance value of each pixel of the line currently being processed. Then, it is determined whether the ambiguous pixel belongs to a bright stripe or a dark stripe, and “0” and “non-inverted” are set for “bright”, and “1” and “1” are set for “dark”. "Reverse" is registered (step S149). Then, it is determined whether the processing has been completed for all the lines (step S151). If there is an unprocessed line, the process returns to step S131. On the other hand, when the processing has been performed for all the lines, the processing returns to the original processing.
[0063]
In this way, the determination of “bright” or “dark” and the determination of “inverted” or “non-inverted” for each pixel in the final stage are performed.
[0064]
Returning to the processing flow of FIG. 16, the light-dark reversal determination unit 53 and the phase angle calculation unit 54 perform a comparison process between the image of the final stage serving as a reference and the image of each stage (step S121). This comparison process will be described with reference to FIG.
[0065]
First, the stage number n is set to n = 2 (step S161). Then, the light-dark reversal determination unit 53 calculates the absolute value of the difference between the illuminance values of the image of the n-th stage and the image of the final stage, and stores the absolute value in, for example, the work memory area 58 (step S163). Then, the phase angle calculation unit 54 specifies one line to be processed (step S165), and further specifies one pixel to be processed (step S167). Then, the fringe wave number determination unit 541 determines whether the absolute value of the difference between the illuminance values of the pixel is equal to or greater than a first predetermined value (step S169). When the absolute value of the difference between the illuminance values of the pixel is equal to or larger than the first predetermined value, it is determined that the stripe is inverted, and for example, the determination result (in the table of FIG. If it is “bright (0)” in the final stage, “dark (1)” or vice versa is registered, and for example, data representing “reversal” is registered in the table of FIG. 18B (step S171). Then, control goes to a step S185.
[0066]
On the other hand, when it is determined that the absolute value of the difference between the illuminance values of the pixel is not equal to or greater than the first predetermined value, the stripe wave number determination unit 541 determines that the absolute value of the difference between the illuminance values of the pixel is the second value. Is determined to be less than or equal to a predetermined value (step S173). When the absolute value of the difference between the illuminance values of the pixel is equal to or smaller than the second predetermined value, the stripe is determined to be non-inverted, and for example, the determination result ( In the final stage, “bright (0)” is registered as “bright (0)”, and “dark (1)” is registered as “dark (1)”. The data representing "inversion" is registered (step S175). Then, control goes to a step S185.
[0067]
On the other hand, if the absolute value of the difference between the illuminance values of the pixel is less than the first predetermined value and exceeds the second predetermined value, the light / dark inversion determination unit 53 outputs the illuminance value of the (n-1) th stage. Then, the difference between the illuminance value and the illuminance value of the n-th stage is calculated and stored in the work memory area 58, for example (step S177). Then, the fringe wave number determination unit 541 determines whether the absolute value of the calculation result is equal to or greater than the third threshold (step S179). The reason why the data of the previous stage is referred to in this way is as described in step S95. If the absolute value of the calculation result is equal to or larger than the third threshold value, a determination result opposite to the determination result in the (n-1) th stage is registered in, for example, the table of FIG. An inversion state opposite to the (n-1) stage is registered (step S181). If the absolute value of the calculation result in step S177 is equal to or larger than the third threshold value, it means that it has been inverted in comparison with the (n-1) th stage, and the "brightness" “(0)” is registered as “dark (1)”, and “dark (1)” is registered as “bright (0)”. Further, if it is determined in the (n-1) th stage that it is "inverted", it means that it is further inverted. Therefore, if it is determined that it is "non-inverted", it is inverted from that state. Register “Reverse”.
[0068]
If it is determined in step S179 that the absolute value of the calculation result in step S177 is smaller than the third threshold, the pixel is temporarily registered as an ambiguous pixel in, for example, the table of FIG. 18A (step S183). Then, it is determined whether or not processing has been performed for all pixels on the specific line (step S185). If there is an unprocessed pixel, the process returns to step S167. On the other hand, when processing is performed on all the pixels of a specific line, for the pixels registered as ambiguous pixels, the stripe section is determined based on the stripe section length of the n-th stage and the illuminance value of each pixel of the line currently being processed. It is determined whether the ambiguous pixel belongs to a bright stripe or a dark stripe, and “0” is set for “bright” and “0” is set for “dark” in the table of FIG. “1” is registered, and “reversal” or “non-reversal” is registered in the table of FIG. 18B based on the determination result and the brightness of the corresponding pixel in the final stage (step S187).
[0069]
Then, it is determined whether the processing has been completed for all the lines (step S189). If there is an unprocessed line, the process returns to step S165. On the other hand, when the processing has been performed for all the lines, the processing returns to the original processing.
[0070]
In this way, the determination of “bright” or “dark” and the determination of “inversion” or “non-inversion” are performed for each pixel of each stage.
[0071]
Returning to the processing flow of FIG. 16, the fringe wave number determination unit 541 determines that “non-inverted” pixels are shadows in all the stages in the table of FIG. "Shadow" (step S123). In the example of FIG. 18A, the first pixel is determined to be a shadow.
[0072]
Then, the fringe wave number determination unit 541 determines the fringe wave number from the code sequence from the second stage to the final stage shown in FIG. 18A, and registers it in the table of FIG. 18A (step S125). ). For example, if a code sequence "00000" is obtained for a pixel, the pixel belongs to the stripe with stripe number "0". Therefore, the fringe wave number is “0”. Further, if a code sequence “00001” is obtained, the pixel belongs to the stripe having the stripe number “1”. Therefore, the fringe wave number is “0”. Further, if a code sequence “00010” is obtained, the pixel belongs to the stripe with stripe number “2”. Therefore, the fringe wave number is “1”. By performing such processing, the fringe wave number is determined.
[0073]
Then, in each line of the image of the lattice pattern of the final stage, a fringe wave section is set by a pixel whose fringe wave number is determined and whose illuminance value is maximum and minimum, and based on a position in the fringe wave section. The fringe internal phase angle is calculated and registered in, for example, the table of FIG. 18A (step S127). As described with reference to FIGS. 11A and 11B, the pixel having the maximum illuminance value indicates a position １ ／ of the fringe wave section, and the pixel having the minimum illuminance value indicates the fringe wave section. Is shown, the phase angle can be calculated based on the position in the fringe wave section.
[0074]
In this way, data of the fringe wave number and the fringe internal phase angle can be obtained for pixels other than the shadow. For example, at least part of the data in the table of FIG. 18A is registered in the measurement data storage unit 59. Also, if the fringe wave number and the fringe internal phase angle can be obtained, the overall phase can be calculated for each pixel by 2π × the fringe wave number + the fringe internal phase angle. The altitude of the measured object 3 from the reference plane 2 can be calculated. This data is stored in the measurement data storage unit 59, for example.
[0075]
By performing the above-described processing, the stripe wave number and the stripe internal phase angle of each pixel can be specified more efficiently.
[0076]
Although the embodiment of the present invention has been described above, the present invention is not limited to this. For example, the method of inverting the brightness of the stripe is not limited to the above-described embodiment, and another method may be used.
[0077]
Further, for example, the functional block diagram shown in FIG. 3 is an example, and the functional block diagram does not necessarily have to be the liquid crystal projector 1, and another projector may be used. In some cases, the actual program modules do not correspond to the functional units of the image processing apparatus 5. Further, the photographing control unit 56 may be realized by a device different from the image processing device 5 in some cases. Further, the liquid crystal projector 1 and the camera 4 may be configured to have some functions of the image processing device 5.
[0078]
Further, in the processing flow described above, a processing flow in which processing is performed for each projection and imaging of a lattice pattern is shown, but processing may be performed after projection and imaging of all lattice patterns. The above-described processing flow is an example, and the same function can be realized for different processing steps.
[0079]
(Appendix 1)
A first image acquisition step of acquiring a first image of the measured object when projecting a first grid pattern composed of light and dark stripes on the measured object;
A second image obtaining step of obtaining a second image of the measured object when projecting a second grid pattern in which the brightness of a part of the first grid pattern is inverted on a measured object; When,
Identifying a pixel associated with a stripe whose brightness has been inverted based on the first and second images;
An image processing method in a grid pattern projection method including:
[0080]
(Appendix 2)
Identifying a fringe wave number of the inverted stripe corresponding to the identified pixel
2. The image processing method according to claim 1, further comprising:
[0081]
(Appendix 3)
For a predetermined pixel in the first or second image, calculating a fringe internal phase angle based on a position in a fringe wave section specified from the fringe wave number of the pixel;
3. The image processing method according to claim 2, further comprising:
[0082]
(Appendix 4)
In the second image acquiring step,
The second grid pattern is changed a plurality of times so that the brightness of all the stripes is inverted at least once, and a second image corresponding to each of the second grid patterns is obtained.
The image processing method according to claim 1, wherein:
[0083]
(Appendix 5)
Identifying pixels related to shading based on a second image corresponding to all the second grid patterns and the first image;
5. The image processing method according to claim 4, further comprising:
[0084]
(Appendix 6)
The second image obtaining step includes:
Up to 2 based on the shooting stage number n (n is an integer of 1 or more) ⁿ Obtaining an image of each of the photographing stages for the measured object when a grid pattern including light and dark stripes of a book is projected on the measured object;
The image processing method according to claim 1, comprising:
[0085]
(Appendix 7)
Assigning a first code for pixels associated with light stripes and a second code for pixels associated with dark stripes based on the first image;
Identifying a pixel associated with a stripe whose light and darkness has not been inverted based on the first and second images;
Assigning, for the pixel associated with the non-inverted stripe, the same code as the code assigned to the corresponding pixel of the first image;
Setting a code different from a code assigned to a corresponding pixel of the first image for a pixel associated with the inverted stripe;
Obtaining a code sequence of each pixel for the predetermined imaging stage;
2. The image processing method according to claim 1, further comprising:
[0086]
(Appendix 8)
Up to 2 based on the shooting stage number n (n is an integer of 1 or more) ⁿ In the case where a grid pattern including light and dark stripes of a book is projected on the measured object, a step of acquiring an image of each of the imaging stages for the measured object,
For a predetermined pixel of the image in each of the shooting stages, a determination step of determining whether to belong to a bright or dark stripe,
For the predetermined pixel, identifying a code sequence by a light-dark sequence or a code associated with light and a code associated with dark for the predetermined imaging stage,
An image processing method in a grid pattern projection method including:
[0087]
(Appendix 9)
Identifying a fringe wave number for the predetermined pixel based on the light / dark sequence or the code sequence
9. The image processing method according to claim 8, further comprising:
[0088]
(Appendix 10)
In the determining step,
Using an image at a specific shooting stage and an image before the specific shooting stage, it is determined whether a predetermined pixel of the image belongs to a light or dark stripe.
The image processing method according to claim 8, wherein
[0089]
(Appendix 11)
For a predetermined pixel of the image in the final stage, calculating a fringe internal phase angle based on the position in the fringe wave section identified from the fringe wave number of the pixel,
9. The image processing method according to claim 8, further comprising:
[0090]
(Appendix 12)
Means for projecting a first grid pattern composed of light and dark fringes and a second grid pattern obtained by inverting the brightness of some fringes of the first grid pattern onto an object to be measured;
Means for capturing a first image of the object to be measured with respect to the first grid pattern and a second image of the object to be measured with respect to the second grid pattern;
An image processing device,
Means for identifying a pixel associated with a stripe whose brightness has been inverted based on the first and second images;
Means for identifying a fringe wave number of the inverted stripe corresponding to the identified pixel,
An image processing device having
A measuring device having:
[0091]
(Appendix 13)
Means for projecting, onto an object to be measured, a first grid pattern composed of light and dark fringes, and a second grid pattern obtained by inverting the brightness of some fringes of the first grid pattern.
Means for acquiring a first image of the object to be measured with respect to the first grid pattern and a second image of the object to be measured with respect to the second grid pattern;
Means for identifying a pixel associated with a stripe whose brightness has been inverted based on the first and second images;
An image processing apparatus having:
[0092]
(Appendix 14)
A first image acquisition step of acquiring a first image of the measured object when projecting a first grid pattern composed of light and dark stripes on the measured object;
A second image obtaining step of obtaining a second image of the measured object when projecting a second grid pattern in which the brightness of a part of the first grid pattern is inverted on a measured object; When,
Identifying a pixel associated with a stripe whose brightness has been inverted based on the first and second images;
A program for causing a computer to execute.
[0093]
(Appendix 15)
Up to 2 based on the shooting stage number n (n is an integer of 1 or more) ⁿ Means for projecting a grid pattern including light and dark stripes of the book on the measured object,
Means for photographing an image of each photographing stage for the object to be measured,
An image processing device,
Means for determining whether a given pixel of the image in each of the imaging stages belongs to a bright or dark stripe,
Means for specifying a code sequence by a code associated with a light-dark sequence or light and dark for the predetermined imaging stage, for the predetermined pixel,
Means for identifying a fringe wave number based on the light / dark sequence or the code sequence for the predetermined pixel;
An image processing device having
A measuring device having:
[0094]
(Appendix 16)
Up to 2 based on the shooting stage number n (n is an integer of 1 or more) ⁿ Means for projecting a grid pattern including light and dark stripes of the book on the measured object,
Means for acquiring an image of each of the imaging stages for the measured object,
Means for determining whether a given pixel of the image in each of the imaging stages belongs to a bright or dark stripe,
Means for specifying a code sequence by a code associated with a light-dark sequence or light and dark for the predetermined imaging stage, for the predetermined pixel,
An image processing apparatus having:
[0095]
(Appendix 17)
Up to 2 based on the shooting stage number n (n is an integer of 1 or more) ⁿ In the case where a grid pattern including light and dark stripes of a book is projected on the measured object, a step of acquiring an image of each of the imaging stages for the measured object,
For a predetermined pixel of the image in each of the shooting stages, a determination step of determining whether to belong to a bright or dark stripe,
For the predetermined pixel, identifying a code sequence by a code associated with a light-dark sequence or light and dark for the predetermined shooting stage,
A program for causing a computer to execute.
[0096]
【The invention's effect】
As described above, according to the present invention, a fringe wave number can be automatically acquired even in a discontinuous portion of a fringe in a grid pattern projection method.
[0097]
As a result, it is possible to obtain an altitude measurement value even for a discontinuous portion of a stripe without manual intervention.
[Brief description of the drawings]
FIG. 1 is a diagram showing an outline of a measuring device in a conventional technique.
FIG. 2 is a diagram for explaining a discontinuous portion of a stripe.
FIG. 3 is a diagram showing a functional block diagram of a measuring device of the present invention.
FIG. 4 is a diagram for explaining shadows.
FIG. 5 is a diagram showing a processing flow according to the first embodiment of the present invention.
FIG. 6A shows a normal grid, and FIGS. 6B to 6D show grids obtained by inverting one stripe.
FIGS. 7A and 7B are diagrams illustrating an example of a data table used for processing.
FIG. 8 is a diagram illustrating a first example of a process flow of a process of specifying an inverted pixel;
FIG. 9 is a diagram illustrating a second example of the processing flow of the inversion pixel specifying process.
FIG. 10 is a diagram showing a processing flow of a specific processing of a stripe wave number + of an ambiguous pixel.
FIGS. 11A and 11B are diagrams schematically showing a method of specifying a fringe wave number + of an ambiguous pixel.
FIGS. 12A to 12D are diagrams illustrating examples of a grating of each stage according to the second embodiment of the present invention.
FIG. 13 is a diagram showing a main processing flow according to the second embodiment of the present invention.
FIG. 14 is a diagram showing a processing flow of light / dark determination processing.
FIG. 15 is a diagram showing an example of a data table used for processing.
FIG. 16 is a diagram showing a second main processing flow in the second embodiment of the present invention.
FIG. 17 is a diagram illustrating a processing flow of light / dark determination processing.
FIGS. 18A and 18B are diagrams showing an example of a data table used for processing.
FIG. 19 is a diagram depicting a processing flow of a comparison processing;
[Explanation of symbols]
1 LCD projector 2 Reference plane 3 Object to be measured
4 Camera 5 Image processing device
11 Projection unit 12 Liquid crystal panel 13 Lens
41 lens 42 CCD
51 Image acquisition unit
53 Light / dark reversal judgment section 54 Phase angle calculation section
541 fringe wave number judgment unit 55 altitude calculation unit
56 Projection control unit 57 Grid pattern image storage unit
58 Work memory area 59 Measurement data storage

Claims (10)

A first image acquisition step of acquiring a first image of the measured object when projecting a first grid pattern composed of light and dark stripes on the measured object;
A second image obtaining step of obtaining a second image of the measured object when projecting a second grid pattern in which the brightness of a part of the first grid pattern is inverted on a measured object; When,
Identifying a pixel associated with a stripe whose brightness has been inverted based on the first and second images;
An image processing method in a grid pattern projection method including: 2. The image processing method according to claim 1, further comprising a step of specifying a fringe wave number of the inverted stripe corresponding to the specified pixel. For a predetermined pixel in the first or second image, calculating a fringe internal phase angle based on a position in a fringe wave section specified from the fringe wave number of the pixel;
The image processing method according to claim 2, further comprising: In the second image acquiring step,
2. The method according to claim 1, wherein the second grid pattern is changed a plurality of times so that the brightness of all the stripes is inverted at least once, and a second image corresponding to each of the second grid patterns is obtained. The image processing method described in the above. The second image obtaining step includes:
An image of each of the imaging stages for the measured object when a maximum of ²ⁿ grid patterns including bright and dark stripes are projected on the measured object based on the imaging stage number n (n is an integer of 1 or more) 2. The image processing method according to claim 1, further comprising the step of: Assigning a first code for pixels associated with light stripes and a second code for pixels associated with dark stripes based on the first image;
Identifying a pixel associated with a stripe whose light and darkness has not been inverted based on the first and second images;
Assigning, for the pixel associated with the non-inverted stripe, the same code as the code assigned to the corresponding pixel of the first image;
Setting a code different from a code assigned to a corresponding pixel of the first image for a pixel associated with the inverted stripe;
Obtaining a code sequence of each pixel for the predetermined imaging stage;
The image processing method according to claim 1, further comprising: An image of each of the imaging stages for the measured object when a maximum of ²ⁿ grid patterns including bright and dark stripes are projected on the measured object based on the imaging stage number n (n is an integer of 1 or more) Obtaining the
For a predetermined pixel of the image in each of the shooting stages, a determination step of determining whether to belong to a bright or dark stripe,
For the predetermined pixel, identifying a light / dark sequence from a first stage to a final stage or a code sequence by a code associated with light and a code associated with dark;
An image processing method in a grid pattern projection method including: Means for projecting a first grid pattern composed of light and dark fringes and a second grid pattern obtained by inverting the brightness of some fringes of the first grid pattern onto an object to be measured;
Means for capturing a first image of the object to be measured with respect to the first grid pattern and a second image of the object to be measured with respect to the second grid pattern;
An image processing device,
Means for identifying a pixel associated with a stripe whose brightness has been inverted based on the first and second images;
Means for identifying a fringe wave number of the inverted stripe corresponding to the identified pixel,
An image processing device having
A measuring device having: A first image acquisition step of acquiring a first image of the measured object when projecting a first grid pattern composed of light and dark stripes on the measured object;
A second image obtaining step of obtaining a second image of the measured object when projecting a second grid pattern in which the brightness of a part of the first grid pattern is inverted on a measured object; When,
Identifying a pixel associated with a stripe whose brightness has been inverted based on the first and second images;
A program for causing a computer to execute. Means for projecting a grid pattern including a maximum of 2 ⁿ light and dark stripes on the object to be measured based on a shooting stage number n (n is an integer of 1 or more);
Means for acquiring an image of each of the imaging stages for the measured object,
Means for determining whether a given pixel of the image in each of the imaging stages belongs to a bright or dark stripe,
Means for specifying a code sequence by a code associated with a light-dark sequence or light and dark for the predetermined imaging stage, for the predetermined pixel,
An image processing apparatus having:

Images