JP2016008837A - Shape measuring method, shape measuring device, structure manufacturing system, structure manufacturing method, and shape measuring program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a shape measuring method that can give highly accurate three-dimensional shape.SOLUTION: A device for implementing a shape measuring method comprises: a first arithmetic unit that acquires first three-dimensional information by combining information on a first pixel of a first image with information on a second pixel near a pixel matching the first pixel in a second image on which a pattern different from the first image is projected, and calculates the shape of part of a measurement object 2 from the first three-dimensional information; a second arithmetic unit that acquires second three-dimensional information by combining information on the first pixel of the first image with information on a third pixel different from the second pixel near a pixel matching the first pixel in the second image, and calculates the shape of part of the measurement object 2 from the second three-dimensional information; a selector that compares the partial shape figured out from the first three-dimensional information and the partial shape figured out from the second three-dimensional information, and chooses acquisition of three-dimensional information having a smoothly continuous shape; and a third arithmetic unit that figures out the shape of the measurement object 2 on the basis of the combination of pixels used for acquisition of the three-dimensional information.

Description

  The present invention relates to a shape measuring method, a shape measuring device, a structure manufacturing system, a structure manufacturing method, and a shape measuring program.

  A phase shift method is known as a method for measuring the three-dimensional shape of an object to be measured. A shape measuring apparatus using this phase shift method includes a projection unit, an imaging unit, and a calculation unit. The projection unit projects striped light (hereinafter referred to as pattern light) having a sinusoidal light intensity distribution onto the object to be measured, and shifts the initial phase by, for example, π / 2 three times. The imaging unit is arranged at a position different from the position of the projection unit. The imaging unit images the object to be measured in a state in which pattern light having an initial phase of 0, π / 2, π, and 3π / 2 is projected. The calculation unit applies the luminance data of each pixel in the four images captured by the imaging unit to a predetermined calculation formula, and obtains the phase of the fringe having the initial phase 0 in each pixel according to the surface shape of the object to be measured. Then, the calculation unit calculates the three-dimensional coordinate data of the object to be measured from the phase of the stripes in each pixel using the principle of triangulation.

  An apparatus using this phase shift method is disclosed in Patent Document 1, for example. This apparatus measures a three-dimensional shape of an object to be measured using a phase shift method. And this apparatus performs the defect inspection of the measured object based on the three-dimensional shape of the measured object.

US Patent Application Publication No. 2012/0236318

  When blur occurs between the time when the imaging unit projects the pattern light of the initial phase on the object to be measured and the time when the image of the initial phase pattern is projected and projected, the object to be measured calculated by the arithmetic unit The accuracy of the three-dimensional coordinate data decreases.

  An object of an aspect of the present invention is to obtain a highly accurate three-dimensional shape even when blurring occurs.

  According to the first aspect of the present invention, a plurality of different patterns are projected onto the object to be measured, the objects to be measured on which the plurality of patterns are projected are each captured, and the object to be measured on which the captured pattern is projected is included. The information of the first pixel of the first image and the second pixel in the vicinity of the pixel corresponding to the first pixel in the second image including the object to be measured and projected with a pattern different from the first image. The first three-dimensional information is acquired in combination with the information of the two pixels, the first three-dimensional information is repeatedly acquired, the shape of a part of the object to be measured is calculated, and the first image of the first image is calculated. The second three-dimensional information is obtained by combining the pixel information and the third pixel information different from the second pixel in the vicinity of the pixel corresponding to the first pixel of the second image, and 2) Obtaining the three-dimensional information repeatedly, calculating the shape of a part of the object to be measured, and obtaining from the first three-dimensional information Comparing the shape of a part of the object to be measured with the shape of a part of the object to be measured obtained from the second three-dimensional information, and selecting the acquisition of the three-dimensional information used for the calculation of smoothly connected shapes, A shape measuring method for obtaining the shape of an object to be measured based on a combination of pixels used to acquire selected three-dimensional information is provided.

  According to the second aspect of the present invention, a projection unit that projects a plurality of different patterns onto the measurement object, an imaging unit that images each of the measurement objects on which the plurality of patterns are projected, and the captured pattern are projected. The first pixel information of the first image including the measured object corresponds to the first pixel in the second image including the measured object that is captured and projected with a pattern different from the first image. The first three-dimensional information is acquired by combining with the information of the second pixel in the vicinity of the pixel to be processed, and the first three-dimensional information is repeatedly acquired to calculate the shape of a part of the object to be measured. Combining the calculation unit, the information of the first pixel of the first image, and the information of the third pixel different from the second pixel in the vicinity of the pixel corresponding to the first pixel of the second image The second three-dimensional information is acquired, and the second three-dimensional information is repeatedly acquired. The second computing unit that computes the part of the shape of the object to be measured obtained from the first three-dimensional information and the shape of the part of the object to be obtained obtained from the second three-dimensional information are compared. A third calculation for obtaining the shape of the object to be measured based on a combination of a selection unit that selects acquisition of three-dimensional information used for calculation of smoothly connected shapes and a pixel used for acquisition of the selected three-dimensional information And a shape measuring device including the unit.

  According to the third aspect of the present invention, there is provided a design apparatus for producing design information related to the shape of the structure, a molding apparatus for producing the structure based on the design information, and a shape of the produced structure. 6. A structure manufacturing system including the shape measuring apparatus according to 6, and an inspection apparatus that compares shape information related to the shape of the structure obtained by the shape measuring apparatus with design information.

  According to the fourth aspect of the present invention, the design information relating to the shape of the structure is produced, the structure is produced based on the design information, and the shape of the produced structure is claimed in claims 1 to 4. 5. A structure manufacturing method comprising: measuring with the shape measuring method according to any one of 5; and comparing shape information and design information regarding the shape of the structure obtained by the shape measuring method is provided. Is done.

  According to the fifth aspect of the present invention, the computer of the shape measuring apparatus includes: a projection unit that projects a plurality of different patterns onto the measurement object; and an imaging unit that images each of the measurement objects on which the plurality of patterns are projected. In addition, information on the first pixel of the first image including the object to be measured on which the captured pattern is projected, and a second object including the object to be measured on which the pattern different from the first image is projected. The first three-dimensional information is acquired by combining the information of the second pixel in the vicinity of the pixel corresponding to the first pixel in the image, and the first three-dimensional information is repeatedly acquired. A first calculation process for calculating the shape of the part, information on the first pixel of the first image, and a second pixel different from the second pixel in the vicinity of the pixel corresponding to the first pixel of the second image. The second three-dimensional information is obtained by combining the information of the three pixels, and the second three-dimensional information is obtained. It is obtained from the second calculation process for repeatedly obtaining information and calculating the shape of a part of the object to be measured, the shape of the part of the object to be obtained obtained from the first three-dimensional information, and the second three-dimensional information. A combination of a selection process for selecting the acquisition of three-dimensional information used to calculate a smooth series of shapes, and a combination of pixels used to acquire the selected three-dimensional information. A shape measurement program for executing a third calculation process for obtaining the shape of the object to be measured based on the third calculation process is provided.

  According to the aspect of the present invention, a highly accurate three-dimensional shape can be obtained even when blurring occurs.

It is a figure which shows the structure of a shape measuring apparatus. It is a figure which shows intensity distribution of the fringe pattern (pattern) in a projection area | region. It is a block diagram which shows the structure of the shape measuring apparatus shown in FIG. It is a block diagram which shows the structure of the calculating part shown in FIG. It is a wave form diagram which shows the sine wave calculated | required from the luminance value of a predetermined pixel. It is a figure which shows intensity distribution of the space code pattern (pattern) in a projection area | region. It is a figure which shows a standard pattern, four space code patterns, and a monochrome reference pattern. It is a figure which shows the relationship between a space code and an area number. It is a flowchart for demonstrating an example of a shape measuring method. It is a wave form diagram which shows the period of the wave | undulation produced by shaking. It is a figure which shows the combination of the neighboring pixel in the image data of each fringe pattern image. It is a figure which shows the relationship between the combination of a neighboring pixel, and a plane measurement error. It is a figure which shows the one part area | region which calculates | requires the combination of a pixel with a small plane measurement error. It is a block diagram which shows the structure of a structure manufacturing system. It is a flowchart which shows an example of a structure manufacturing method.

  Embodiments of the present invention will be described below with reference to the drawings. However, the present invention is not limited to this. Further, in the drawings, in order to describe the embodiment, the scale is appropriately changed and expressed, for example, partly enlarged or emphasized.

<First Embodiment>
FIG. 1 is a diagram showing the configuration of the shape measuring apparatus 1. FIG. 2 is a diagram showing the intensity distribution of the fringe pattern in the projection region 200. In FIG. 1, the right direction of the paper surface is the X1 axis, the direction orthogonal to the X1 axis and penetrating through the paper surface is the Y1 axis, and the direction orthogonal to the X1 axis and the Y1 axis is the Z1 axis. Further, when a three-axis coordinate system is set as shown in FIG. 1, in FIG. 2, the right direction of the paper surface is the X1 axis, the downward direction of the paper surface is the Y1 axis, and the direction from the back of the paper surface to the front is Z1 It becomes an axis.

  The shape measuring device 1 is a device that measures the three-dimensional shape of an object to be measured (measuring object, object to be measured) 2. As shown in FIG. 1, the shape measuring apparatus 1 includes a projection unit 10, an imaging unit 50, and an arithmetic processing unit 60.

  The projection unit 10 projects (projects) linear light, that is, one-dimensional light (hereinafter, referred to as line light 100) onto the projection region 200. As illustrated in FIG. 1, the projection unit 10 includes a light generation unit 20, a projection optical system 30, and a scanning unit 40. The light generation unit 20 includes a laser light source, a condensing lens, a cylindrical lens, and the like, and generates the line light 100 that is a primary image that is not modulated in the second direction. The projection optical system 30 forms an image of the line light 100 generated by the light generation unit 20 at a predetermined position in the projection area 200. The projection optical system 30 includes a transmission optical element or a reflection optical element such as one or a plurality of condenser lenses. The line light 100 emitted from the projection optical system 30 is projected onto the projection region 200 via the scanning unit 40. In the projection region 200, the one-dimensional direction of the line light 100 is the second direction D2. The length in the second direction D2 in the projection region 200 depends on the viewing angle θ when the projection unit 10 projects the line light 100 and the distance from the projection unit 10 (that is, the scanning unit 40) to the DUT 2. It is determined. In the example shown in FIG. 1, the DUT 2 is arranged in the projection area 200.

  The scanning unit 40 scans (scans) the line light 100 in the first direction D1 in the projection region 200. The scanning unit 40 is configured by, for example, a MEMS mirror. The MEMS mirror is a micro-reflecting mirror that vibrates at a constant rotation period. The MEMS mirror reflects the one-dimensional line light 100 while vibrating at a predetermined amplitude angle and a predetermined vibration frequency. As a result, the one-dimensional line light 100 is scanned in the projection region 200 at the vibration period of the MEMS mirror (vibration period = 1 / vibration frequency). The first direction D1, which is the scanning direction, is a direction orthogonal to the second direction D2, as shown in FIGS. The vibration direction of the MEMS mirror is set so that the scanning direction is the first direction D1. The length in the first direction D1 in the projection region 200 is determined by the amplitude angle of the MEMS mirror and the distance from the projection unit 10 (that is, the scanning unit 40) to the DUT 2.

  As shown in FIG. 2, the light intensity of the line light 100 from the light generation unit 20 changes in a sine wave shape in accordance with scanning. Therefore, when the scanning unit 40 scans the line light 100 in the first direction D1, a fringe pattern having a sinusoidal periodic light intensity distribution appears in the projection region 200 along the first direction D1. . This stripe pattern is simply referred to as a “pattern”. In the phase shift method, such a fringe pattern is used for measuring a three-dimensional shape. The fringe pattern has a light-dark pattern with a bright part (white part in FIG. 2) and a dark part (black part in FIG. 2). Further, since the stripe pattern is a vertical stripe pattern, it is also expressed as a vertical stripe pattern. The first direction D1 is also referred to as a light / dark direction or a light / dark direction. The stripe pattern shown in FIG. 2 has a predetermined length in the second direction D2, and is scanned in the first direction D1 over a predetermined length, so that the rectangular projection region 200 is in space. Formed.

  In the projection area 200, the phase in each part of the fringe pattern is shifted three times by π / 2 every predetermined time. The projection unit 10 shifts the phase of the fringe pattern by changing the timing for synchronizing the cycle of the sine wave of the line light 100 and the vibration of the scanning unit 40 based on the command signal from the arithmetic processing unit 60. Here, for example, when the angular velocity of vibration of the scanning unit 40 changes, the synchronization includes adjusting the phase of the sine wave of the line light 100 to the change in the angular velocity.

  The imaging unit 50 is disposed at a position different from the position of the projection unit 10. The imaging unit 50 images the device under test 2 onto which the line light 100 is projected from a direction different from the projection direction by the projection unit 10. The imaging area captured at a time is within the area of the projection area 200 and is narrower than the projection area 200. However, it is sufficient that the imaging area does not protrude at least outside the projection area 200. For example, the imaging area may be the same area as the projection area 200. The imaging area is an area used for processing described later, and the area actually captured may be larger than the projection area 200.

  The imaging unit 50 includes a light receiving optical system (photographing lens) 51 and an imaging device 52. The light receiving optical system 51 receives the fringe pattern reflected from the surface of the DUT 2 on the imaging region, and guides the received fringe pattern to the imaging device 52. The imaging device 52 generates image data of the DUT 2 based on the fringe pattern from the light receiving optical system 51 and stores the generated image data. As described above, the phase of the fringe pattern in the projection region 200 is shifted three times by π / 2 every predetermined time. The imaging device 52 images the device under test 2 and generates image data at timings when the initial phase of the fringe pattern is 0, π / 2, π, and 3π / 2. Here, the initial phase means the left end phase of the fringe pattern scanned from left to right.

  The arithmetic processing unit 60 controls the generation of the line light 100 by the light generation unit 20. Further, the arithmetic processing unit 60 controls the light generation unit 20 and the scanning unit 40 so as to synchronize the cycle of the sine wave of the line light 100 generated by the light generation unit 20 and the vibration cycle of the scanning unit 40. . Further, the arithmetic processing unit 60 controls the imaging timing of the imaging unit 50 so that the imaging unit 50 images the device under test 2 at the timing when the initial phase of the fringe pattern is 0, π / 2, π, 3π / 2. To do. In addition, the arithmetic processing unit 60 outputs luminance data (signals) of each pixel in the four image data captured by the imaging unit 50 (image data when the initial phase of the fringe pattern is 0, π / 2, π, 3π / 2). Based on (strength), the three-dimensional shape of the DUT 2 is calculated.

  Next, detailed configurations of the projection unit 10, the imaging unit 50, and the arithmetic processing unit 60 included in the shape measuring apparatus 1 will be described with reference to FIGS. 3 and 4. FIG. 3 is a block diagram showing a configuration of the shape measuring apparatus 1 shown in FIG. FIG. 4 is a block diagram showing a configuration of the calculation unit 65 shown in FIG. As illustrated in FIG. 3, the projection unit 10 includes a laser controller 21, a laser diode (light source) 22, a line generation unit 23, a projection optical system 30, and a scanning unit 40. That is, the light generation unit 20 illustrated in FIG. 1 includes a laser controller 21, a laser diode 22, and a line generation unit 23.

  The laser controller 21 controls irradiation of the laser light by the laser diode 22 based on a command signal from the control unit 62. The laser diode 22 is a light source that irradiates the line generation unit 23 with laser light based on a control signal from the laser controller 21. The laser diode 22 irradiates a laser beam whose light intensity changes in a sine wave shape with the passage of time when a voltage signal corresponding to the movement of the scanning unit 40 is input. Further, the laser diode 22 can irradiate the laser beam while changing the light intensity of the laser beam stepwise based on a control signal from the laser controller 21. The line generator 23 generates the one-dimensional line light 100 from the laser light irradiated by the laser diode 22.

  As described with reference to FIG. 1, the projection optical system 30 projects the line light 100 generated by the line generation unit 23. As described in FIG. 1, the scanning unit 40 scans the one-dimensional line light 100 generated by the line generation unit 23 along the scanning direction (the first direction D1 in the projection region 200). In FIG. 3, the first direction D1 is a direction perpendicular to the paper surface, and the second direction D2 is the left-right direction (lateral direction) in the paper surface.

  The imaging unit 50 includes a light receiving optical system 51, a CCD imaging device 52a (a camera using a charge coupled device), and an image memory 52b. That is, the imaging device 52 shown in FIG. 1 includes a CCD imaging device 52a and an image memory 52b. As described with reference to FIG. 1, the light receiving optical system 51 receives the fringe pattern reflected from the surface of the object to be measured 2 on the imaging region 210 and performs CCD imaging of the fringe pattern projected on the surface of the object to be measured 2. An image is formed on the light receiving surface of the element 52a.

  The CCD image sensor 52a photoelectrically converts the intensity of the image light on the light receiving surface into a charge amount corresponding to the intensity, and sequentially reads out the charge amount and converts it into an electrical signal. Thereby, image data of the DUT 2 on which the fringe pattern is projected is generated. The image data is composed of luminance data (luminance value) for each pixel. For example, the image data is 512 × 512 = 262144 pixels. In addition, one imaging range is a 23 cm square at the reference position. The CCD image sensor 52a captures the object to be measured 2 and generates image data at the timing when the fringe pattern whose initial phase is 0, π / 2, π, 3π / 2 is projected. The image memory 52b stores image data generated by the CCD image sensor 52a.

  The arithmetic processing unit 60 includes an operation unit 61, a control unit 62, a setting information storage unit 63, a capture memory 64, a calculation unit 65, an image storage unit 66, and a display control unit 67. The control unit 62, the calculation unit 65, and the display control unit 67 in the calculation processing unit 60 correspond to processing executed by a calculation processing device such as a CPU (Central Processing Unit) according to a control program (shape measurement program).

  The operation unit 61 outputs an operation signal corresponding to a user operation to the control unit 62. The operation unit 61 includes, for example, buttons and switches operated by the user, a touch panel on the display screen of the display device 70, and the like.

  The control unit 62 executes the following control according to the control program stored in the setting information storage unit 63. The control unit 62 causes the laser diode 22 to emit laser light by outputting a command signal to the laser controller 21. At this time, the control unit 62 instructs not only the start and end of laser light irradiation but also the light intensity (laser output) of the laser light in the command signal. The laser controller 21 controls the laser diode 22 so as to irradiate the laser beam having the light intensity commanded by the command signal from the control unit 62.

  Further, the control unit 62 outputs a command signal to the laser controller 21 and the scanning unit 40, so that the intensity change of the fringe pattern on which the light intensity in the laser diode 22 and the vibration of the scanning unit (MEMS mirror) 40 are projected is changed. The laser controller 21 and the scanning unit 40 are controlled so as to be a sine wave. In addition, when the period of the sine wave of the light intensity in the laser diode 22 and the vibration of the scanning unit 40 are not synchronized, the position of the stripe in the stripe pattern is shifted every time the scanning unit 40 reciprocates. Further, the control unit 62 outputs a command signal to the laser controller 21 and the scanning unit 40, so that the phase of the fringe pattern is sequentially shifted by π / 2 every predetermined time. 40 is controlled. Note that the left end phase when the scanning unit 40 scans from left to right is referred to as an initial phase.

  Further, the control unit 62 outputs a command signal to the scanning unit 40 and the CCD imaging device 52a, thereby synchronizing the imaging of the DUT 2 by the CCD imaging device 52a with a plurality of scanning of the fringe pattern by the scanning unit 40. Control to do. Specifically, the vibration frequency of the scanning unit 40 is 500 Hz (that is, the vibration period of the scanning unit 40 is 2 ms reciprocating), and the shutter speed of the CCD image sensor 52a (that is, the imaging time of the CCD image sensor 52a) is 40 ms. It is assumed that In this case, the scanning unit 40 scans the line light 100 for 20 reciprocations while the CCD imaging device 52a captures one image. As described above, the control unit 62 synchronizes the one-time imaging of the DUT 2 by the CCD imaging device 52 a with the 20 reciprocating scans of the line light 100 by the scanning unit 40. Further, the control unit 62 outputs a command signal to the CCD image sensor 52a to synchronize the imaging of the DUT 2 by the CCD image sensor 52a with the timing at which the phase of the fringe pattern is shifted.

  The setting information storage unit 63 stores a control program for causing the control unit 62 to execute control. In addition, the setting information storage unit 63 stores a control program for causing the calculation unit 65 to execute calculation processing of a three-dimensional shape. The setting information storage unit 63 also stores calibration information and the like used when calculating the actual coordinate value of the DUT 2 from the phase of the fringe pattern in the calculation process of the calculation unit 65.

  The capture memory 64 captures and stores the image data stored in the image memory 52b. The capture memory 64 is provided with storage areas corresponding to four image data when the initial phase of the fringe pattern is 0, π / 2, π, 3π / 2. For example, image data when the initial phase of the fringe pattern is 0 is stored in the image memory 52 b, and the image data is transferred to the first storage area of the capture memory 64. Further, the image data when the initial phase of the fringe pattern is π / 2 is stored in the image memory 52 b and the image data is transferred to the second storage area of the capture memory 64. Further, the image data when the initial phase of the fringe pattern is π is stored in the image memory 52 b and the image data is transferred to the third storage area of the capture memory 64. Further, the image data when the initial phase of the fringe pattern is 3π / 2 is stored in the image memory 52 b, and the image data is transferred to the fourth storage area of the capture memory 64.

  The computing unit 65 is configured to obtain the three-dimensional shape data (3) of the DUT 2 from the image data stored in the four storage areas of the acquisition memory 64 according to the control program and calibration information stored in the setting information storage unit 63. (Dimensional coordinate data) is calculated.

  The three-dimensional shape data of the DUT 2 calculated by the calculation unit 65 when the imaging unit 50 integrated with the projection unit 10 is shaken during imaging of the DUT 2 on which the fringe patterns having different initial phases are projected. The accuracy of is reduced. Therefore, in the aspect of the present invention, in order to obtain a highly accurate three-dimensional shape even when blurring occurs, the calculation unit 65 calculates the three-dimensional shape data of the DUT 2 by the following process. First, in the partial area of the DUT 2 in each striped pattern image, the calculation unit 65 extracts the same or different neighboring pixels for each striped pattern image and combines four pixels (the initial phase of the striped pattern). For all combinations, the pixel combinations when 0 is π / 2, π, and 3π / 2). And the calculating part 65 calculates three-dimensional shape data based on brightness | luminance data for every combination about all the acquired combinations. The arithmetic unit 65 compares the calculated three-dimensional shape data and selects the combination with the smallest measurement error. Then, the calculation unit 65 calculates the three-dimensional shape data of the DUT 2 based on the selected combination. As a processing unit that performs such processing, as shown in FIG. 4, the calculation unit 65 includes a first calculation unit 65a, a second calculation unit 65b, a selection unit 65c, and a third calculation unit 65d. Yes.

  The first calculation unit 65a is configured to detect luminance data of a predetermined pixel in the image data when the initial phase of the fringe pattern is 0 and the vicinity of the predetermined pixel in the image data when the initial phase of the fringe pattern is π / 2. The luminance data of the pixel (hereinafter referred to as pixel m-1) and the luminance of the pixel adjacent to pixel m-1 (hereinafter referred to as pixel m-2) in the image data when the initial phase of the fringe pattern is π. Based on a combination of the data and luminance data of a pixel adjacent to pixel m-2 (hereinafter referred to as pixel m-3) in the image data when the initial phase of the fringe pattern is 3π / 2, The three-dimensional coordinates of the corresponding device under test 2 are calculated. The calculation of the three-dimensional coordinates is performed for, for example, 10 × 10 pixels centered on a predetermined pixel, and a part of the three-dimensional shape data (dataM) of the DUT 2 is calculated.

  The second calculation unit 65b is configured to detect luminance data of a predetermined pixel in the image data when the initial phase of the fringe pattern is 0 and a neighborhood of the predetermined pixel in the image data when the initial phase of the fringe pattern is π / 2. Luminance data of a pixel that is different from the pixel m-1 (hereinafter referred to as pixel n-1), and a pixel adjacent to the pixel n-1 in the image data when the initial phase of the fringe pattern is π ( Hereinafter, the luminance data of the pixel n-2) and the luminance of the pixel adjacent to the pixel n-2 in the image data when the initial phase of the fringe pattern is 3π / 2 (hereinafter referred to as the pixel n-3). Based on the combination with the data, the three-dimensional coordinates of the DUT 2 corresponding to the predetermined pixel are calculated. The calculation of the three-dimensional coordinates is performed for, for example, 10 × 10 pixels centering on a predetermined pixel, and a part of the three-dimensional shape data (dataN) of the DUT 2 is calculated.

  The selection unit 65c includes a part of the three-dimensional shape of the DUT 2 obtained from the three-dimensional shape data (dataM) calculated by the first calculation unit 65a and the three-dimensional shape data calculated by the second calculation unit 65b. The three-dimensional shape of a part of the object 2 to be measured obtained from (dataN) is compared, and the three-dimensional shape data (dataM or dataN) used for the calculation of smoothly connected three-dimensional shapes is selected. Although the description has been made here by comparing two pieces of three-dimensional shape data, in practice, a three-dimensional shape is obtained by combining all the three-dimensional shape data, and the three-dimensional shape data used for the calculation of the three-dimensional shape connected most smoothly is selected. The third calculation unit 65d obtains the three-dimensional shape data of the DUT 2 based on the combination of pixels used in the three-dimensional shape data selected by the selection unit 65c. The details of the processing for the calculation unit 65 to obtain the three-dimensional shape data of the DUT 2 will be described later (see FIGS. 11 and 16).

  The image storage unit 66 stores the three-dimensional shape data of the DUT 2 calculated by the calculation unit 65. The display control unit 67 reads the three-dimensional shape data stored in the image storage unit 66 in accordance with the operation of the operation unit 61 by the user or automatically. Then, the display control unit 67 performs control to display a three-dimensional pseudo image of the DUT 2 on the display screen of the display device 70 based on the read three-dimensional shape data.

  The display device 70 is a device that displays a three-dimensional pseudo image of the DUT 2. The display device 70 is composed of, for example, a liquid crystal display. In FIG. 1, the display device 70 is not included in the shape measuring device 1, but may be included in the shape measuring device 1.

In the present embodiment, the shape measuring device 1 is configured as a small portable device, for example. In the case of such an apparatus, the projection unit 10, the imaging unit 50, the arithmetic processing unit 60, and the display device 70 are accommodated in a portable casing. According to such a configuration, the measurer can easily carry the shape measuring apparatus 1 to the site where the object to be measured 2 is located. Further, for example, the shape can be easily measured with respect to the object 2 to be measured which is difficult to measure with a stationary shape measuring apparatus such as a back surface or a back surface of a large-sized device. On the other hand, camera shake tends to occur when the measurer performs measurement using the shape measuring apparatus 1.

  Next, the principle of the phase shift method will be described with reference to FIG. The phase shift method is a method of measuring a distance using the principle of triangulation. The phase shift method has the same measurement principle as the light section method. In the light cutting method, a laser projects a line-shaped laser beam onto an object. Further, the imaging unit images the line light reflected from the object surface. Then, the arithmetic unit restores the three-dimensional shape of the object based on the distance between the imaging unit 50 and the scanning unit 40, the projection direction of the slit light, and the imaging direction. In the case of the light cutting method, only one line can be measured at a time. Therefore, in order to obtain a measurement value of the entire screen of the imaging unit, it is necessary to perform the projection of the laser beam by the laser and the imaging by the imaging unit as many times as necessary while shifting the laser beam according to the required resolution.

  On the other hand, in the phase shift method, the distance is measured by analyzing the fringe image picked up by shifting the phase of the sinusoidal fringe pattern. At this time, the fringe pattern projected from the projection unit becomes four types of images with the phase shifted by π / 2 as described above.

  Each time the initial phase of the fringe pattern shifts to 0, π / 2, π, 3π / 2, the shading of the fringe is projected with a shift corresponding to the phase difference. When the DUT 2 is located within the imaging region 210, a stripe pattern appears on the surface of the DUT 2. The imaging unit 50 (that is, the CCD imaging device 52a) images the DUT 2 on which the fringe pattern appears on the surface at timings when the initial phase of the fringe pattern is 0, π / 2, π, and 3π / 2. Thereby, four images when the initial phase of the fringe pattern is 0, π / 2, π, 3π / 2 are obtained. These images are called “stripe pattern images”.

The luminance value I _n (x, y) (n = 0, 1, 2, 3) is a luminance value of a predetermined pixel (x, y) of each image captured when the fringe pattern of each phase is projected. . That is, I ₀ is the luminance value of the image captured when the fringe pattern with the initial phase 0 is projected. I ₁ is a luminance value of an image captured when a fringe pattern having an initial phase π / 2 is projected. I ₂ is the luminance value of the image captured when the fringe pattern of the initial phase π is projected. I ₃ is a luminance value of an image captured when a fringe pattern having an initial phase of 3π / 2 is projected. The luminance value I _n (x, y) (n = 0, 1, 2, 3) is expressed by the following equation (1).

I _n (x, y) = A (x, y) cos (φ (x, y) + nπ / 2) + B (x, y) (1)

FIG. 5 is a waveform diagram showing a sine wave obtained from the luminance value of a predetermined pixel. As shown in FIG. 5, in equation (1), B (x, y) represents a bias component. A (x, y) indicates the contrast strength of the sine wave at the time of imaging. Φ (x, y) is a phase of a sine wave at a predetermined pixel (x, y). As shown in FIG. 5, the absolute values of the luminance values I _{0 to} I ₃ at the same pixel (same position) on the four images vary depending on the surface property and color of the object. However, the relative luminance value difference always shows a change by the phase difference of the fringe pattern. Therefore, the phase φ (x, y) of the stripe pattern in the predetermined pixel (x, y) is obtained from the luminance value at the same pixel of the four images by the following equation (2).

φ (x, y) = tan ⁻¹ {(I ₃ (x, y) −I ₁ (x, y)) / (I ₀ (x, y) −I ₂ (x, y))}... (2)

  In this way, the phase at the initial phase 0 of the sine wave can be obtained for each pixel of the image. A line (equal phase line) obtained by connecting points having the same phase φ (x, y) represents the shape of a cross section obtained by cutting an object at a certain plane in the same manner as the cutting line in the optical cutting method. Therefore, a three-dimensional shape (height information at each point of the image) is obtained by the principle of triangulation based on this phase φ (x, y).

  Next, a spatial code pattern and other patterns used in the spatial code method will be described. In the phase connection in the above-described phase shift method, when the surface shape of the DUT 2 is a continuous surface shape that smoothly changes, the phase of 2π corresponding to one stripe is −π to π to 3π to It is possible to simply connect 5π. However, when the surface shape of the DUT 2 is a discontinuous surface shape with a sudden step change, it is impossible to know which fringe phase the phase is. For example, it is not clear whether the phase of a certain stripe is a phase of −π to π or a phase of π to 3π. In this case, a so-called phase skip phenomenon occurs in which phases are not continuously connected. In order to prevent such a phase jump phenomenon, in this embodiment, the three-dimensional shape of the DUT 2 is measured by combining the phase shift method and the spatial code method. Therefore, in the present embodiment, the projection unit 10 projects the fringe pattern used in the phase shift method and the spatial code pattern used in the spatial code method in the projection region 200. This spatial code pattern is also simply referred to as a “pattern”.

  FIG. 6 is a diagram showing the intensity distribution of the spatial code pattern in the projection area 200. As shown in FIG. A plurality of patterns are used as the spatial code pattern, and one of them is shown in FIG. As shown in FIG. 6, the spatial code pattern has a rectangular wave shape in the light intensity profile along the first direction D <b> 1 in the projection region 200. In this spatial code pattern, bright portions (white portions in FIG. 6) and dark portions (black portions in FIG. 6) appear alternately. That is, in the spatial code pattern shown in FIG. 6, eight white lines in the second direction D2 and eight black lines in the second direction D2 are alternately arranged.

  As shown in FIG. 1, the one-dimensional line light 100 is generated in the second direction. Here, the laser diode 22 outputs a laser beam whose light intensity changes in a rectangular wave shape with the passage of time when a voltage signal changing in a rectangular wave shape is input. Therefore, the line light 100 in the second direction generated by the line generation unit 23 becomes light having a periodic light intensity distribution in the form of a rectangular wave along the first direction.

  The line light 100 generated by the line generation unit 23 passes through the projection optical system 30 and is then guided to a MEMS mirror as the scanning unit 40. The MEMS mirror vibrates at a predetermined amplitude angle and vibration frequency in a direction in which the line light 100 is scanned in the first direction D1 in the projection region 200. The MEMS mirror reflects the line light 100, whereby the line light 100 is projected in the second direction D2 in the projection region 200 and scanned in the first direction D1. As a result, a spatial code pattern of black and white stripes is projected over the entire projection area 200.

  FIG. 7 is a diagram showing a standard pattern, four spatial code patterns, and a monochrome reference pattern. FIG. 7A shows a standard pattern. FIG. 7B shows a spatial code pattern. FIG. 7C shows a black and white reference pattern. When the display control unit 67 displays a three-dimensional pseudo image of the DUT 2 on the display screen of the display device 70, a process (texture) for attaching a color to the surface of the three-dimensional shape is performed. The standard pattern shown in FIG. 7A is imaged to acquire a standard color that is pasted on the surface of the three-dimensional shape. Note that an image of the DUT 2 when the standard pattern is projected is referred to as a “standard image”.

  The spatial code pattern shown in FIG. 7B is a pattern for assigning numbers called spatial codes to a plurality of areas in the projection area 200. In the example shown in FIG. 7B, the left half of the spatial code pattern of (a) is white and the right half is black. In the spatial code pattern (b), two white lines and two black lines are alternately arranged. In the spatial code pattern (c), four white lines and four black lines are alternately arranged. In the spatial code pattern (d), eight white lines and eight black lines are alternately arranged. Note that the image of the DUT 2 when the spatial code pattern is projected is referred to as a “spatial code image”.

  The monochrome reference pattern shown in FIG. 7C is a pattern that is referenced when discriminating between “1” and “0” of the spatial code assigned to each area in the projection area 200, that is, white and black. When the calculation unit 65 determines “1” of the space code, the white pattern of the monochrome reference pattern is referred to. Further, when determining “0” of the space code, the black pattern of the monochrome reference pattern is referred to. Note that an image of the DUT 2 when the black and white reference pattern is projected is referred to as a “black and white reference image”. An image of the DUT 2 when a white pattern is projected is referred to as a “white image”. An image of the DUT 2 when the black pattern is projected is referred to as a “black image”. In the present embodiment, the standard pattern shown in FIG. 7A and the white pattern of the monochrome reference pattern shown in FIG. 7C are the same pattern. Here, black generally indicates a lightness of 0, but here, the minimum brightness that can be identified is referred to as black. Here, the minimum brightness that can identify black is to identify a portion irradiated with black and a non-irradiated portion.

  FIG. 8 is a diagram showing the relationship between the space code and the region number. In the “space code” shown in FIG. 8, “0” corresponds to black and “1” corresponds to white. The first level of the “space code” corresponds to the space code pattern (a) in FIG. The second level of the “space code” corresponds to the space code pattern (b) in FIG. The third level of the “space code” corresponds to the space code pattern (c) in FIG. The fourth level of the “space code” corresponds to the space code pattern (d) in FIG. The “area number” shown in FIG. 8 is a number assigned to the 16-divided area identified by the spatial code. For example, “0” of “area number” is “0000” from the top. That is, the black pattern is projected on any of the four spatial code patterns. Further, “10” of “area number” is “1010” from the top. The 16 divisions in the projection area 200 are identified by the arithmetic unit 65 based on such numbers.

  Next, the operation of the shape measuring apparatus 1 according to the first embodiment will be described.

  FIG. 9 is a flowchart for explaining an example of the shape measuring method. As shown in FIG. 9, the control unit 62 outputs a command signal to the laser controller 21 so as to turn on the laser diode 22. The laser controller 21 turns on the laser diode 22 based on the command signal from the control unit 62 (step S1). Moreover, the control part 62 starts the scanning by the scanning part 40 by outputting a command signal to the scanning part 40 (step S2). And the control part 62 determines whether the shutter operation by the user was performed (step S3).

  When the shutter operation is performed by the user (step S3: YES), that is, when the control unit 62 inputs a signal indicating that the shutter operation has been performed from the operation unit 61, the control unit 62 images the standard pattern ( Step S4). In this process, the laser diode 22 outputs laser light having unmodulated light intensity (high light intensity constant) shown in FIG. The line generation unit 23 generates the one-dimensional line light 100 from the laser light output from the laser diode 22. Then, when the scanning unit 40 scans the one-dimensional line light 100, a standard pattern as shown in FIG. The CCD image sensor 52a captures the standard pattern shown in FIG. 7A and generates image data of a standard image. The image data of the standard image is temporarily stored in the image memory 52 b and then stored in a storage area (standard image area) provided in the capture memory 64.

  Next, the control unit 62 captures an image of the spatial code pattern that is divided into two in the scanning direction (first direction) shown in FIG. 7B (step S5). In this process, the laser diode 22 has a rectangular wave shape with light intensity so that the four spatial code patterns shown in FIGS. 7A to 7D are projected by the projection unit 10 at predetermined time intervals. The cycle is switched every predetermined time.

  Specifically, the laser diode 22 has a rectangular wave-shaped laser beam (that is, the first stage “spatial code” in FIG. 8) such that the spatial code pattern shown in FIG. 1111111100000000 "pulsed laser light) is output. In addition, after a predetermined time has elapsed, the laser diode 22 has a rectangular wave-shaped laser beam (that is, two stages of “spatial code” in FIG. 8) such that the spatial code pattern shown in (b) of FIG. A pulsed laser beam representing the eye "1111000011110000"). Further, after a predetermined time has elapsed, the laser diode 22 has a rectangular wave-shaped period of laser light (that is, three stages of “spatial code” in FIG. 8) such that the spatial code pattern shown in FIG. The pulsed laser beam representing the eye “1100110011001100” is output. Further, after a lapse of a predetermined time, the laser diode 22 is a laser beam having a rectangular wave-like period so as to form the spatial code pattern shown in FIG. 7B (d) (that is, one stage of the “spatial code” in FIG. The pulsed laser beam representing the eyes “10101010101010110” is output. It should be noted that the switching of the spatial code pattern may be performed by detecting the completion of photographing, which will be described later, instead of elapse of a predetermined time.

  Then, the line generation unit 23 generates the one-dimensional line light 100 from the laser light output from the laser diode 22. Thereafter, the scanning unit 40 scans the one-dimensional line light 100, whereby each spatial code pattern as shown in (a) to (d) of FIG. 7B is projected onto the projection region 200 at predetermined time intervals. The The CCD image sensor 52a captures the spatial code patterns shown in (a) to (d) of FIG. 7B to generate image data of a spatial code image. The image data of the spatial code image is once stored in the image memory 52b, and then each storage area (first spatial code area, second spatial code area, third spatial code area) provided in the capture memory 64, respectively. , The fourth spatial code area).

  Next, the control unit 62 images the monochrome reference pattern shown in FIG. 7C (step S6). In this process, the laser diode 22 outputs laser light having unmodulated light intensity (constant light intensity at a high level) shown in FIG. The line generation unit 23 generates the one-dimensional line light 100 from the laser light output from the laser diode 22. Then, when the scanning unit 40 scans the one-dimensional line light 100, a white pattern of a monochrome reference pattern as shown in FIG. Further, the laser diode 22 outputs laser light having unmodulated light intensity (constant light intensity at a low level) shown in FIG. The line generation unit 23 generates the one-dimensional line light 100 from the laser light output from the laser diode 22. Then, when the scanning unit 40 scans the one-dimensional line light 100, the black pattern of the monochrome reference pattern as shown in FIG. The CCD image sensor 52a captures the monochrome reference pattern shown in FIG. 7C and generates image data of the monochrome reference image. The image data of the black and white reference image is temporarily stored in the image memory 52b and then stored in order in each storage area (white image area and black image area) provided in the capture memory 64.

  Next, the control unit 62 images four fringe patterns whose phases are shifted by π / 2 (step S7). Based on the command signal from the control unit 62, the CCD image pickup device 52a images the object to be measured 2 at the timing when the initial phase of the fringe pattern is 0, π / 2, π, and 3π / 2, and four fringes are obtained. Image data of a pattern image is generated. The image data of the fringe pattern image of each phase captured by the CCD image sensor 52a is temporarily stored in the image memory 52b, and then each storage area (first storage area, second storage area) provided in the capture memory 64, respectively. Storage area, third storage area, and fourth storage area).

  Next, the calculation unit 65 performs a process of obtaining a pixel combination for each fringe pattern image that minimizes the measurement error of the three-dimensional shape due to the shake (steps S8 to S10). During the period in which the imaging unit 50 is imaging four stripe patterns, the shape measuring apparatus 1 may be shaken. In particular, when the shape measuring device 1 is a small portable device, there is a high possibility that the shake of the measurer will occur. In this case, a measurement error occurs in the three-dimensional shape data calculated by the calculation unit 65. That is, a measurement error occurs due to a difference in the portion of the DUT 2 corresponding to the same pixel of the CCD image pickup element 52a when the fringe pattern image of each phase is acquired.

  FIG. 10 is a waveform diagram showing the period of undulation caused by shaking. In FIG. 10, the dotted waveform indicates a sinusoidal periodic light intensity distribution in the fringe pattern. In FIG. 10, the solid line waveform indicates a planar measurement error that occurs on the three-dimensional shape data of the DUT 2 calculated by the calculation unit 65. This plane measurement error is a measurement error generated in the plane portion of the DUT 2 and appears as a waviness having a period twice as long as the period of the sinusoidal light intensity in the stripe pattern as shown in FIG. When such a plane measurement error occurs, the flatness of the plane portion of the DUT 2 in the three-dimensional shape data deteriorates.

  For example, when the amount of movement (movement amount) and the direction of the shake are known based on the feature points of the device under test 2 (for example, corners or protrusions on the device under test 2), The image data of the fringe pattern image can be corrected based on the blur direction. However, when there is no feature point in the DUT 2, the calculation unit 65 does not know the amount of blur and the direction of blur. Therefore, in the present embodiment, the calculation unit 65 obtains a pixel combination for each stripe pattern image corresponding to the amount of blur and the direction of blur, and based on the luminance value of the pixel of the combination, the three-dimensional of the DUT 2 Calculate shape data. Hereinafter, a specific example of processing for obtaining a pixel combination for each stripe pattern image according to the amount of blur and the direction of blur will be described.

  FIG. 11 is a diagram illustrating combinations of neighboring pixels in the image data of each stripe pattern image. In FIGS. 11A to 11D, one square corresponds to one pixel. Accordingly, in FIGS. 11A to 11D, 3 pixels × 3 pixels (= 9 pixels) are shown. FIG. 11A shows nine pixels in a partial area provided at a predetermined location (for example, the central portion) in the image data in which the initial phase of the fringe pattern is 0 degrees. The middle pixel of the nine pixels is set to pixel “0”. FIG. 11B shows nine pixels (pixel “1” to pixel “9”) in the image data in which the initial phase of the stripe pattern is 90 degrees (π / 2). The middle pixel “5” of the nine pixels (pixel “1” to pixel “9”) is a pixel corresponding to the pixel “0” in FIG. Pixels “1”, “2”, “3”, “4”, “6”, “7”, “8”, and “9” are neighboring pixels of pixel “5”. Here, the neighboring pixels are pixels adjacent to each other.

  FIG. 11C shows nine pixels (pixel “11” to pixel “19”) in the image data in which the initial phase of the fringe pattern is 180 degrees (π). In the example shown in FIGS. 11B and 11C, the middle pixel “15” of 9 pixels (pixel “11” to pixel “19”) is a pixel corresponding to the pixel “6” in FIG. is there. Pixels “11”, “12”, “13”, “14”, “16”, “17”, “18”, and “19” are neighboring pixels of the pixel “15”. FIG. 11D shows nine pixels (pixel “21” to pixel “29”) in the image data in which the initial phase of the stripe pattern is 270 degrees (3π / 2). In the example shown in FIGS. 11C and 11D, the middle pixel “25” of nine pixels (pixel “21” to pixel “29”) is a pixel corresponding to the pixel “13” in FIG. is there. Pixels “21”, “22”, “23”, “24”, “26”, “27”, “28”, and “29” are neighboring pixels of pixel “25”.

  First, the computing unit 65 extracts the middle pixel “0” of the nine pixels in the image data shown in FIG. Next, the calculation unit 65 combines the extracted pixel “0” with a pixel “5” corresponding to the pixel “0” in the image data shown in FIG. The neighboring pixels “1”, “2”, “3”, “4”, “6”, “7”, “8”, and “9” of “5” are extracted. Next, the calculation unit 65 is a pixel in the image data shown in FIG. 11C as a pixel combined with each of the extracted 9 pixels “1” to “9”, and each of the 9 pixels “1” to “9”. 9 pixels corresponding to, and neighboring pixels of these 9 pixels are extracted. Further, the arithmetic unit 65 combines a plurality of pixels corresponding to each of the plurality of pixels in the image data shown in FIG. 11D as pixels to be combined with each of the plurality of extracted pixels, and each of the plurality of pixels. And neighboring pixels.

  By such processing, the arithmetic unit 65 extracts 729 (9 × 9 × 9) pixel combinations as combinations of neighboring pixels in the image data of each stripe pattern image. In the example illustrated in FIG. 11, a combination of the pixel “0”, the pixel “6”, the pixel “13”, and the pixel “27” is extracted. The arithmetic unit 65 performs the process of extracting the pixel combinations for each image data as described above for all the pixels in a partial area in the image data in which the initial phase of the fringe pattern is 0 degrees.

  Then, the computing unit 65 calculates the phase φ (x, y) for each combination based on the luminance values (luminance data) of the combination pixels with respect to all the pixels in the partial area (step S8). In addition, the calculation unit 65 calculates, for each combination, three-dimensional shape data (three-dimensional coordinate data) of a partial region from the calculated phase φ (x, y) for each combination (step S9). Further, the calculation unit 65 selects a combination of pixels having the smallest plane measurement error in the calculated three-dimensional shape data of a partial region (step S10). Note that a plurality of pixels are required to form the surface. Therefore, for example, assuming that the same blur occurs for 10 × 10 pixels, the plane can be obtained by applying the same combination to each pixel.

  FIG. 12 is a diagram illustrating a relationship between a combination of neighboring pixels and a plane measurement error. In FIG. 12, the horizontal axis indicates a combination of pixels, and the vertical axis indicates a plane measurement error. In addition, double circles (）) in FIG. 12 indicate the values of the plane measurement error in each combination. As described above, when blurring occurs during the imaging of the fringe pattern image, a plane measurement error occurs on the three-dimensional shape data of a partial region calculated by the calculation unit 65. On the other hand, as shown in FIG. 12, the value of the plane measurement error changes depending on the combination of pixels. Therefore, the calculation unit 65 selects a combination of pixels having the smallest plane measurement error value. That is, the calculation unit 65 selects a combination of pixels having the highest flatness in the planar portion of the DUT 2 in the three-dimensional shape data.

  Specifically, the calculation unit 65 compares the flatness of the three-dimensional shape data for each combination in a partial region of the DUT 2. At this time, the calculation unit 65 evaluates the flatness with the frequency component of the unevenness of the three-dimensional shape data. As described above, the plane measurement error appears as a waviness having a period twice the period of the sinusoidal light intensity in the fringe pattern. Accordingly, the calculation unit 65 selects a pixel combination having the least number of high-frequency components that are twice the period of the sinusoidal light intensity in the fringe pattern among the frequency components of the unevenness of the three-dimensional shape data. Since the swell is twice the period of the sinusoidal light intensity in the fringe pattern, a partial area of the DUT 2 is an area having a width that allows at least one set of bright and dark stripes in the fringe pattern to enter. It is said.

  For example, the calculation unit 65 may use a combination of pixels as illustrated in FIG. 11 (pixel “0”, pixel “6”, pixel “13”, and pixel “27”) as a combination of pixels having the smallest plane measurement error value. Select. In this case, during the period from when the imaging unit 50 captures an image with an initial phase of 0 degrees to when an image with an initial phase of 90 degrees is captured, the shape measuring apparatus 1 moves in the direction from pixel “6” to pixel “5” ( It shows that it has moved to the left. In addition, during the period from when the imaging unit 50 captures an image with an initial phase of 90 degrees until it captures an image with an initial phase of 180 degrees, the shape measuring apparatus 1 moves in the direction from the pixel “13” to the pixel “15” (left It shows that the camera shakes in the diagonally downward direction. In addition, during the period from when the imaging unit 50 captures an image with an initial phase of 180 degrees to when an image with an initial phase of 270 degrees is captured, the shape measuring apparatus 1 moves in the direction from the pixel “27” to the pixel “25” (right It shows that the camera shakes in the diagonally upward direction.

  Returning to the description of FIG. 9, the arithmetic unit 65 obtains the phase distribution φ (i, j) at the initial phase 0 of each pixel from the image data of the four stripe pattern images based on the combination of the pixels selected in step S <b> 10. . That is, the phase recovery process is executed (step S11). Specifically, when the combination of the selected pixels is a combination of the pixel “0”, the pixel “6”, the pixel “13”, and the pixel “27” illustrated in FIG. It is determined that the amount of blur is one pixel in the left direction, the left diagonally downward direction, and the right diagonally upward direction. Then, the calculation unit 65 obtains the phase distribution φ (i, j) at the initial phase 0 of each pixel after shifting the image data of each stripe pattern image in accordance with the determined blur direction and blur amount.

  Next, the arithmetic unit 65 recognizes the spatial code of the region divided into 16 in the projection region 200 based on the image data of the four spatial code images stored in the spatial code region of the capture memory 64. And the calculating part 65 detects the order of the fringe of a fringe pattern image based on the recognized space code of each area | region (step S12).

  The computing unit 65 performs a phase connection process (unwrapping process) using the order of each fringe specified in the process of step S12 (step S13). That is, the computing unit 65 specifies the position of the nth stripe on the image based on the stripe order. Then, the calculation unit 65 correctly connects the phase recovery values of the nth fringe obtained between −π and π. Thereby, a continuous initial phase distribution φ ′ (i, j) is obtained. Thereafter, the calculation unit 65 uses the above-described triangulation principle to calculate the coordinate data X (3) of the three-dimensional shape of the DUT 2 from the phase distribution φ ′ (i, j) at the initial phase 0 obtained in step S13. x, y, z) is calculated (step S14). The coordinate data X is obtained as coordinates based on the CCD image sensor 52.

  Thereafter, the three-dimensional shape calculation unit 65 stores the calculated coordinate data of the three-dimensional shape of the DUT 2 in the image storage unit 66. Then, the display control unit 67 reads the coordinate data of the three-dimensional shape stored in the image storage unit 66 according to the operation of the operation unit 61 by the user or automatically. The display control unit 67 displays the three-dimensional shape of the DUT 2 on the display screen of the display device 70 based on the read coordinate data of the three-dimensional shape. In the three-dimensional shape, a point group that is a set of points in a three-dimensional space is displayed as a pseudo three-dimensional image. The three-dimensional coordinates that are point group data can also be displayed in the form of (x, y, z), for example.

  The display device 70 not only displays the three-dimensional shape of the DUT 2, but also displays the image captured by the imaging unit 50. That is, the display control unit 67 causes the display device 70 to display an image captured by the imaging unit 50 based on the image data stored in the capture memory 64. According to such a configuration, the user can confirm whether or not the device under test 2 has been accurately imaged at the imaging site based on the image captured by the imaging unit 50.

  The display device 70 may be configured to display at least one of the image captured by the imaging unit 50 and the three-dimensional shape calculated by the three-dimensional shape calculation unit (calculation unit) 65. In this case, based on at least one of the image picked up by the image pickup unit 50 and the three-dimensional shape calculated by the three-dimensional shape calculation unit 65, the user can accurately pick up the object to be measured 2 at the image pickup site. It can be confirmed whether or not. Thereafter, the user connects the shape measuring apparatus 1 to a computer or the like, and captures an image or a three-dimensional shape into the computer or the like. Then, the user displays an image or a three-dimensional shape on a display unit such as a computer.

  In addition, about the failure part which the calculating part 65 failed in calculation of the point cloud data of a three-dimensional shape, and the occlusion part which became a shadow by the shape of the to-be-measured object 2, it displays in the aspect which a user can confirm. For example, the display control unit 67 blinks the failed part in red and blinks the occlusion part in blue. The three-dimensional shape point cloud data calculated by the calculation unit 65 may be stored in a non-volatile storage medium such as an SD card that can be carried by the user.

  In the first embodiment described above, the combination of neighboring pixels is a combination of neighboring pixels, but is not limited to such a combination, and may be a combination of neighboring pixels. For example, a predetermined pixel, a neighboring pixel that is two or more pixels away from the predetermined pixel (for example, two or three pixels), a neighboring pixel that is two or more pixels away from the neighboring pixel, and two or more pixels from the neighboring pixel A combination with a pixel in the vicinity is also possible. Note that the range of neighboring pixels is determined in advance based on the assumed amount of blur.

  As described above, in the first embodiment, the projection unit 10 that projects a plurality of different patterns 100 onto the DUT 2 and the imaging unit 50 that images each DUT 2 on which the plurality of patterns 100 are projected. And the first pixel (for example, pixel “0” in FIG. 11A) of the first image (for example, the image whose initial phase of the fringe pattern is 0) including the DUT 2 onto which the captured pattern 100 is projected. ) And the first pixel in the second image (for example, the image whose initial phase of the fringe pattern is π / 2) including the DUT 2 on which the pattern 100 different from the first image is projected. The first three-dimensional information is acquired by combining the information of the second pixel in the vicinity of the pixel corresponding to (for example, the pixel “6” in FIG. 11B), and the first three-dimensional information is acquired. A first calculation unit that repeatedly calculates a part of the shape of the DUT 2 5a, information on the first pixel of the first image, and a third pixel different from the second pixel in the vicinity of the pixel corresponding to the first pixel of the second image (for example, FIG. 11B) The second calculation unit that acquires the second three-dimensional information in combination with the information of the pixel “2”) and repeats the acquisition of the second three-dimensional information to calculate the shape of a part of the DUT 2. 65b and the shape of a part of the DUT 2 obtained from the first three-dimensional information and the shape of a part of the DUT 2 obtained from the second three-dimensional information are compared, and the shape is smoothly connected. A selection unit 65c that selects acquisition of the three-dimensional information used for the calculation, and a third calculation unit 65d that obtains the shape of the DUT 2 based on the combination of pixels used to acquire the selected three-dimensional information; including.

  According to such a configuration, the calculation unit 65 can calculate shape data (three-dimensional shape data) based on a combination of pixels corresponding to the amount and direction of blur, so that even when blur occurs, accuracy is increased. High shape data can be obtained. Further, even when there is no feature point in the DUT 2, the calculation unit 65 can obtain highly accurate shape data. In addition, since the calculation unit 65 selects a combination of pixels that is less affected by blurring in some areas, the calculation amount does not become enormous, and the time taken to obtain the calculation result (measurement result) of the shape data is shortened. The

  In the first embodiment, the pixel corresponding to the first pixel and the second pixel are adjacent to each other in the second image, and the pixel corresponding to the first pixel and the third pixel are It is an adjacent pixel. According to such a configuration, it is possible to avoid increasing the amount of calculation. In the first embodiment, the selection unit 65c evaluates the smoothness by the frequency component of the shape unevenness in the comparison of a part of the shape of the DUT 2, and acquires three-dimensional information with a small number of high-frequency components. select. According to such a configuration, the calculation unit 65 can reliably evaluate the magnitude of the measurement error of the shape data. In addition, since a smooth curved surface can be regarded as a plane when viewed locally, the calculation unit 65 can evaluate the magnitude of the plane measurement error based on the smoothness of the DUT 2.

Second Embodiment
In the first embodiment described above, the position of a part of the area is not specified. However, in the second embodiment, the calculation unit 65 replaces one or more flat parts of the DUT 2 with a part of the area. Choose as.

  FIG. 13 is a diagram illustrating a partial region for obtaining a combination of pixels with a small planar measurement error. As illustrated in FIG. 13, the calculation unit 65 selects one flat area 221 of the DUT 2 as a partial area. Alternatively, the calculation unit 65 selects a plurality of flat regions 221, 222, 223 of the DUT 2 as a partial region. As a method of selecting a flat region by the calculation unit 65, for example, the following method can be considered. That is, the calculation unit 65 calculates the three-dimensional shape data in which undulation has occurred based on the image data of the four stripe pattern images. The calculation unit 65 can recognize the rough three-dimensional shape of the DUT 2 even if undulation occurs on the calculated three-dimensional shape data. Accordingly, the calculation unit 65 recognizes a flat portion of the DUT 2 from the calculated three-dimensional shape data, and selects the recognized flat portion as a partial region.

  As described above, in the second embodiment, the calculation unit 65 (the first calculation unit 65 a and the second calculation unit 65 b) is a flat area of the DUT 2 as a partial area of the DUT 2. Select a part. According to such a configuration, the accuracy of measurement error evaluation is improved. Moreover, in 2nd Embodiment, the calculating part 65 (selection part 65c) performs the comparison of a one part shape and the selection of the one part shape connected smoothly about several parts of the image containing the to-be-measured object 2. FIG. According to such a configuration, it is possible to select a pixel combination that is less affected by blurring even for the DUT 2 having a plane or a curved surface inclined at various angles with respect to the shape measuring apparatus 1.

<Structure manufacturing system and structure manufacturing method>
FIG. 14 is a block diagram illustrating an example of an embodiment of a structure manufacturing system. The structure manufacturing system SYS illustrated in FIG. 14 includes the shape measuring device 1, the design device 710, the molding device 720, the control device (inspection device) 730, and the repair device 740.

  The design apparatus 710 creates design information related to the shape of the structure. Then, the design device 710 transmits the produced design information to the molding device 720 and the control device 730. Here, the design information is information indicating the coordinates of each position of the structure obtained from a plurality of two-dimensional design information, for example. The object to be measured is a structure.

  The forming device 720 forms a structure based on the design information transmitted from the design device 710. The molding process of the molding apparatus 720 includes casting, forging, cutting, or the like. The shape measuring device 1 measures the three-dimensional shape of the structure (measurement object 2) produced by the forming device 720, that is, the coordinates of the structure. Then, the shape measuring device 1 transmits information indicating the measured coordinates (hereinafter referred to as shape information) to the control device 730.

  The control device 730 includes a coordinate storage unit 731 and an inspection unit 732. The coordinate storage unit 731 stores design information transmitted from the design device 710. The inspection unit 732 reads design information from the coordinate storage unit 731. Further, the inspection unit 732 compares the design information read from the coordinate storage unit 731 with the shape information transmitted from the shape measuring device 1. And the test | inspection part 732 test | inspects whether the structure was shape | molded according to design information based on the comparison result.

  Further, the inspection unit 732 determines whether or not the structure molded by the molding device 720 is a non-defective product. Whether or not the structure is a non-defective product is determined based on, for example, whether or not the error between the design information and the shape information is within a predetermined threshold range. If the structure is not molded according to the design information, the inspection unit 732 determines whether the structure can be repaired according to the design information. If it is determined that it can be repaired, the inspection unit 732 calculates a defective portion and a repair amount based on the comparison result. Then, the inspection unit 732 transmits information indicating a defective portion (hereinafter referred to as defective portion information) and information indicating a repair amount (hereinafter referred to as repair amount information) to the repair device 740.

  The repair device 740 processes the defective portion of the structure based on the defective portion information and the repair amount information transmitted from the control device 730.

  FIG. 15 is a flowchart showing processing by the structure manufacturing system SYS, and shows an example of an embodiment of a structure manufacturing method. As shown in FIG. 15, the design device 710 creates design information related to the shape of the structure (step S31). The design device 710 transmits the produced design information to the molding device 720 and the control device 730. The control device 730 receives the design information transmitted from the design device 710. Then, the control device 730 stores the received design information in the coordinate storage unit 731.

  Next, the molding apparatus 720 molds the structure based on the design information created by the design apparatus 710 (step S32). Then, the shape measuring device 1 measures the three-dimensional shape of the structure formed by the forming device 720 (step S33). Thereafter, the shape measuring apparatus 1 transmits shape information that is a measurement result of the structure to the control device 730. Next, the inspection unit 732 compares the shape information transmitted from the shape measuring device 1 with the design information stored in the coordinate storage unit 731 to determine whether or not the structure has been molded according to the design information. Inspect (step S34).

  Next, the inspection unit 732 determines whether or not the structure is a good product (step S35). If it is determined that the structure is a non-defective product (step S35: YES), the process by the structure manufacturing system SYS is terminated. On the other hand, when the inspection unit 732 determines that the structure is not a non-defective product (step S35: NO), the inspection unit 732 determines whether the structure can be repaired (step S36).

  When the inspection unit 732 determines that the structure can be repaired (step S36: YES), the inspection unit 732 calculates the defective portion of the structure and the repair amount based on the comparison result of step S34. Then, the inspection unit 732 transmits the defective part information and the repair amount information to the repair device 740. The repair device 740 performs repair (rework) of the structure based on the defective part information and the repair amount information (step S37). Then, the process proceeds to step S33. That is, the process after step S33 is performed again with respect to the structure which the repair apparatus 740 performed repair. On the other hand, when the inspection unit 732 determines that the structure cannot be repaired (step S36: NO), the process by the structure manufacturing system SYS is terminated.

  As described above, in the structure manufacturing system SYS and the structure manufacturing method, based on the measurement result of the structure by the shape measuring apparatus 1, the inspection unit 732 determines whether the structure is manufactured according to the design information. . Accordingly, it can be accurately determined whether or not the structure manufactured by the molding apparatus 720 is a non-defective product, and the determination time can be shortened. Further, in the structure manufacturing system SYS described above, when the inspection unit 732 determines that the structure is not a non-defective product, the structure can be repaired immediately.

  In the structure manufacturing system SYS and the structure manufacturing method described above, the forming device 720 may be configured to execute the processing again instead of the repair device 740 executing the processing.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. Various modifications or improvements can be added to the above-described embodiment without departing from the spirit of the present invention. In addition, one or more of the requirements described in the above embodiments may be omitted. Such modifications, improvements, and omitted forms are also included in the technical scope of the present invention. In addition, the above-described embodiments can be applied in appropriate combination.

  In each of the above-described embodiments, the first direction D1 and the second direction D2 are orthogonal to each other. However, as long as the first direction D1 and the second direction D2 are different directions, they may not be orthogonal to each other. Good. For example, the second direction D2 may be set to an angle of 60 degrees or 80 degrees with respect to the first direction D1.

  In each of the above-described embodiments, one or a plurality of optical elements are shown in each drawing. Unless the number to be used is specified, an optical element to be used is used as long as the same optical performance is exhibited. The number of is arbitrary.

  Further, in each of the above-described embodiments, the scanning unit 40 uses an optical element that reflects or diffracts pattern light, but is not limited thereto. For example, a refractive optical element or parallel flat glass may be used. The pattern light may be scanned by vibrating a refractive optical element such as a lens with respect to the optical axis. As this refractive optical element, a part of the optical elements of the projection optical system 30 may be used.

  In each of the above-described embodiments, the CCD imaging device 52a is used as the imaging unit 50, but is not limited thereto. For example, an image sensor such as a CMOS image sensor (CMOS: Complementary Metal Oxide Semiconductor) may be used instead of the CCD image sensor.

  Further, in each of the above-described embodiments, the 4-bucket method is used in which the phase of the fringe pattern used in the phase shift method is shifted three times during one period, but is not limited thereto. For example, a 3-packet method in which a 0 · π / 2 · π stripe pattern is projected and then a 0 phase stripe pattern is projected again, or a 0 · π / 2 · π · 3π / 2 stripe pattern is projected again. A 5-bucket method for projecting a phase fringe pattern or a 6-bucket method for projecting a π / 2 phase fringe pattern may be used.

  In each of the embodiments described above, the phase shift method and the spatial code method are used in combination. However, the three-dimensional shape of the DUT 2 may be measured using only the phase shift method. Further, in the first embodiment described above, the order of imaging of the stripe pattern, imaging of the spatial code pattern, and imaging of other patterns does not matter.

  Moreover, in each above-mentioned embodiment, although the fringe pattern and the space code pattern were represented by white and black, it is not limited to this, Either one or both may be colored. For example, the stripe pattern and the spatial code pattern may be generated in white and red.

  In each of the above-described embodiments, the spatial code shown in FIG. 8 uses a binary code, but a gray code may be used. Gray code is different from binary code in the way of signing. For this reason, a different pattern is used as the stripe pattern of the spatial code pattern shown in FIG.

  In each of the above-described embodiments, the standard image is acquired. However, this standard image may not be acquired.

  Further, when a vibrating mirror such as a MEMS mirror is used as the scanning unit 40, the intensity of light from the laser diode 121 may be changed corresponding to the nonuniform angular velocity. For example, when the angular velocity decreases before the MEMS mirror rotates in the reverse direction, the laser diode 121 is configured so that the light intensity is increased near the end of the projection region 200 in the scanning direction and the light intensity is decreased near the center. May be controlled. Thereby, the nonuniformity of the brightness which arises in the edge part and center part of the projection area | region 200 can be suppressed.

  Moreover, you may implement | achieve the one part structure of the shape measuring apparatus 1 with a computer. For example, the calculation unit processing unit 60 may be realized by a computer. In this case, the computer generates a pattern light having a distribution of different intensities along the first direction D1 according to the shape measurement program stored in the storage unit, and outputs the pattern light to the first direction D1. Is a scanning process that scans along a second direction D2 on different objects to be measured 2, an imaging process that images the object to be measured 2 on which the pattern light is projected, and an object to be measured 2 obtained by the imaging process. And an arithmetic process for calculating the shape of the DUT 2 based on the signal intensity corresponding to the image. Note that the requirements of the above-described embodiments can be combined as appropriate. Some components may not be used. In addition, as long as it is permitted by law, the disclosure of all publications and US patents relating to the devices cited in the above embodiments and modifications are incorporated herein by reference.

  D1 ... first direction, D2 ... second direction, SYS ... structure manufacturing system, 1 ... shape measuring device, 2 ... object to be measured, 10 ... projection unit, 20 ... light generation unit, 40 ... scanning unit, 50 ... Imaging unit, 52a ... CCD image sensor, 60 ... Calculation processing unit, 62 ... Control unit, 65 ... Calculation unit, 65a ... First calculation unit, 65b ... Second calculation unit, 65c ... Selection unit, 65d ... First 3 computing units, 100 ... line light (pattern), 200 ... projection area

Claims (9)

Project multiple different patterns onto the measurement object,
Each of the objects to be measured on which the plurality of patterns are projected is imaged,
Information on the first pixel of the first image including the object to be measured on which the imaged pattern is projected, and the object to be measured on which the pattern different from the imaged first image is projected The first three-dimensional information is obtained by combining the information of the second pixel in the vicinity of the pixel corresponding to the first pixel in the second image including the first image, and acquiring the first three-dimensional information. Repeatedly calculating the shape of a part of the object to be measured,
Combining the information of the first pixel of the first image with the information of the third pixel different from the second pixel in the vicinity of the pixel corresponding to the first pixel of the second image. Acquiring second three-dimensional information, repeating the acquisition of the second three-dimensional information, calculating the shape of the part of the object to be measured,
The partial shape of the object to be measured obtained from the first three-dimensional information is compared with the partial shape of the object to be measured obtained from the second three-dimensional information, and the parts are smoothly connected. Select acquisition of 3D information used for shape calculation,
A shape measuring method for obtaining a shape of the object to be measured based on a combination of the pixels used for acquiring the selected three-dimensional information.   In the second image, the pixel corresponding to the first pixel and the second pixel are adjacent to each other, and the pixel corresponding to the first pixel and the third pixel are adjacent to each other. The shape measuring method according to claim 1, wherein the pixels are matched pixels.   The shape measuring method according to claim 1, wherein a flat portion of the device under test is selected as the part of the device under test.   The shape measuring method according to any one of claims 1 to 3, wherein the comparison and selection are performed on a plurality of portions of an image including the object to be measured.   5. The comparison of the partial shapes of the object to be measured, wherein the smoothness is evaluated by the frequency component of the shape unevenness, and acquisition of three-dimensional information with a small number of high frequency components is selected. The shape measuring method according to one item. A projection unit that projects a plurality of different patterns onto the object to be measured;
An imaging unit that images each object to be measured on which the plurality of patterns are projected;
Information on the first pixel of the first image including the object to be measured on which the imaged pattern is projected, and the object to be measured on which the pattern different from the imaged first image is projected The first three-dimensional information is obtained by combining the information of the second pixel in the vicinity of the pixel corresponding to the first pixel in the second image including the first image, and acquiring the first three-dimensional information. A first calculation unit that repeatedly calculates the shape of a part of the object to be measured;
Combining the information of the first pixel of the first image with the information of the third pixel different from the second pixel in the vicinity of the pixel corresponding to the first pixel of the second image. A second calculation unit that acquires second three-dimensional information, repeats the acquisition of the second three-dimensional information, and calculates the shape of the part of the object to be measured;
The partial shape of the object to be measured obtained from the first three-dimensional information is compared with the partial shape of the object to be measured obtained from the second three-dimensional information, and the parts are smoothly connected. A selection unit that selects acquisition of the three-dimensional information used in the calculation of the shape;
A shape measuring apparatus including: a third computing unit that obtains the shape of the object to be measured based on the combination of the pixels used for acquiring the selected three-dimensional information. A design device for creating design information on the shape of the structure;
A molding apparatus for producing the structure based on the design information;
The shape measuring apparatus according to claim 6, which measures the shape of the manufactured structure,
A structure manufacturing system comprising: an inspection device for comparing shape information relating to the shape of the structure obtained by the shape measuring device with the design information. Creating design information on the shape of the structure;
Producing the structure based on the design information;
Measuring the shape of the produced structure by the shape measuring method according to any one of claims 1 to 5,
Comparing shape information on the shape of the structure obtained by the shape measurement method with the design information. In a computer of a shape measuring apparatus comprising: a projection unit that projects a plurality of different patterns onto a measurement object; and an imaging unit that images each of the measurement objects on which the plurality of patterns are projected.
Information on the first pixel of the first image including the object to be measured on which the imaged pattern is projected, and the object to be measured on which the pattern different from the imaged first image is projected The first three-dimensional information is obtained by combining the information of the second pixel in the vicinity of the pixel corresponding to the first pixel in the second image including the first image, and acquiring the first three-dimensional information. A first calculation process for repeatedly calculating a shape of a part of the device under test;
Combining the information of the first pixel of the first image with the information of the third pixel different from the second pixel in the vicinity of the pixel corresponding to the first pixel of the second image. Second calculation processing for acquiring second three-dimensional information, repeating the acquisition of the second three-dimensional information, and calculating the partial shape of the object to be measured;
The partial shape of the object to be measured obtained from the first three-dimensional information is compared with the partial shape of the object to be measured obtained from the second three-dimensional information, and the parts are smoothly connected. A selection process for selecting acquisition of three-dimensional information used for shape calculation;
A shape measurement program for executing a third calculation process for obtaining the shape of the object to be measured based on the combination of the pixels used for acquiring the selected three-dimensional information.