JP2007101276A - Three-dimensional measuring projector and system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve efficiency in wide-area, noncontact three-dimensional measurement and facilitate its automation using a projector projecting a target pattern. <P>SOLUTION: The three-dimensional measuring projector 80 comprises a pattern formation section 492 for forming a measurement pattern P including a label with a color code CT having a position detection pattern P1 for indicating a measurement position and a color code pattern P3 which is arranged with predetermined positional relationship with respect to the position detection pattern P1 and includes a plurality of colors for identifying a label, a projection section 12 for projecting the measurement pattern formed by the pattern formation section 492, and a pattern detection section 491 for detecting the position detection pattern P1 and the color code pattern P3 through a photographed image of the measurement pattern projected by the projection section 12 and identifying the color code. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a three-dimensional measurement projector and a three-dimensional measurement system. More specifically, the present invention relates to a projection device that projects a target pattern for three-dimensional measurement and a three-dimensional measurement system that can automatically measure a wide area using a captured image that includes the projected pattern.

  Conventionally, when non-contact three-dimensional measurement is performed, measurement is performed for each small area with a relatively large device in which a pattern projector called a non-contact three-dimensional measurement device and a CCD camera are integrated, and the small size is further reduced. Each target pasted for each area was measured by a photogrammetry method, and these small areas were integrated from the coordinate points to perform a wide range of measurements.

In addition, when three-dimensional measurement is performed using only an image from a digital camera or the like, setting of a stereo pair, orientation of two or more images, and setting of a measurement position are performed manually or semi-automatically. (See Patent Documents 1 and 2)
JP 2003-284098 A (paragraphs 0018 to 0073, FIGS. 1 to 11) JP-A-2005-174151 (paragraphs 0021 to 0087, FIGS. 1 to 24)

  To measure a wide area, measure a large number of small areas with a large non-contact 3D measuring instrument, and then use a photogrammetry technique to shoot the image connection target attached to each small area with a camera. , Each target point is three-dimensionally measured with high accuracy, and the camera coordinate system and the target three-dimensional coordinate system (Earth coordinate system, etc.) in a small area measured by each three-dimensional measuring instrument are integrated to cover a wide whole area I was trying.

  However, according to this method, it is necessary to use separate measuring instruments for measuring a small area and a wide area, which is complicated, and there is a problem that automation through the entire three-dimensional measurement is not possible. . In particular, when integrating a large number of small areas over a wide range with high accuracy, each measurement range is narrowed, resulting in an enormous number of measurement areas, resulting in complicated and inefficient work. For example, the number of small areas is 100 cuts or more just by measuring the side of the car. As described above, there is a problem that even though the individual work is simple, the whole work is troublesome and the work is inefficient.

  An object of the present invention is to improve the efficiency and automation of wide-area non-contact three-dimensional measurement using a projection device that projects a target pattern.

  In order to solve the above-described problem, the three-dimensional measurement projector 80 according to claim 1 includes, for example, a position detection pattern P1 for indicating a measurement position and a position detection pattern P1 as shown in FIG. A pattern forming unit 492 that forms a measurement pattern P including a color-coded label CT having a color code pattern P3 arranged in a predetermined positional relationship and provided with a plurality of colors for identifying the label; The projection unit 12 that projects the measurement pattern formed by the forming unit 492 onto the measurement object, the position detection pattern P1 and the color code pattern P3 are detected from the captured image of the measurement pattern projected by the projection unit 12, and the color code And a pattern detection unit 491 for identifying.

Here, it is assumed that the measurement pattern includes an orientation pattern. Further, the three-dimensional measurement may be performed using absolute coordinates or relative coordinates. The position detection pattern typically includes a retro target and a template pattern. However, the position detection pattern is not limited to this, and any pattern can be used as long as the position can be specified using a lattice pattern or a dot pattern. Further, as the color code pattern, a pattern in which a plurality of rectangular unit areas are adjacent to each other is typically used. However, the color code pattern is not limited to this, and a pattern colored in a plurality of retro targets may be used. In addition, one unit area may be used with different coloring. The pattern projection control unit, the pattern detection unit, and the pattern formation unit are typically realized in a personal computer and may be physically separated from the projection unit.
With this configuration, each color-coded sign can be identified, so that it becomes easy to search for corresponding points in stereo images, connect adjacent images, and set a stereo matching area, and can be automated. In addition, this makes it possible to improve the efficiency of orientation and three-dimensional measurement and promote automation.

  The three-dimensional measurement projector according to claim 2 has a predetermined positional relationship with respect to the position detection pattern P1 and the position detection pattern P1 as shown in FIG. 25, for example. A pattern storage unit 495 that stores a plurality of measurement patterns P including a color-coded marker CT having a color code pattern P3 that is arranged and has a plurality of colors for identifying the marker, and is stored in the pattern storage unit 495 A pattern selection unit 496 that selects a measurement pattern to be projected from the plurality of measurement patterns, a projection unit 12 that projects the measurement pattern selected by the pattern selection unit 496 onto the measurement object, and a measurement that is projected by the projection unit 12 Pattern detection for identifying the color code by detecting the position detection pattern P1 and the color code pattern P3 from the captured image of the pattern. And a part 491.

  Here, the pattern selection unit is typically realized in a personal computer, and the pattern storage unit is realized in a personal computer or an external storage device, so these are physically separated from the projection unit. Also good. With this configuration, each color-coded sign can be identified, so that it is easy to search for corresponding points in a stereo image, connect adjacent images, and set a stereo matching area, and can be automated. In addition, this makes it possible to improve the efficiency of orientation and three-dimensional measurement and promote automation.

  Further, the invention described in claim 3 is the three-dimensional measurement projection apparatus according to claim 1 or 2, wherein, for example, as shown in FIG. 1, an imaging unit that captures a measurement pattern projected by the projection unit 12 The pattern detection unit 491 detects the position detection pattern P1 and the color code pattern P3 from the captured image of the measurement pattern captured by the imaging unit 10, and identifies the color code.

  The three-dimensional measurement projection apparatus according to claim 4 is the three-dimensional measurement projection apparatus according to claim 1 or 3, wherein the pattern forming unit 492 includes a single-color target pattern including only a position detection pattern. Form. With this configuration, for example, the color code pattern can be used for the reference point measurement, and the single color target pattern can be used for the single color target pattern.

  The three-dimensional measurement projection apparatus according to claim 5 is the three-dimensional measurement projection apparatus according to claim 2 or 3, wherein the pattern storage unit 495 includes a single-color target pattern composed of only a position detection pattern. Remember. With this configuration, for example, the color code pattern can be used for the reference point measurement, and the single color target pattern can be used for the single color target pattern.

  A three-dimensional measurement projection apparatus according to claim 6 is the three-dimensional measurement projection apparatus according to any one of claims 1 to 5, wherein the projection unit 12 is controlled to project a measurement pattern. A pattern projection control unit 493 for causing the projection unit 12 to project a random pattern in which position detection patterns are arranged at random, and a measurement mode for projecting a measurement pattern and a random pattern mode for projecting a random pattern. Can be switched. With this configuration, for example, it is possible to easily switch between orientation and three-dimensional measurement.

  A projection apparatus for 3D measurement according to claim 7 is the projection apparatus for 3D measurement according to any one of claims 1 to 5, wherein the projection unit 12 is controlled to project a measurement pattern. A pattern projection control unit 493 for causing the projection unit 12 to project an overlapping shooting range display pattern for displaying an overlapping range of stereo images, and a measurement mode for projecting a measurement pattern and an overlapping shooting range display. The shooting range display mode for projecting the pattern can be switched. With this configuration, for example, the orientation and the setting of the imaging range can be easily switched.

  The invention according to claim 8 is a pattern projection control unit that controls the projection unit 12 to project a measurement pattern in the three-dimensional measurement projection apparatus according to any one of claims 1 to 5. 493, and the pattern projection control unit 493 inputs one of the focal length, the shooting distance, the base line length, and the overlap rate related to the imaging unit 10, thereby determining the arrangement of the measurement points of the measurement pattern and the pattern density. It can be adjusted. Here, the measurement points include orientation points. If comprised in this way, a suitable measurement pattern can be selected according to the focal distance etc. of an imaging part.

  The invention according to claim 9 is a pattern projection control unit that controls the projection unit 12 to project a measurement pattern in the three-dimensional measurement projection apparatus according to any one of claims 1 to 5. The pattern projection control unit 493 has the projection unit 12 perform illumination for texture acquisition that illuminates the object with uniform illumination light, and performs measurement mode for projecting the measurement pattern and illumination for texture acquisition. The texture lighting mode can be switched. If comprised in this way, the three-dimensional shape of a measuring object can be roughly grasped | ascertained using texture illumination mode.

  The invention described in claim 10 is the projection apparatus for three-dimensional measurement according to claim 9, wherein the pattern detection unit 491 is based on the color acquired from the captured image of the pattern projected in the texture illumination mode. The color correction unit 494 corrects the color of the color-coded target CT projected by the projection unit 12. If comprised in this way, the color of the target with a color code can be corrected according to the brightness of a picked-up image, and identification of a color code can be made easy.

  A three-dimensional measurement system 100 according to an eleventh aspect includes the three-dimensional measurement projection apparatus according to any one of the first to tenth aspects. With this configuration, it is possible to project a target with a color code, search for corresponding points in a stereo image, connect adjacent images, and set a stereo matching area, and can be automated. In addition, this makes it possible to improve the efficiency of orientation and three-dimensional measurement and promote automation.

  The calculation processing unit 49 of the projection device for three-dimensional measurement according to claim 12 is arranged in a predetermined positional relationship with respect to the position detection pattern P1 and the position detection pattern P1 for indicating the measurement position, A pattern forming unit 492 that forms a measurement pattern P including a color-coded marker CT having a color code pattern P3 with a plurality of colors for identifying the marker, and the projection unit 12 to project the measurement pattern A pattern projection control unit 493 to be detected, and a pattern detection unit 491 that detects the position detection pattern P1 and the color code pattern P3 from the captured image of the measurement pattern projected by the projection unit 12 and identifies the color code. The invention is a calculation processing unit corresponding to the three-dimensional measurement projector according to claim 1.

The three-dimensional measurement pattern projection method according to claim 13 is, for example, as shown in FIG. 24, a position detection pattern P1 for indicating a measurement position and a predetermined positional relationship with respect to the position detection pattern P1. A pattern forming step S810 for forming a measurement pattern P including a color-coded marker CT having a color code pattern P3 provided with a plurality of colors for identifying the marker. A projection step S840 for projecting the measurement pattern onto the measurement object, an imaging step S850 for photographing the measurement pattern projected by the projection step S840, and a position detection pattern P1 from the photographed image of the measurement pattern photographed by the imaging step S850. A pattern detection step 860 for detecting the color code pattern P3 and identifying the color code.
With this configuration, each color-coded sign can be identified, so that it is easy to search for corresponding points in a stereo image, connect adjacent images, and set a stereo matching area, and can be automated. In addition, this makes it possible to improve the efficiency of orientation and three-dimensional measurement and promote automation.

The three-dimensional measurement pattern projection method according to claim 14 is, for example, as shown in FIG. 28, a position detection pattern P1 for indicating a measurement position and a predetermined positional relationship with respect to the position detection pattern P1. A pattern storage step S820 for storing a plurality of measurement patterns P including a color-coded label CT having a color code pattern P3 provided with a plurality of colors for identifying the marker and stored in the pattern storage step S820 A pattern selection step S830 for selecting a measurement pattern to be projected from the plurality of measured patterns P, a projection step S840 for projecting the measurement pattern selected in the pattern selection step S830 onto the measurement object, and a projection step S840 are used for projection. An imaging process S850 for capturing the measured pattern, and a measurement pattern captured by the imaging process S850. Detecting the over emissions photographing position detection pattern from the image P1 and the color code pattern P3, and a identifying pattern detection step S860 the color code.
With this configuration, each color-coded sign can be identified, so that it is easy to search for corresponding points in a stereo image, connect adjacent images, and set a stereo matching area, and can be automated. In addition, this makes it possible to improve the efficiency of orientation and three-dimensional measurement and promote automation.

  The invention described in claim 15 is the three-dimensional measurement pattern projection method according to claim 13, wherein the pattern forming step S810 forms a single-color target pattern consisting only of the position detecting pattern, and the pattern detecting step S860. Detects a monochromatic target pattern. With this configuration, the measurement can be made more efficient by using, for example, a color code pattern for reference point measurement and a monochromatic target pattern for precise measurement.

  Further, in the invention described in claim 16, in the pattern projection method for three-dimensional measurement according to claim 14, the pattern storage step S820 stores a single color target pattern consisting only of the position detection pattern, and the pattern detection step S860. Detects a monochromatic target pattern. With this configuration, the measurement can be made more efficient by using, for example, a color code pattern for reference point measurement and a monochromatic target pattern for precise measurement.

The invention described in claim 17 is the three-dimensional measurement projection method according to claim 15 or claim 16, wherein the captured image is a paired stereo image in the imaging step S850, and the stereo image is oriented. A standardization step S30 to be performed and a three-dimensional measurement step S50 for measuring the three-dimensional shape of the measurement object. As a point, the monochromatic target pattern is projected as a reference point. Here, the target pattern may be projected onto the measurement point indicating the measurement reference position and used for photographing as it is, or the target pattern may be pasted and used for photographing. If comprised in this way, measurement can be made efficient.

  ADVANTAGE OF THE INVENTION According to this invention, the non-contact three-dimensional measurement of a wide area can be made efficient and automation can be accelerated | stimulated using the projection apparatus which projects a target pattern.

[First Embodiment]
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. In the present embodiment, an example of reconstructing a measurement pattern used for orientation or three-dimensional measurement using projection of a measurement pattern (including an orientation pattern) as a pre-measurement preparation, and a target (signpost) constituting the measurement pattern ) Shows an example using a target with a color code.

[Configuration of Projector]
FIG. 1 shows a block diagram of a basic configuration example of a projection apparatus 80 according to the present embodiment. In FIG. 1, 12 is a projector as a projection unit that projects various projection patterns such as measurement patterns, 10 is a stereo camera as an imaging unit that captures the projected pattern, and 49 is a calculation processing unit. A pattern detection unit 491 that detects a feature point, a measurement point, a sign (target), etc. of the measurement object 1 from the captured image, various projection patterns, a target CT with a color code used for these projection patterns, a reference point RF, etc. A pattern forming unit 492 that forms pattern elements, and a pattern projection control unit 493 that projects the projection pattern formed by the pattern forming unit 492 onto the projection unit 12 are provided. In addition, a color correction unit 494 that corrects the color of the projection pattern is provided. For example, the color correction unit 494 corrects the color of the color-coded target CT in the projection pattern based on the color of the captured image obtained in the texture illumination mode.

  The projection pattern includes various patterns such as a measurement pattern, an orientation pattern, a random pattern, a measurement preparation pattern, an overlapping photographing range display pattern, and a texture illumination pattern. The measurement pattern P displays a measurement point Q (position detection pattern or the like) used for three-dimensional measurement, and the measurement point Q projected on the measurement object is used as a three-dimensional shape measurement point. The orientation pattern displays orientation points used for orientation, and the orientation points projected on the measurement object are stereo-photographed and used for orientation. By the way, when comparing the measurement pattern and the orientation pattern, the number of measurement points is usually larger than the number of orientation points, but there is no other distinction, and when it is used for three-dimensional measurement, it is usually called a measurement pattern. When it is used for, it is called orientation pattern.

  The random pattern is a type of measurement pattern in which measurement points are randomly arranged. The measurement preparation pattern is a pattern used for preliminary measurement of orientation and three-dimensional measurement. For example, a grid pattern as shown in FIG. 16 or a pattern in which a large number of small circles are arranged in an array is used. The center point of the small point is used as the measurement point and the orientation point. However, the present invention is not limited to these, and normal orientation patterns and measurement patterns can also be used as measurement preparation patterns. In the present embodiment, the measurement pattern, orientation pattern, random pattern, and measurement preparation pattern are included in the measurement pattern, and the orientation point is included in the measurement point.

  The overlapping shooting range display pattern displays the overlapping range of stereo images. For example, as shown in FIG. 7A, target CTs with color codes are arranged at the four corners, and the two left and right stereo images are the color of these four corners. An overlap portion is defined so as to include a corded target, and this is displayed. The texture illumination pattern is a pattern for performing texture acquisition illumination that illuminates the object with uniform illumination light without using a tangible pattern.

[System configuration]
FIG. 2 shows a block diagram of an overall configuration example of the three-dimensional measurement system 100 in the present embodiment. The three-dimensional measuring apparatus 100 includes an imaging unit 10, a projecting unit 12, a captured image data storage unit 13, a corresponding unit 40, a calculation processing unit 49, a display image forming unit 50, and a display device 60. Among these, the captured image data storage unit 13, the corresponding unit 40, the calculation processing unit 49, and the display image forming unit 50 are configured by a computer, for example. The measurement object 1 is a tangible object that is a construction object or a production object, and corresponds to, for example, various works such as buildings and factories, people, and landscapes.

  The projection unit 12 projects various patterns onto the measurement object 1. The imaging unit 10 acquires a captured image of the measurement object 1 (typically a stereo image, but single photo images may be used in pairs). For example, a surveying stereo camera or a general-purpose digital camera is used. A device in which a camera and a device that compensates for lens aberration with respect to an image of the measurement object 1 photographed by these cameras is combined is used. The captured image data storage unit 13 stores a captured image of the measurement object 1 and stores, for example, a stereo image of the measurement object 1 captured by the imaging unit 10.

  The correspondence unit 40 performs orientation and stereo matching by associating a pair of captured images or model images related to the measurement object 1. When the image of the measurement object 1 is a stereo image, a sign with a color code is used. After extraction, reference point setting, and corresponding point search, orientation processing is performed. In addition, stereo matching is performed for three-dimensional measurement. Extraction unit 41, reference point setting unit 42, corresponding point search unit 43, orientation unit 44, corresponding point instruction unit 45, identification code determination unit 46, pattern information storage unit 47, photographing / model image display unit 48, model image formation unit 48A, a model image storage unit 48B, and a calculation processing unit 49. Among these, the extraction unit 41, the identification code determination unit 46, and the pattern information storage unit 47 also function as the pattern detection unit 491 of the calculation processing unit 49. The matching processing unit 70 has a main function of stereo matching, and includes a reference point setting unit 42, a corresponding point searching unit 43, and a corresponding point instruction unit 45.

  The reference point setting unit 42 searches for a point that is in the vicinity of a specified point on one image (reference image) of the stereo image and matches the feature point, and uses the point that matches the feature point as a reference point. Set. The feature points include, for example, the center position, the center of gravity position, the corner position of the measurement object 1, and a sign (target) attached to or projected on the measurement object 1. The corresponding point search unit 43 determines a corresponding point corresponding to the reference point set by the reference point setting unit 42 on the other image (search image) of the stereo image. When the operator designates an instruction point in the vicinity of the feature point using the corresponding point instruction unit 45, the feature point originally intended by the operator in the reference image is displayed by the reference point setting unit 42 even if the operator does not strictly specify the feature point. The corresponding points of the search image are determined by the corresponding point search unit 43.

  The orientation unit 44 uses the reference points set by the reference point setting unit 42 and the corresponding points obtained by the corresponding point search unit 43 to obtain corresponding point relationships between a pair of images such as stereo images, and performs orientation calculation processing. Do. The corresponding point instruction unit 45 determines a corresponding point on the search image when the operator designates a point other than the vicinity of the feature point of the reference image. The operator compares the indicated point on the reference image displayed on the display device 60 with the display position of the corresponding point on the search image determined by the corresponding point indicating unit 45, and thereby the feature point of the measurement object 1 Can be easily recognized. Further, the relative orientation by the orientation unit 44 is performed using the position correspondence by the corresponding point instruction unit 45.

  The calculation processing unit 49 receives image data from the imaging unit 10 to detect various patterns, generates various patterns, and causes the projection unit 12 to project them. The pattern detection unit 491 detects various patterns. Among these, the functions of the extraction unit 41 and the identification code determination unit 46 will be described later with reference to FIG. The pattern information storage unit 47 stores pattern information such as the position coordinates of the color code target, the color code, and the reference point position coordinates detected and determined by the extraction unit 41 and the identification code determination unit 46. Note that color correction of the extracted captured image is performed by the color correction unit 312 in the extraction unit 41, and color correction of the projection pattern formed or selected is performed by the color correction unit 494 of the calculation processing unit 49.

  The model image forming unit 48A forms a model image from the parameters (position and tilt of the photographed camera) subjected to the orientation calculation processing by the orientation unit 44. Here, the model image is also referred to as a displacement corrected image, and corresponding points of the left and right images (stereo images) that are a pair of the captured images are rearranged on the same epipolar line EP (see FIG. 10B), It is rearranged in a viewable image. The model image storage unit 48B stores the model image of the measurement object 1 formed by the model image forming unit 48A. The shooting / model image display unit 48 uses the captured image or the model image formed by the model image forming unit 48A as a pair of images in the processing such as extraction, reference point setting, corresponding point search, and stereo matching performed by the corresponding unit 40. It is displayed on the display device 60.

  The display image forming unit 50 creates and displays a stereoscopic two-dimensional image of the measurement object 1 from any direction based on the three-dimensional coordinate data of the measurement object 1 and the captured image or model image of the measurement object 1. Therefore, the three-dimensional coordinate data calculation unit 51 obtains the three-dimensional measurement position coordinates related to the measurement object 1 by calculation, and the result is stored in the three-dimensional coordinate data storage unit 53. The stereoscopic two-dimensional image forming unit 54 creates a stereoscopic two-dimensional image based on the obtained three-dimensional coordinate data, and stores the result in the stereoscopic two-dimensional image storage unit 55. The stereoscopic two-dimensional image display unit 57 displays a stereoscopic two-dimensional image viewed from an arbitrary direction on the display device 60 based on the information stored in the stereoscopic two-dimensional image storage unit 55.

[Target with color code]
FIG. 3 shows an example of a target with a color code. 3A shows three color code unit areas, FIG. 3B shows six color code targets, and FIG. 3C shows nine color code targets. 3A to 3C, the color-coded targets CT (CT1 to CT3) include a position detection pattern (retro target portion) P1, a reference color pattern (reference color portion) P2, and a color code pattern (color code portion). ) P3 and an empty pattern (white portion) P4. These position detection pattern P1, reference color pattern P2, color code pattern P3, and sky pattern P4 are arranged at predetermined positions in the target CT1 with color code. That is, the reference color pattern P2, the color code pattern P3, and the sky pattern P4 are arranged in a predetermined positional relationship with respect to the position detection pattern P1.

  The retro target portion P1 is used for detecting the target itself, detecting the center of gravity thereof, detecting the direction of the target (representing the degree of inclination), and detecting the target area.

  The reference color portion P2 is used for reference at the time of relative comparison and for color calibration for correcting the color misregistration in order to cope with color misregistration due to photographing conditions such as illumination and a camera. Furthermore, the reference color portion P2 can be used for color correction of the target CT with a color code created by a simple method. For example, when using a target CT with a color code printed by a color printer (inkjet, laser, sublimation type printer, etc.) that has not been color-controlled, individual differences may occur in the color of the printer used. By comparing and correcting the colors of the part P2 and the color code part P3, the influence of individual differences can be suppressed.

  The color code part P3 expresses a code by a combination of color schemes for each unit area. The number of codes that can be expressed varies depending on the number of code colors used for the code. For example, if the number of code colors is n, the color code target CT1 in FIG. 3A has three unit areas of the color code portion P3, and therefore can represent n × n × n codes. In order to increase the reliability, even when the condition that the colors used in other unit areas are not used redundantly, n × (n−1) × (n−2) codes can be expressed. If the number of code colors is increased, the number of codes can be increased. Furthermore, if the condition that the number of unit areas of the color code part P3 is equal to the number of color codes is imposed, all code colors are used for the color code part P3, so that not only the comparison with the reference color part P2, By relatively comparing the colors between the unit areas of the color code part P3, the color of each unit area can be confirmed to determine the identification code, and the reliability can be improved. Furthermore, if a condition for making the area of each unit region the same is added, the target CT with a color code can also be used when detected from the image. This is because the area occupied by each color is the same even between the target CTs with color codes having different identification codes, so that almost the same dispersion value is obtained from the detection light from the entire color code part. In addition, since the boundary between the unit regions is repeated at equal intervals and a clear color difference is detected, it is possible to detect the target CT with a color code from the image from such a repeated pattern of detection light. .

  The white portion P4 is used for detecting the direction of the color-coded target CT and for correcting color misregistration. Among the four corners of the color-coded target CT, there is a place where no retro target is arranged at one place, and this can be used for detecting the direction of the color-coded target CT. Thus, the white part P4 should just be a pattern different from a retro target. Therefore, a character string such as a number for visually confirming the code may be printed on the white portion, and it may be used as a code area such as a barcode. Furthermore, in order to increase the detection accuracy, it can be used as a template pattern for template matching.

  FIG. 4 shows a configuration example of the extraction unit 41 that extracts the color-coded target CT and the identification code determination unit 46 that determines the color code. The extraction unit 41 includes a search processing unit 110, a retro target grouping processing unit 120, a target detection processing unit with color code 130, and an image / color pattern storage unit 140. Also, the identification code determination unit 46 determines the color code detected by the color code-added target detection processing unit 130 and assigns a code number.

The search processing unit 110 detects a position detection pattern P1 such as a retro target pattern from a color image (captured image or model image) read from the captured image data storage unit 13 or the model image storage unit 48B. When a template pattern is used as the position detection target instead of the retro target pattern, the template pattern is detected.
The retro target grouping processing unit 120 is the same group of those in which the retro targets detected by the search processing unit 110 are determined to belong to the same color coded target CT (for example, the coordinates are in the color coded target CT). Group as candidates belonging to.

  The color code target detection processing unit 130 detects a region and direction of the color code target CT from a group of retro targets determined to belong to the same color code target. The reference color portion P2 of the color-coded target CT, the color arrangement in the color code portion P3, the color detection processing portion 311 for detecting the color of the measurement object 1 in the image, and the reference color pattern P2 A code part P3, a color correction part 312 for correcting the color of the measuring object 1 in the image, and a confirmation processing part 313 for confirming whether or not the grouping is properly performed. Note that color correction of the extracted captured image is performed by the color correction unit 312, and color correction of the projection pattern formed or selected is performed by the color correction unit 494.

  The image / color pattern storage unit 140 is a color code for a read image storage unit 141 that stores an image (photographed image / model image) read by the extraction unit 41 and a plurality of types of color-coded target CTs that are scheduled to be used. A type code number indicating the type of the attached target CT is stored, and for each type of target CT with a color code, a color code target correspondence table 142 is recorded which records the correspondence between the pattern arrangement and the code number.

  The identification code discriminating unit 46 discriminates an identification code from the color arrangement in the color code unit P3 and converts it into an identification code. The color code-added target CT detected by the color code target detection processing unit 130 Based on the direction data, the coordinate conversion processing unit 321 for converting the coordinates of the color-coded target CT and the color code in the color code portion P3 of the coordinate-converted target CT with the color code discriminates the identification code. The code conversion processing unit 322 converts the code into a code.

[System Operation]
FIG. 5 shows an example of a flowchart for explaining the operation of the three-dimensional measurement system. Here, a basic flow excluding the flow related to various pattern projections is shown, and the flow related to various pattern projections will be described later with reference to FIG.
First, a target with a color code is attached to the object 1 (S01). When projection is performed, the projection is performed instead of or in combination with the pasting. The position where the target with the color code is pasted is used as a measurement point Q for orientation and three-dimensional measurement. Next, an image (typically a stereo image) of the measurement object 1 photographed using the imaging unit 10 such as a digital camera is photographed (S10), and the photographed image is registered in the photographed image data storage unit 13. (S11).

  FIG. 6 shows an example of overlap photography. The measurement object 1 is photographed so as to overlap with one, two, or a plurality of cameras 10 (S10). The number of cameras 10 may be one to a plurality or any number, and there is no particular limitation. Basically, as shown in FIG. 6 (b), a stereo image is taken with a set of two cameras, and a series of stereo images are acquired while overlapping a part of the stereo images, and orientation and three-dimensional images are obtained. As shown in Fig. 6 (a), you may shoot so that it overlaps from multiple directions with one camera, or shoot so that it overlaps with multiple multi-cameras. Also good. In this case, a pair is formed by two overlapping images, but one image may form, for example, one pair with the image on the left and another pair with the image on the right.

  FIG. 7 shows an example of a photographed image photographed by the left and right stereo cameras. FIG. 7A shows how the stereo images overlap. The basic range to be measured is an overlap range of two (a pair of) stereo shot images. At this time, it is preferable to perform imaging so that the four color-coded target CTs fall within the overlapping range. In this way, three-dimensional measurement is possible using a stereo image. FIG. 7B shows an example of how to overlap adjacent stereo images. In this way, it is preferable to take a series of images so as to overlap, including two color-coded targets CT in the vertical and horizontal directions. In this way, non-contact three-dimensional measurement over a wide area can be automated.

  Returning to FIG. 5, the correspondence unit 40 then transfers the captured image registered in the captured image data storage unit 13 or the model image stored in the model image storage unit 48 </ b> B to the image / color pattern storage unit 140 of the extraction unit 41. Read. The extraction unit 41 extracts the color-coded target CT from the captured image (S14). The identification code determination unit 46 determines the identification code of the extracted color-coded target CT (S15), and the extracted position coordinates and identification code of the color-coded target CT are stored in the pattern information storage unit 47.

[Stereo pair settings]
Next, the process proceeds to stereo pair setting. Among the images registered in the photographed image data storage unit 13 using the identification code, a set of left and right images forming a stereo pair is set (S16).

  FIG. 8 shows a flow example of stereo pair selection (S16). First. The numbers of the target CTs with color codes registered for each image are listed (S550), and a stereo pair is selected from images containing a plurality of target CTs with common code numbers (S560). As shown in FIG. 7 (a), when taking a stereo image so as to include four color-coded target CTs, there are two images each containing four color-coded target CTs. Stereo pairs can be set. In addition, as shown in FIG. 7B, when two color-coded target CTs having a common code number are included in each stereo pair, there is a relationship between adjacent images on the upper, lower, left, and right sides of the image. Therefore, the arrangement relationship between the stereo pairs can be determined (S570). This stereo pair selection flow can be performed automatically.

  Next, the reference point setting unit 42 searches for a point that matches the feature point in the vicinity of the point designated on one image (reference image) of the stereo image, and finds a point that matches the feature point. A reference point is set (S18). In addition, the corresponding point search unit 43 determines a corresponding point corresponding to the reference point on the other image (search image) of the stereo image (S19).

[orientation]
Next, orientation is performed by the orientation unit 44 (S30). The relative orientation of the stereo image of the measuring object 1 stored in the captured image data storage unit 13 is performed, and the corresponding point relationship with respect to the model image of the stereo image is obtained.
Here, the corresponding points (same points) of two or more images are designated on the reference image by the reference point setting unit 42 and the corresponding point searching unit 43 on the reference image by the operator on the reference image. On the other hand, the image coordinates of the corresponding point corresponding to the reference point matching the feature point are read. Six or more corresponding points are required for each normal image. If the three-dimensional coordinate data separately measured with a three-dimensional position measuring device (not shown) for the measurement object 1 is stored in the three-dimensional coordinate data storage unit 53 in advance, the reference point coordinates and the image are associated with each other to perform absolute orientation. Execute. If it is not stored, relative orientation is executed.

For example, if there are four color code targets in the overlapped stereo image and there are three position detection patterns (retro targets) in one color code target, a total of 12 position detection patterns Orientation processing can be performed from the coordinates of the center of gravity of the (retro target). Since orientation can be performed with at least 6 points or more, the position detection pattern may be at least 2 points in a target with a color code. In that case, the orientation process is performed at 8 points.
The orientation process can be performed automatically, manually, or semi-automatically. In semi-automatic, automatic position detection can be performed by clicking the vicinity of the position detection pattern P1 in the color-coded target CT with the mouse.

  Next, for each image selected as a stereo pair, the orientation calculation process is performed by the orientation unit 44 using the coordinates of the corresponding points. Through the orientation calculation process, the positions and tilts of the captured left and right cameras, the positions of corresponding points, and the measurement accuracy can be obtained. The orientation calculation processing is performed by mutual orientation for associating a pair of captured images or a pair of model images, and for orientation between a plurality of images or all images by bundle adjustment.

  FIG. 9 is an explanatory diagram of a model coordinate system XYZ and a camera coordinate system xyz in a stereo image. The origin of the model coordinate system is taken as the left projection center, and the line connecting the right projection centers is taken as the X axis. For the scale, the base line length is taken as the unit length. The parameters to be obtained at this time are 5 of the left camera Z-axis rotation angle κ1, the Y-axis rotation angle φ1, the right-hand camera Z-axis rotation angle κ2, the Y-axis rotation angle φ2, and the X-axis rotation angle ω2. One rotation angle. In this case, since the rotation angle ω1 of the X axis of the left camera is 0, there is no need to consider it. The parameters required for determining the positions of the left and right cameras can be obtained from the coplanar conditional expression.

  The model image forming unit 48A forms a pair of model images based on the parameters determined by the orientation unit 44 (S42), and the model image formed by the model image forming unit 48A is stored in the model image storage unit 48B. (S43). The shooting / model image display unit 48 displays the model image on the display device 60 as a stereo image (S44).

  FIG. 10 shows an example of a target having a reference point RF. By performing orientation using a measurement pattern having a reference point RF and by repeating orientation, the accuracy of orientation is improved, but such orientation is usually based on a model image that has been subjected to orientation processing once. Done. The model image is read from the model image storage unit 48B into the read image storage unit 141 of the extraction unit 41 and used for re-orientation. In FIG. 10A, a plurality of retro targets are arranged as reference points RF. In the planar measurement object 1, only the target CT with the color code may be used. However, when the curved surface of the measurement object 1 is complicated or the curvature is large, these are added to the target CT with the color code. The reliability of orientation and measurement becomes higher when a large number of retro targets are attached as reference points RF.

  FIG. 11 shows a flow example of automatic association of reference points. Here, automatic position detection and association of the reference point RF will be described. First, the position of the position detection pattern (retro target) P1 in the color-coded target CT is detected (S110). In FIG. 10A, the position detection patterns P1 in the four color-coded targets CT have a total of 12 points and 6 points or more, and an orientation process is possible. Therefore, orientation processing is performed, and then displacement correction processing is performed. In general, the position and inclination of the photographing camera are obtained by the orientation work (S120), and an operation for creating a deviation corrected image is performed using the result (S130). Here, the model image forming unit 48A uses the orientation result of the orientation unit 44 to create a displacement corrected image.

  The model image is also referred to as a displacement corrected image, and is obtained by rearranging the corresponding points of the left and right images, which are a pair of captured images, on the same epipolar line EP and rearranging the images in a stereoscopic view. A displacement correction image (model image) is created by the displacement correction processing. The deviation corrected image is an image obtained by rearranging the epipolar lines EP of the left and right images so as to be aligned horizontally. Accordingly, as shown in FIG. 10B, the reference points RF of the left and right images are rearranged on the same epipolar line EP. When a model image is formed using the result of the orientation process, such a displacement corrected image can be obtained.

  Next, a target to be a reference point RF on the same epipolar line EP is searched (S140). If the image is a displacement correction image, only a one-dimensional search on the same line is required, and the search is easy. In other cases, not only the epipolar line EP but also the vicinity thereof is searched for several lines. As shown in FIG. 10B, when the reference point RF is found on the same line, it is associated as a corresponding point and identified (numbered) (S150). Here, if there are a plurality of reference points RF on the same line, each reference point RF is identified from its left and right positions. Next, the detected reference point RF is added and orientation is performed again (S160). Here, the reliability of the orientation can be improved by performing the orientation again. If the accuracy of the orientation result is sufficient (S170) and there is no problem, the process ends. If it is insufficient, the bad point is deleted (S180), and the orientation is performed again (S160).

[Matching area decision]
Returning to FIG. 5, next, the matching unit 40 determines a matching area (determines a three-dimensional measurement range) (S45), and the three-dimensional coordinate calculation unit 51 performs three-dimensional measurement (stereo measurement) (S50). ), The three-dimensional coordinates of the corresponding points of the stereo image are registered in the three-dimensional coordinate data storage unit 53. Manual measurement, semi-automatic measurement, and automatic measurement are possible for the determination of the matching area (S45). In semi-automatic, automatic position detection can be performed by clicking the vicinity of the position detection pattern P1 in the color-coded target CT with the mouse.

  Next, the automatic determination of the stereo matching area will be described. As shown in FIG. 7A, the matching range is automatically set by the corresponding point search unit 43 so as to include the color-coded targets CT arranged at the four corners of the stereo image. Prior to matching, a series of model images related to the measurement object 1 are arranged so that the identification codes of the color-coded markers CT shared by the adjacent model images match for a series of model images. It may be used.

  FIG. 12 shows an example of a processing flow for automatically determining the stereo matching area. FIG. 13 is a diagram for explaining stereo matching area setting. FIG. 13 shows an example in which the color-coded target CT has three retro targets for position detection. First, the color-coded target CTs arranged at the four corners of the stereo image are extracted (S160). Next, each retro target part P1 in the four color-coded targets CT is detected (S170). Refer to the description of FIGS. 17 and 18 for these detections. Next, an area connecting the outermost retro target portions P1 is set as a measurement area so as to include all retro target portions P1 from the detected coordinate values of each retro target portion P1. That is, if the upper left is the origin (0, 0), the horizontal upper line is connected if the points with the smallest Y coordinate are connected, the horizontal lower line is connected with the point where the Y coordinate is the maximum value, and the X coordinate is the minimum value. The matching area to be measured can be automatically determined, such as connecting a point to the left line in the vertical direction and connecting the point with the maximum X coordinate to the right line in the vertical method.

  In this way, by determining the matching region, as shown in FIG. 7B, the overlap between the model images can be reliably obtained. In other words, the color matching target CT is placed near the four corners of the screen, and the area connecting the outermost retro targets of these color coding targets CT is always used as the matching area, so that the stereo matching area is automatically set. At the same time, the overlap between the model images can be ensured. In this case, if at least two or more position detection patterns (retro target portions) P1 are arranged on each color code target CT (in the case of two points, they are arranged diagonally), matching area automatic setting processing can be performed. .

  In the fully automatic case, if the number of code identifications is large, a pair of images (typically stereo images) can be taken as a basic unit while overlapping adjacent images so that images can be taken in any order. If the shooting order is determined, automation is possible even if the number of code identifications is small. In that case, it is only necessary to identify the target CT with a color code in two images that have been stereo-photographed (overlapped). Three-dimensional measurement (stereo measurement) is performed on the region where the matching area is determined (S45) (S50). For the three-dimensional measurement, for example, image correlation processing using a cross-correlation coefficient method is used. The image correlation process is performed by the function of the corresponding unit 40 (extracting unit 41, reference point setting unit 42, corresponding point searching unit 43, etc.) and the calculation process of the three-dimensional coordinate data calculating unit 51.

  The three-dimensional coordinates of the measurement object 1 are obtained by the calculation process of the three-dimensional coordinate data calculation unit 51 and stored in the three-dimensional coordinate data storage unit 53. From the three-dimensional coordinates obtained by the three-dimensional coordinate data calculation unit 51 or the three-dimensional coordinates read from the three-dimensional coordinate data storage unit 53, the three-dimensional two-dimensional image forming unit 54 performs the three-dimensional measurement of the measurement object 1. A three-dimensional image is created, and the stereoscopic two-dimensional image storage unit 55 stores the stereoscopic two-dimensional image. The stereoscopic two-dimensional image display unit 57 displays a stereoscopic two-dimensional image viewed from an arbitrary direction on the display device 60 based on the information stored in the stereoscopic two-dimensional image storage 55.

Such a stereoscopic two-dimensional image screen is a stereoscopic two-dimensional image of the measuring object 1 that can represent a perspective state from an arbitrary direction, and an image can be displayed by a wire frame surface or texture mapping. . Texture mapping refers to attaching a texture that expresses a three-dimensional effect to the two-dimensional image of the measurement object 1.
In this way, automatic measurement is possible from photographing (S10) to three-dimensional measurement (S50), the three-dimensional coordinates of the measuring object 1 are obtained, and a three-dimensional image is displayed on the display device 60. (See Patent Document 2)

[Use of projection device]
In the present embodiment, the following processing is enabled by using the projection unit (projector) 12 in this basic processing flow.
(A) The projector 12 illuminates a range to be captured by the camera, and the stereo camera 10 is adjusted to capture the range.
It is also possible to arrange the color-coded target CT at the four corners of the projection pattern to display the imaging range (overlapping imaging range) and to use it for connecting adjacent images.
(B) The projector 12 projects texture illumination (illumination only), and the camera 10 captures a stereo pair image as a texture image (an image of the measurement object) of one model image.

(C) As a pre-measurement preparation, a measurement preparation pattern is projected from the projector 12, and this is photographed in stereo. As the measurement preparation pattern, for example, a pattern in which a large number of small circles are arranged in an array as shown in FIG. 16 or a lattice pattern can be used. Any pattern may be used as long as the shape of the measurement object 1 can be visually confirmed or calculated. Check by visual inspection or calculation. Since the projection pattern is deformed according to the shape of the measurement object 1, the outline of the shape of the measurement object 1 can be grasped by checking the deformation point of the pattern.

  It is possible to take measures such as attaching the reference point RF to the deformation point extracted in the pre-measurement preparation or increasing the measurement point Q (including the orientation point). In addition, the size, number, and arrangement of orientation points can be calculated and reflected in the actual pattern projection.

  When checking the deformation point in the pre-measurement preparation, it is possible to perform a rough measurement together. That is, the captured image is sent to the orientation unit 44 via the pattern detection unit 491, and orientation calculation is performed. Furthermore, when the number of measurement points in the measurement preparation pattern is sufficiently large, the measurement process can be completed using the projected orientation points as measurement points.

(D) In the orientation process, the color-coded target CT and the reference point RF are projected from the projector 12. Here, the color-coded target CT is pasted at the irradiated position. If it is affixed in preparation before measurement, affix it to other points. In addition, when measuring from the projected pattern, it is not necessary to paste, and this is stereo-photographed and used again for the orientation process.

(E) In the three-dimensional measurement, a measurement pattern is projected from the projector 12. In this case, for example, a random pattern is irradiated for stereo matching. The necessary accuracy of the measurement pattern is calculated in advance by the camera condition, and the measurement pattern having a size corresponding to the accuracy is irradiated. This is stereo-photographed and used for three-dimensional measurement.

(F) When moving to the next photographing position, the next photographing position may be roughly navigated by the projector 12.
Note that the above processing can be fully automated. In that case, the pasting operation is not performed, and the pre-measurement preparation, the orientation, and the three-dimensional measurement are performed using the projection patterns from the projectors.

FIG. 14 shows an example of a measurement flow using the projection apparatus. The projector 12 projects a first measurement pattern (preliminary measurement pattern or the like), and forms and projects a second measurement pattern (precision measurement pattern or the like) based on the deformation of the projected pattern.
First, a photographing condition is input (S200). The imaging unit 10 is composed of an optical system with variable focal length. The imaging condition can be obtained by inputting one of the number of pixels of the digital camera to be used, the approximate pixel size, the focal length, the imaging distance, the base line length, and the overlap rate as the camera parameters of the imaging unit 10. Resolution, angle of view, measurement area, etc. can be calculated. That is, the projection unit 12 can set a pattern range to be projected according to the shooting range in the imaging unit 10. Thereby, the arrangement of the measurement points of the preliminary measurement pattern and the pattern density can be adjusted. In addition, the number of shots can be calculated by inputting the side wrap rate, the size of the area to be measured, and the like. Alternatively, it is also possible to calculate a camera shooting distance, a base line length, and the like by including camera parameters and necessary accuracy (pixel resolution).

Next, camera parameters are calculated according to the input conditions (S210). In this case, according to the condition, possible for a plane pixel, depth resolution, the size of a measurement range in a pair of stereo images, a necessary number of images for obtaining an image of the entire measurement object, and the like are calculated.
The calculation formulas for plane resolution and depth resolution are as follows (* is a multiplication operator).

Δxy (planar resolution) = δp (pixel size) * H (shooting distance) / f (focal length)
Δz (depth resolution) = δp * H * H / (f * B (base length: distance between cameras))

Next, the position of the projector 12 and the projection conditions are set so as to match the camera parameter calculation result of the imaging unit 10 (S220).

  Next, the projector 12 is set in the shooting range display mode, and the range shot by the camera 10 is projected (S230). The projection may be illumination only in a range captured by the left and right stereo cameras 10. In this case, the range is illuminated or displayed automatically calculated from the previously input condition and the angle of view of the projector 12 to be used. The effective range where orientation and three-dimensional measurement are possible is determined from the overlapping range of the left and right shooting ranges. The overlapping imaging range display pattern displays the overlapping range (overlap portion) of the stereo image, and the pattern projection control unit 493 projects four color code target CTs, as shown in FIG. The patterns are set so as to be arranged at the four corners of the overlapping range, and the pattern forming unit 492 forms the pattern in this arrangement state as an overlapping photographing range display pattern. When the overlapping shooting range display pattern is superimposed on the various measurement patterns and projected, or when four color-coded target CTs are added and projected on the various measurement patterns at the same position as the overlapping shooting range display pattern, a stereo image is obtained. Can be photographed so as to include four color-coded target CTs, and a series of stereo images over the entire measurement object can be photographed so that adjacent images can be connected.

  Next, the camera position is set so that the projected range is almost full of the screen (S240). In this case, the four color-coded target CTs in the overlapping photographing range display pattern are set so as to surely enter the left and right stereo photographing screens. In addition, since the approximate camera position is known by the condition input previously, you may be able to project and confirm those camera conditions on a measurement object.

  Next, the illumination for texture is projected from the projector 12 using the projection mode as the texture illumination mode (S245). The texture illumination illuminates the object with uniform illumination light without using a tangible pattern, and is also effective for examining the position where the target is attached to the measurement object 1. If only the illumination is projected in the step of projecting the photographing range (S230), this operation is unnecessary.

Next, one model (stereo pair: two images) is photographed as a texture image (first photographing S250). When a texture image is not necessary, it is not necessary to take a picture here. Note that the color of the target CT with the color code projected by the projection unit 12 can be corrected based on the color acquired from the captured image of the pattern projected in the texture illumination mode. For example, the correction is performed by the pattern forming unit 492 using the color correcting unit 494, and the measurement pattern corrected by the pattern projection control unit 493 is projected on the projection unit 12.
In addition, the above process is performed at least before photographing (S10) before the processing flow of FIG.

  Next, preparation before measurement (preliminary measurement) (S255) will be described. The reason for preparing for measurement is to make it possible to efficiently perform actual orientation and three-dimensional measurement. Therefore, it may or may not be performed depending on the object. Or once you do it, you don't need to measure similar objects.

  FIG. 15 shows an example of a processing flow of preparation before measurement (S255). The pre-measurement preparation (S255) is performed before the photographing (S10) in FIG. First, the measurement preparation pattern is projected using the projection mode of the projector as the measurement mode (S300). The measurement preparation pattern is also one mode of the measurement pattern P. FIG. 16 shows an example of a measurement preparation pattern. FIG. 16A shows a pattern in which many small circles are arranged in an array (referred to as a small circle pattern), and FIG. 16B shows a lattice pattern. However, it is not restricted to these, What is necessary is just a pattern which can understand the shape of the measuring object 1 visually or by calculation processing. In other words, any pattern may be used as long as it can be arranged at appropriate intervals at positions where the shape of the measurement object has undulations. In the case of a small circle pattern or a lattice pattern, the projected pattern is deformed according to the shape of the measurement object 1. Therefore, the displacement of the measurement point is found by checking the deformation point of the pattern, and the outline of the shape of the measurement object 1 is obtained. Can be grasped. Here, the displacement refers to the displacement from the measurement point when the surface of the measurement object is projected onto a plane perpendicular to the projection light.

  Since the photographing condition is input in the photographing condition input step (S200), the size, number, arrangement, and the like of the orientation points are calculated from this value, and the measurement pattern is formed by the pattern forming unit 492. The pattern projection control unit You may make it project on the projection part 12 by 493. At the same time, the reference point RF and the target CT with a color code may be inserted or replaced at the intersection of the lattice, the center of gravity of the small circle, or the like, or a measurement preparation pattern may be inserted, or another shape may be inserted or replaced. Good.

  Next, the deformation point of the pattern is checked (S310). Check by visual inspection or calculation. Here, it is only necessary to be able to predict an approximate shape on a preliminary basis, so it is often possible even visually. In the case of the calculation process, the pattern detection unit 491 detects the displacement of the intersection of the lattice and the center of gravity of the small circle in the captured image from the stereo camera 10. Grid intersections and small circle centroids are included in the measurement points. For example, what is necessary is just to detect as a deformation | transformation point (point where the displacement has arisen) etc. where the intersection of a grid | lattice and the gravity point of a small circle are not equal intervals. In the case of a small circle pattern, the center of gravity point is detected by the center of gravity position detection algorithm, and the position is measured. As a result, if the measurement preparation pattern is the first measurement pattern P, and the intersection of the lattice and the center of gravity of the small circle are the measurement points Q, the pattern detection unit 491 can change the displacement of the measurement points in the first measurement pattern. Will be detected.

When a deformation point (a point where displacement has occurred) is detected, a reference point is pasted on the deformation point or a reference point is added to the measurement preparation pattern (S320). When the check is visual, the reference point may be pasted when the deformation point is checked. In the case of automation processing, reference points are added to the measurement preparation pattern in accordance with the magnitude of displacement, that is, the magnitude of deformation.
In this way, by performing preparation before measurement, pattern deformation points are found in advance, a target is pasted on the measurement object 1 as a reference point, or a projection pattern with reference points added is created. Increase the reference point. Thereby, orientation and three-dimensional measurement can be performed effectively. When creating a projection pattern with reference points added, the pattern forming unit 492 adds the measurement points based on the displacement of the measurement points in the first measurement pattern detected by the pattern detection unit 491. 2 measurement patterns are formed.

  Returning to the flow of FIG. 14, next, the color-coded target CT and the reference point RF are projected (S260). This step corresponds to S01 in FIG. When the pre-measurement preparation is performed, as a result, the reference point RF is increased to a deformation point of the measurement object 1 or a point corresponding to this of the measurement pattern P. Here, the color-coded target CT is attached to the irradiated position of the measurement object 1. If it is affixed in preparation, attach it to other points. In addition, when measuring using the projected measurement pattern P, it is not necessary to paste.

  Next, stereo shooting is performed (second shooting S270). This step corresponds to S10 in FIG. The second imaging is performed including the target CT with the color code attached to the measurement object or added to the measurement pattern and the reference point RF. The captured image obtained by the second imaging is used for orientation, and efficient orientation is performed (S275 corresponds to S11 to S30 in FIG. 5). Also, the measurement pattern used in the second imaging is standardized as the first measurement pattern, and new deformation points (displacements) are found at the orientation points (included in the measurement points). When the reference point near the deformation point is increased, the pattern detection unit 491 detects the displacement of the measurement point in the first measurement pattern, and the pattern formation unit 492 adds the measurement point based on this displacement. The measurement pattern is formed.

  Next, a measurement pattern for three-dimensional measurement is projected using the projection mode as a random pattern mode, for example (S280). This step corresponds to S01 in FIG. In this case, for example, a random pattern or the like is irradiated for stereo matching (three-dimensional measurement). Since the required accuracy of the measurement pattern is calculated in advance by the camera condition, the measurement pattern having a size corresponding to the accuracy is irradiated. When the pre-measurement preparation is performed, reference points are also added to the deformation points in the measurement pattern.

Next, stereo shooting is performed (third shooting S290). This step corresponds to S10 in FIG. The third imaging is performed including the target CT with the color code attached to the measurement object or added to the measurement pattern and the reference point RF. The captured image obtained by the third imaging is used for three-dimensional measurement, and efficient shape measurement is performed (S295, corresponding to S42 to S50 in FIG. 5). In addition, three-dimensional measurement is performed using the measurement pattern used in the third imaging as the first measurement pattern, and new deformation points (displacements) are found at the measurement points. When the reference point is increased, the pattern detection unit 491 detects the displacement of the measurement point in the first measurement pattern, and the pattern formation unit 492 forms a second measurement pattern in which the measurement point is added based on this displacement. Will be.
When measurement is performed at this installation position, the camera moves to the next shooting position (S298). That is, returning to S220 (returning to S200 in some cases), imaging is repeated until three-dimensional data of the entire measurement object is obtained.

  At this time, navigation may be performed for the next shooting position by the projector. Navigation is, for example, selecting the number and location of orientation points based on the degree of distortion of the projected grid pattern, performing rough measurements, examining the location of orientation points, or searching for mismatched areas and increasing the orientation points Or to consider. That is, the paste position or projection position of the pattern is determined according to the result of navigation.

In forming the second measurement pattern, the measurement points can be reduced or changed. For example, change the reference point to a target with a color code, change the color code pattern of a target with a color code, move a measurement point near a feature point to a feature point, or return to the previous stage to perform measurement. There are cases where the orientation defect is deleted or the orientation defect point is deleted.
It is also possible to fully automate the process of FIG. In this case, the target is not attached, and measurement preparation, orientation, and three-dimensional measurement are performed using the projection pattern from the projector. In addition, if the labeling with color code (S01) is replaced with the labeling with color code by using the projection device in the processing flow of FIG. 5, this flow can be fully automated).

[Detection flow of target with color code]
Next, a detection flow of a target with a color code will be described. Detection of a target with a color code is performed manually or automatically. In the case of automatic processing, it differs depending on the number of color identifications of the color-coded target CT or the imaging method. First, a case where the number of color identifications of the color-coded target CT is large will be described. In this case, there is no restriction on the order of shooting, and fully automatic processing is possible.

FIG. 17 shows a flow example of detection of a target with a color code. It is an example of the process of S14-S15 of FIG.
First, a color image (photographed image or model image) to be processed is read into the read image storage unit 141 of the extraction unit 41 (S500). Next, a target CT with a color code is extracted from each read image (S510).

  The search method is (1) a method for searching for a position detection pattern (retro target) P1 in a color-coded target CT, (2) a method for detecting chromatic dispersion of the color code portion P3, or (3) a coloration. There are various methods such as a method using a position detection pattern.

  (1) When a retro-target is included in the color-coded target CT, a pattern with a clear brightness difference is used. The retro target can be easily detected by binarizing this image.

  FIG. 18 is an explanatory diagram of the center-of-gravity position detection using a retro target. (A1) is a retro target in which the brightness of the inner circle portion 204 is bright and the brightness of the outer circle portion 206 is dark, and (A2) is the retro target in (A1). (B1) is a retro target in which the brightness of the inner circle portion 204 is dark and the brightness of the outer circle portion 206 is bright, and (B2) is a brightness distribution diagram in the diameter direction of the retro target of (B1). Show. When the lightness of the inner circle portion 204 is bright as shown in FIG. 18A1, the retro target has a bright portion with a large amount of reflected light at the position of the center of gravity in the captured image of the measurement object 1, and thus the light amount distribution of the image. As shown in FIG. 18A2, it is possible to obtain the inner circle portion 204 and the center position of the retro target from the threshold value To of the light amount distribution.

When the target existence range is determined, the barycentric position is calculated by, for example, the moment method. For example, the plane coordinates of the retro target 200 shown in FIG. 18 (A1) are (x, y). Then, (Expression 1) and (Expression 2) are calculated for points in the x and y directions where the brightness of the retro target 200 is greater than or equal to the threshold value To.
xg = {Σx * f (x, y)} / Σf (x, y) ---- (Equation 1)
yg = {Σy * f (x, y)} / Σf (x, y) ---- (Formula 2)
Here, (xg, yg) is the coordinates of the center of gravity position, and f (x, y) is the density value on the (x, y) coordinates.
In the case of the retro target 200 shown in FIG. 18B1, (Expression 1) and (Expression 2) are calculated for points in the x and y directions whose lightness is equal to or less than the threshold value To.
Thereby, the position of the center of gravity of the retro target 200 is obtained.

(2) Usually, the color code portion of the color-coded target CT uses a large number of code colors and has a feature that the color dispersion value is large. For this reason, the target CT with a color code can be detected by finding a portion having a large dispersion value from the image.
(3) The retro-targets at the three corners used in the color-coded target CT have different colors, and the colors reflected by the respective retro targets are made different. Since the retro targets at the three corners have different colors, it is easy to distinguish each retro target belonging to one color code target. In the retro target grouping process, even when a large number of retro targets are used, the process can be simplified by selecting the retro targets of different colors that are closest to each other as group candidates.

  When a large number of retro targets are used as the reference point RF, the retro target of the color code target CT and the single retro target are mixed, so that the retro target of the color code target CT is colored as a single retro target. If the target is white, it is easy to distinguish.

Here, the example (1) will be described. In FIG. 17, the retro target detection processing unit 111 stores the coordinates of a plurality of retro targets detected from the color image in the read image storage unit 141.
Next, the retro target grouping processing unit 120 determines from the retro target coordinates stored in the read image storage unit 141 that the retro target detected by the search processing unit 110 belongs to the same color coded target CT. (For example, those whose coordinates are in the color-coded target CT) are grouped as candidates belonging to the same group, and the combination of the candidates (one set of three) is stored as a group in the read image storage unit 141 (S520). The confirmation can be performed, for example, by measuring the distance between the three retro targets in the detected color-coded target CT and the apex angle of the triangle connecting the three retro targets (see S530).
Further, by comparing the detected pattern of the target CT with color code with the target table 142 with color code, it is confirmed which type of target with the color code.

  Next, the target detection processing unit with color code 130 stores the region of the target CT with color code from the center of gravity position of the retro target in units of retro targets stored in the read image storage unit 141 in the region direction detection processing unit 131. A direction is obtained (S530). Before or after obtaining the area and direction, the color detection processing unit 311 detects the color of the reference color part P2, the color code part P3, and the measurement object 1 in the image. If necessary, the color correction unit 312 corrects the color of the color code portion P3 and the measurement object 1 in the image based on the color of the reference color portion P2. In addition, when a target with a color code of a print color that is not a reference is used, the reference color portion is corrected together. Then, the confirmation processing unit 313 confirms whether the grouping is performed properly, that is, whether the center of gravity of the retro targets once grouped belongs to the same color code target CT. If it is determined that they belong to the same group, the process proceeds to the next identification code determination process (S535). If it is determined that they do not belong to the same group, the process returns to the grouping process (S520) again.

19 and 20 show an example of a processing flow of the target area direction detection processing unit 131 with a color code. The retro target code reading will be described with reference to FIGS. Here, a procedure for reading a code from the color-coded target CT1 in FIG.
In order to read the code from the color-coded target CT1, it is necessary to know the area and direction of the color-coded target CT1, and therefore, the center points of the three position detection retro targets are labeled R1, R2, and R3 (FIG. 21). (See (a)).

The labeling method creates a triangle that passes through the center-of-gravity points R1 to R3 of the three retro targets of interest (S600). Appropriately select one of the centroid points R1 to R3 of the three retro targets and temporarily label it with T1 (S610), and temporarily label the remaining two centroid points with T2 and T3 (S612, (Refer FIG.21 (b)).
Next, the sides passing through the barycentric points are labeled. The side passing through T1 and T2 is L12, the side passing through T2 and T3 is L23, and the side passing through T3 and T1 is L31 (S614, see FIG. 22A).

Next, the inside of the triangle is scanned in an arc shape for pixel values that are separated from each vertex (center of gravity) by a radius R, and the color changes within the scanned range (see FIG. 22B).
At the center of gravity T1, L12 to L31 are scanned clockwise, at the center of gravity T2, L23 to L12 are scanned clockwise, and at the center of gravity T3, L31 to L23 are scanned clockwise (S620 to S625).
The method of determining the radius is determined by multiplying the size of the retro target on the image according to the scanning angle. When the retro target is photographed from an oblique direction, it becomes an ellipse, so the scan range is also an ellipse. The magnification is determined by the size of the retro target and the distance between the retro target center of gravity position and the reference color portion P2.

The confirmation processing for labeling is performed by the confirmation processing unit 313. The barycentric point having the color change in the scan result is set as R1, and the remaining two barycentric points are labeled R2 and R3 by clockwise labeling from the barycentric point having the color change (S630 to S632). In this example, the center of gravity T2 is R1, the center of gravity T3 is R2, and the center of gravity T1 is R3. If one centroid point with a color change and two centroid points with no color change are not detected, a retro target combination error occurs (S633), and three new retro targets are selected. (S634), the process returns to S600. Thus, it can be confirmed from the processing results whether the three selected retro targets belong to the same color-coded target CT1. In this way, the retro target grouping is determined.
Although the above-described labeling method has been described using the color-coded target CT1 in FIG. 3A as an example, the same processing can be performed for other color-coded targets CT1 by changing a part of the processing.

[Identification of code]
Returning to FIG. 17, the identification code determination unit 46 sets the target CT1 with color code based on the barycentric positions of the retro targets grouped by the coordinate conversion processing unit 321 with respect to the target CT1 with color code extracted by the extraction unit 41. Then, the code conversion processing unit 322 identifies the color code (S535), performs code conversion to obtain the identification code of the target CT1 with the color code (S540), and reads the read image. The data is stored in the storage unit 141 (S545).

  This processing flow will be described with reference to FIG. The photographed image of the target with the color code that is distorted in shape, such as affixed to the curved surface or photographed from an oblique direction, is converted into a front view without distortion using the labels R1, R2, and R3 (S640). ). By converting the coordinates, the retro target portion P1, the reference color portion P2, the color code portion P3, and the white portion P4 can be easily identified with reference to the design value of the target with color code, and the subsequent processing can be easily performed.

  Next, it is checked whether or not there is a white portion P4 as designed on the coordinate-converted target CT1 with color code (S650). If it is not as designed, a detection error occurs (S633). If the white portion P4 is present as designed, it is determined that the color-coded target CT1 has been detected (S655).

Next, the color code of the color-coded target CT1 whose color and color correction have been performed and whose region and direction are known is determined.
The color code part P3 expresses a code by a combination of color schemes for each unit area. For example, when the number of code colors is n and the number of unit areas is three, a code of n × n × n can be expressed, and when the condition that the colors used in other unit areas are not used redundantly is imposed, A code of n × (n−1) × (n−2) can be expressed. When the number of code colors is n, the number of unit areas is n, and the condition that the colors are not used redundantly is imposed, a code corresponding to the factorial of n can be expressed.
In the code conversion processing unit 322, the identification code determination unit 46 determines the identification code by comparing / collating the color combination of the unit area in the color code unit P 3 with the color combination of the color code target correspondence table 142.

  Color discrimination methods include (1) a relative comparison method in which the color of the reference color portion P2 and the color of the color code portion P3 are compared, and (2) the color of the reference color portion P2 and the color of the white portion P4. There are two methods: an absolute comparison method in which color correction of the color-coded target CT1 is performed and the code of the color code portion P3 is discriminated with the corrected color. For example, when the number of colors used for the color code portion P3 is small, the reference color is used as a comparative color for relative comparison, and when the number of colors used for the color code portion P3 is large, calibration is performed to correct the colors. Used as a comparative color for absolute comparison. As described above, color detection is performed by the color detection processing unit 311, and color correction is performed by the color correction unit 312.

In the code conversion processing unit 322, the identification code determination unit 46 detects the reference color part P2 and the color code part P3 using either the method (1) or (2) (S660, S670), and the color code. Each color of the part P3 is discriminated (S535 in FIG. 17), and the color is converted into a code to obtain the identification code of the target CT1 with the color code (S680, S540 in FIG. 17).
For each image, the number of the color-coded target CT1 included in the image is registered in the pattern information storage unit 47 (S545 in FIG. 17). The data registered in the pattern information storage unit 47 is used for orientation and three-dimensional measurement to improve efficiency.

[Pattern projection method for 3D measurement]
FIG. 23 shows an example of a processing flow of the method for projecting a three-dimensional measurement pattern (preparation before measurement) in the present embodiment. In the present embodiment, the projector 12 projects a first measurement pattern (preliminary measurement pattern or the like), and forms and projects a second measurement pattern (precision measurement pattern or the like) based on the deformation of the projected pattern. To do.
First, a plurality of measurement patterns for displaying measurement points on the surface of the measurement object are stored in the pattern storage unit 495 (pattern storage step: S710). Next, the pattern projection control unit 493 causes the projection unit 12 to project one of the plurality of measurement patterns as a first measurement pattern (first projection step: S720). Next, the imaging unit 10 captures the first measurement pattern projected in the projection process (imaging process: S730). Next, the pattern detection unit 491 detects measurement points from the captured image of the first measurement pattern captured by the imaging process (pattern detection process: S740). Next, the pattern detection unit 491 detects the displacement of the measurement point in the first measurement pattern detected in the pattern detection process (displacement detection process: S750). Next, the pattern forming unit 492 forms a second measurement pattern in which measurement points are added, deleted, or changed based on the detected displacement of the measurement points in the first measurement pattern (pattern formation step: S760). Next, the pattern projection control unit 493 causes the projection unit 12 to project the second measurement pattern (second projection step: S770).
In this way, by utilizing the projection device, pre-measurement preparation is performed, and the target pattern used for orientation or three-dimensional measurement is reconfigured to optimize and automate non-contact three-dimensional measurement for various objects. be able to.

FIG. 24 shows another processing flow example of the method for projecting a three-dimensional measurement pattern (projecting a target with a color code) in the present embodiment. In the present embodiment, a measurement pattern including a sign with a color code is formed and projected by the projector 12.
First, in the pattern forming unit 492, a measurement pattern including a color code marker CT having a position detection pattern P1 for indicating a measurement position and a color code pattern P3 on which a plurality of colors for identifying the marker is applied. It forms (pattern formation process: S810). Next, the pattern projection control unit 493 projects the measurement pattern formed in the pattern formation process onto the projection unit 12 (projection process: S840). Next, the imaging unit 10 captures the measurement pattern projected by the projection process (imaging process: S850). Next, the pattern detection unit 492 detects the position detection pattern P1 and the color code pattern P3 from the captured image of the measurement pattern captured in the imaging process, and identifies the color code (pattern detection process: S860).

  In this way, each target can be easily identified by using a target with a color code, and orientation and three-dimensional measurement can be made efficient and automated by automatically connecting an image of a measurement object in a wide area. .

[Second Embodiment]
In the first embodiment, the example in which the measurement pattern is formed by the pattern forming unit has been described. However, a large number of measurement patterns are stored in the pattern storage unit, and the most appropriate measurement pattern is selected according to the conditions. An example of selecting and projecting by the pattern selection unit will be described. In addition, the measurement pattern formed by the pattern forming unit can be stored in the pattern storage unit.

  FIG. 25 shows a configuration example of the three-dimensional measurement pattern projection apparatus 80A in the present embodiment, and FIG. 26 shows a block diagram of an overall configuration example of the three-dimensional measurement system 100A in the present embodiment. As compared with the first embodiment, a pattern storage unit 495 and a pattern selection unit 496 are added in the calculation processing unit 49. The pattern storage unit 495 stores various patterns such as a large number of measurement patterns, random patterns, overlapping shooting range display patterns, and texture illumination patterns. Various patterns formed by the pattern forming unit 492 are also stored. The pattern selection unit 496 appropriately selects a measurement pattern to be projected from the measurement patterns stored in the pattern storage unit 495. The pattern projection control unit 493 controls the projection unit 12 to project the measurement pattern selected by the pattern selection unit 496. When the pattern detection unit 491 receives notification from the pattern detection unit 491 that the displacement of the measurement point in the first measurement pattern is large, the pattern selection unit 496 uses the measurement pattern stored in the pattern storage unit 495 for position measurement. A pattern with many patterns is selected and selected as the third measurement pattern. The pattern projection control unit 493 causes the projection unit 12 to project the third measurement pattern selected by the pattern selection unit 496. If an appropriate pattern is not found in the pattern storage unit 495, the pattern forming unit 492 forms a new second measurement pattern by adding measurement points to the displacement portion based on the first measurement pattern. In this case, the pattern projection control unit 493 causes the projection unit 12 to project the second measurement pattern formed by the pattern forming unit 492.

  Further, the pattern storage unit 495 stores a measurement pattern including a target CT with a color code and a monochromatic target pattern. These arrangements and coloring are various. The pattern selection unit 496 appropriately selects a measurement pattern to be projected from a wide variety of measurement patterns stored in the pattern storage unit 495. The pattern projection control unit 493 causes the projection unit 12 to project the measurement pattern selected by the pattern selection unit 496. The pattern storage unit 495 stores pattern elements such as a target with a color code and a monochromatic target pattern, and the pattern forming unit 492 can edit or form a pattern using these elements. The measurement pattern formed by the pattern forming unit is stored in the pattern storage unit 495, and the pattern projection control unit 493 can cause the projection unit 12 to project the measurement pattern formed by the pattern forming unit 492.

FIG. 27 shows an example of a process flow of the method for projecting a three-dimensional measurement pattern (preparation before measurement) in the present embodiment. In the present embodiment, the projector 12 projects a first measurement pattern (preliminary measurement pattern, etc.), and selects and projects a third measurement pattern (precision measurement pattern, etc.) based on the deformation of the projected pattern. To do.
First, a plurality of measurement patterns for displaying measurement points on the surface of the measurement object are stored in the pattern storage unit 495 (pattern storage step: S710). Next, the pattern projection control unit 493 causes the projection unit 12 to project one of the plurality of measurement patterns as a first measurement pattern (first projection step: S720). Next, the imaging unit 10 captures the first measurement pattern projected in the projection process (imaging process: S730). Next, the pattern detection unit 491 detects measurement points from the captured image of the first measurement pattern captured by the imaging process (pattern detection process: S740). Next, the pattern detection unit 491 detects the displacement of the measurement point in the first measurement pattern detected in the pattern detection process (displacement detection process: S750). Next, the pattern forming unit 492 adds, deletes, or changes a measurement point from the measurement pattern stored in the pattern storage unit 495 based on the detected displacement of the measurement point in the first measurement pattern. Is selected (pattern selection step: S780). Next, the pattern projection control unit 493 causes the projection unit 12 to project the third measurement pattern (third projection step: S790).

FIG. 28 shows a processing flow example of another projection method (projecting a target with a color code) of the three-dimensional measurement pattern in the present embodiment. In the present embodiment, a measurement pattern including a sign with a color code is selected and projected by the projector 12.
First, in the pattern forming unit 492, a measurement pattern including a color code marker CT having a position detection pattern P1 for indicating a measurement position and a color code pattern P3 on which a plurality of colors for identifying the marker is applied. A plurality of patterns are stored (pattern storing step: S820). Next, the pattern selection unit 496 selects a measurement pattern to be projected from the plurality of measurement patterns stored in the pattern storage process (pattern selection process: S830). Next, the pattern projection control unit 493 projects the measurement pattern selected in the pattern selection step onto the projection unit 12 (projection step: S840). Next, the imaging unit 10 captures the measurement pattern projected by the projection process (imaging process: S850). Next, the pattern detection unit 492 detects the position detection pattern P1 and the color code pattern P3 from the captured image of the measurement pattern captured in the imaging process, and identifies the color code (pattern detection process: S860).

[Third Embodiment]
This embodiment shows an example in which only a normal reference point (retro target or template) is used without using a target with a color code. These reference points consist only of position detection patterns, and normally a black and white retro target as shown in FIG. 18 is used, but a monochromatic target pattern may also be used. The pattern formation unit 492 forms various projection patterns including only the reference points and includes the measurement pattern, the pattern storage unit 495 stores these various projection patterns, and the pattern selection unit 496 displays the patterns projected from these various projection patterns. Then, the pattern projection control unit 493 controls the projection unit 12 to project these various projection patterns, and the pattern detection unit 491 detects reference points from the captured images of these projection patterns. Even if a retro target or template is used, the center of gravity can be detected and can be made to correspond as a reference point and a corresponding point in a stereo image, so that not only preparation before measurement but also orientation and three-dimensional measurement are possible. It is also possible to perform the entire processing flow of FIG.
As the system configuration of the present embodiment, the configuration of the extraction unit in FIG. 4 is only the search processing unit 110 as compared with the first and second embodiments, which can be simplified, and the identification code determination unit. 46 can be omitted. Note that when a target with a color code is not used, it is preferable to capture a captured image including a plurality of points with clear coordinates of the measurement object using a wide-angle projector.

[Fourth Embodiment]
In the present embodiment, a method of performing rough measurement or measurement until the pre-measurement preparation (S255) will be described. The system configuration of this embodiment is the same as that of the first or second embodiment.
FIG. 29 shows a processing flow of rough measurement. First, a measurement preparation pattern is projected from the projection unit 12 onto the measurement object 1 using the projection mode as a measurement mode, the projection pattern is photographed by the imaging unit 10, and the photographed image is extracted to the extraction unit 41 (pattern detection unit 491). Transmit (S311). Next, the gravity center position of the measurement point on the captured image is detected by the extraction unit 41 (pattern detection unit 491) (S312). Next, the barycentric position of the measurement point detected in one of the stereo images (reference image) is set as a reference point by the reference point setting unit 42, and the corresponding point corresponding to the reference point in the other stereo image (search image) is set. Obtain (S313). Next, the orientation calculation is performed by the orientation unit 44 (S314). Orientation is possible if there are 6 or more pairs of reference points and corresponding points. If there is a point with poor accuracy from the orientation result (No in S315a), the point is removed (S315), a pair of 6 or more points is selected, and the orientation calculation is performed again (S314). S314 and S315 are repeated until there are no bad points. If there is no bad point (Yes in S315a), the correspondence between the camera image and the model image is determined by orientation, and the point from which the bad point is removed is registered in the pattern information storage unit 47 as a reference point RF. Next, the one near the deformation point is extracted from the registered reference point RF, and the reference point RF is pasted at a position corresponding to the deformation point of the measuring object 1, or the reference point RF is added to the measurement preparation pattern. Thus, a new measurement pattern P is formed (S316).

  If the number of measurement points (including the orientation points) in the measurement preparation pattern is sufficiently large, the measurement process can be completed using the projected orientation points as measurement points. In this case, the processing after S260 in FIG. 14 is not necessary.

If more strictness is required, the approximate surface measurement is performed. FIG. 30 shows an example of a schematic surface measurement processing flow. When the measurement pattern with the added reference point is projected, for example, the measurement mode is used. First, the outermost periphery of the reference point is set as a measurement area (S317). Next, stereo matching is performed (S318). Next, the mismatched area is displayed (S319). Next, a new measurement pattern is formed by pasting a reference point at a point of mismatching or adding a reference point to the measurement pattern (S319a).
Here, if the tie point (for connection: color coded target) is pasted first, the measurement flow in this area is completed instead of the pre-measurement process by repeating the pre-measurement preparation process flow (S300 to S320). It is also possible to make it.

[Fifth Embodiment]
In the present embodiment, the number of pattern projections is increased. The system configuration of this embodiment is the same as that of the first or third embodiment. For example, orientation point projection may be performed a plurality of times, and measurement point projection may be performed a plurality of times. When mismatching occurs due to measurement, the orientation points may be increased and the orientation points projected again, and further measurement may be performed.

  Further, the present invention can also be realized as a computer-readable program for causing a computer to execute the three-dimensional rule pattern projection method or the three-dimensional rule method described in the above embodiments. The program may be stored and used in a built-in memory of the calculation processing unit 49, may be stored and used in a storage device inside or outside the system, or may be downloaded from the Internet and used. Moreover, it is realizable also as a recording medium which recorded the said program.

  Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and it is obvious that various modifications can be made to the embodiment without departing from the spirit of the present invention. It is. For example, in the above embodiment, the example in which the measurement points are increased in forming the second measurement pattern has been mainly described. However, the measurement points can be decreased or changed. Regarding the sign with the color code, configurations other than those shown in FIG. 3 are possible, such as increasing the number of unit areas of the color code, changing the position of the reference code part, enlarging the retro target part, and writing alphanumeric characters in the white part. . In addition, regarding the measurement pattern, even if a marker with color code is arranged at four corners in a rectangular projection range and a single-color target pattern is arranged inside the measurement pattern, the arrangement of the single-color target can be varied, and the inside thereof can be varied. A sign with a color code may be arranged.

  In addition, for a series of captured images, each captured image is captured so as to include four color-coded labels CT, and between two adjacent captured images, two color-coded markers CT are shared, A sequence of captured images may be automatically determined so that the identification codes of the color-coded markers CT shared by adjacent captured images match. Further, the stereo camera, the projector, and the calculation processing unit may be configured integrally or may be configured separately. In the above embodiment, the pattern detection unit of the calculation processing unit is shared with the extraction unit, the reference point setting unit, the corresponding point search unit, and the like in the corresponding unit. However, a separate configuration may be used.

  The present invention is used in a system and method for three-dimensional measurement of an object without contact.

It is a block diagram of the example of basic composition of the projection device in a 1st embodiment. It is a block diagram of the example of whole composition of the three-dimensional measuring system in a 1st embodiment. It is a figure which shows the example of a target with a color code. It is a figure which shows the structural example of an extraction part and an identification code discrimination | determination part. It is a figure which shows the example of a processing flow of the three-dimensional measurement system in 1st Embodiment. It is a figure which shows the example of overlap imaging | photography. It is a figure which shows the example of the picked-up image image | photographed with a stereo camera. It is a figure which shows the example of a flow of a stereo pair selection. It is explanatory drawing of the model image coordinate system XYZ and camera coordinate system xyz in a stereo image. It is a figure which shows the example of the target which has a reference point. It is a figure which shows the example of a flow of the automatic matching of a reference point. It is a figure which shows the example of a processing flow of the automatic determination of a stereo matching area. It is a figure for demonstrating the automatic determination of a stereo matching area. It is a figure which shows the example of the measurement flow using a projection apparatus. It is a figure which shows the example of a processing flow of preparation before a measurement. It is a figure which shows the example of the pattern for a measurement preparation. It is a figure which shows the example of a flow of a detection of the target with a color code. It is explanatory drawing of the gravity center position detection using a retro target. It is a figure which shows the example of a processing flow of the target area direction detection process part with a color code. It is a figure which shows the process flow example (continuation) of the target area | region direction detection process part with a color code. It is FIG. (1) for demonstrating the code reading of a retro target. It is FIG. (2) for demonstrating the code reading of a retro target. It is a figure which shows the example of a processing flow of the projection method (preparation before measurement) of the pattern for three-dimensional measurement in 1st Embodiment. It is a figure which shows the example of a processing flow of another projection method (projecting a target with a color code) of the pattern for three-dimensional measurement in 1st Embodiment. It is a figure which shows the structural example of the projection apparatus for three-dimensional measurement in 2nd Embodiment. It is a block diagram of the example of whole structure of the three-dimensional measurement system in 2nd Embodiment. It is a figure which shows the example of a processing flow of the projection method (preparation before measurement) of the pattern for three-dimensional measurement in 2nd Embodiment. It is a figure which shows the example of a processing flow of another projection method (projecting a target with a color code) of the pattern for three-dimensional measurement in 2nd Embodiment. It is a figure which shows the processing flow of the rough measurement in 4th Embodiment. It is a figure which shows the processing flow of the schematic surface measurement in 4th Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Measurement object 10 Imaging part 12 Projection part 13 Captured image data storage part 40 Corresponding part 41 Extraction part 42 Reference point setting part 43 Corresponding point search part 44 Orientation part 45 Corresponding point instruction | indication part 46 Identification code discrimination | determination part 47 Pattern information storage part 48 shooting / model image display unit 48A model image formation unit 48B model image storage unit 49 calculation processing unit 50 display image formation unit 51 three-dimensional coordinate data calculation unit 53 three-dimensional coordinate data storage unit 54 three-dimensional two-dimensional image formation unit 55 three-dimensional Two-dimensional image storage unit 57 three-dimensional two-dimensional image display unit 60 display device 70 matching processing unit 80, 80A three-dimensional measurement projection device 100, 100A three-dimensional measurement system 110 search processing unit 111 retro target detection processing unit 120 retro target Grouping processing unit 130 Color code target detection processing unit 131 Color Coded target area direction detection processing unit 140 Image / color pattern storage unit 141 Read image storage unit 142 Color code target correspondence table 200 Retro target 204 Inner circle 206 Outer circle 311 Color detection processing unit 312 Color correction unit 313 Confirmation processing unit 321 Coordinate conversion processing unit 322 Code conversion processing unit 491 Pattern detection unit 492 Pattern formation unit 493 Pattern projection control unit 494 Color correction unit 495 Pattern storage unit 496 Pattern selection unit CT, CT1-CT3 Target EP with color code Epipolar lines L12, L23, L31 Side P Measurement pattern P1 Position detection pattern (retro target part)
P2 standard color pattern (standard color part)
P3 Color code pattern (color code part)
P4 empty pattern (white part)
Q Measurement points R1 to R3 Center of gravity RF Reference point To Threshold T1 to T3 Temporary label

Claims (17)

With a color code having a position detection pattern for indicating a measurement position and a color code pattern arranged in a predetermined positional relationship with the position detection pattern and provided with a plurality of colors for identifying signs A pattern forming unit for forming a measurement pattern including a sign;
A projection unit that projects the measurement pattern formed by the pattern formation unit onto a measurement object;
A pattern detection unit that detects the position detection pattern and the color code pattern from a photographed image of the measurement pattern projected by the projection unit, and identifies a color code;
Projector for 3D measurement. With a color code having a position detection pattern for indicating a measurement position and a color code pattern arranged in a predetermined positional relationship with the position detection pattern and provided with a plurality of colors for identifying signs A pattern storage unit for storing a plurality of measurement patterns including a sign;
A pattern selection unit for selecting a measurement pattern to be projected from a plurality of measurement patterns stored in the pattern storage unit;
A projection unit that projects the measurement pattern selected by the pattern selection unit onto a measurement object;
A pattern detection unit that detects the position detection pattern and the color code pattern from a photographed image of the measurement pattern projected by the projection unit, and identifies a color code;
Projector for 3D measurement. An imaging unit that captures the measurement pattern projected by the projection unit;
The pattern detection unit detects the position detection pattern and the color code pattern from the captured image of the measurement pattern captured by the imaging unit, and identifies the color code;
The projection apparatus for three-dimensional measurement according to claim 1 or 2. The pattern forming unit forms a monochromatic target pattern including only the position detection pattern;
The projection apparatus for three-dimensional measurement according to claim 1 or 3. The pattern storage unit stores a measurement pattern including a monochromatic target pattern including only the position detection pattern;
The projection apparatus for three-dimensional measurement according to claim 2 or claim 3. A pattern projection control unit that controls the projection unit to project the measurement pattern;
The pattern projection control unit can project a random pattern in which the position detection patterns are randomly arranged on the projection unit, and switch between a measurement mode for projecting the measurement pattern and a random pattern mode for projecting the random pattern;
The three-dimensional measurement projector according to any one of claims 1 to 5. A pattern projection control unit that controls the projection unit to project the measurement pattern;
The pattern projection control unit projects an overlapping shooting range display pattern for displaying an overlapping range of stereo images onto the projecting unit, and projects a measurement mode for projecting the measurement pattern and a shooting range display for projecting the overlapping shooting range display pattern. The mode can be switched;
The three-dimensional measurement projector according to any one of claims 1 to 5. A pattern projection control unit that controls the projection unit to project the measurement pattern;
The pattern projection control unit can adjust the arrangement of the orientation points of the measurement pattern and the pattern density by inputting any one of a focal length, an imaging distance, a base length, and an overlap rate related to the imaging unit. ;
The three-dimensional measurement projector according to any one of claims 1 to 5. A pattern projection control unit that controls the projection unit to project the measurement pattern;
The pattern projection control unit causes the projection unit to perform illumination for texture acquisition that illuminates an object with uniform illumination light, and performs a measurement mode for projecting the measurement pattern and a texture illumination mode for performing illumination for texture acquisition. Can be switched;
The three-dimensional measurement projector according to any one of claims 1 to 5. The pattern detection unit includes a color correction unit that corrects a color of a target with a color code projected by the projection unit based on a color acquired from a captured image of the pattern projected in the texture illumination mode;
The projection device for three-dimensional measurement according to claim 9. It has the projection device for three-dimensional measurement of any one of Claim 1 thru | or 10.
3D measurement system. With a color code having a position detection pattern for indicating a measurement position and a color code pattern arranged in a predetermined positional relationship with the position detection pattern and provided with a plurality of colors for identifying signs A pattern forming unit for forming a measurement pattern including a sign;
A pattern projection control unit for controlling the projection unit to project the measurement pattern;
A pattern detection unit that detects the position detection pattern and the color code pattern from a captured image of the measurement pattern projected by the projection unit, and identifies a color code;
Calculation processing unit of the projection device for 3D measurement. With a color code having a position detection pattern for indicating a measurement position and a color code pattern arranged in a predetermined positional relationship with the position detection pattern and provided with a plurality of colors for identifying signs A pattern forming process for forming a measurement pattern including a sign; and
A projection step of projecting the measurement pattern formed in the pattern formation step onto a measurement object;
An imaging step of photographing the measurement pattern projected by the projection step;
A pattern detection step of detecting the position detection pattern and the color code pattern from a photographed image of the measurement pattern photographed in the imaging step and identifying the color code;
Pattern projection method for 3D measurement. With a color code having a position detection pattern for indicating a measurement position and a color code pattern arranged in a predetermined positional relationship with the position detection pattern and provided with a plurality of colors for identifying signs A pattern storage step for storing a plurality of measurement patterns including a sign;
A pattern selection step of selecting a measurement pattern to be projected from a plurality of measurement patterns stored in the pattern storage step;
A projection step of projecting the measurement pattern selected in the pattern selection step onto the measurement object;
An imaging step of photographing the measurement pattern projected by the projection step;
A pattern detection step of detecting the position detection pattern and the color code pattern from a photographed image of the measurement pattern photographed in the imaging step and identifying the color code;
Pattern projection method for 3D measurement. The pattern forming step forms a monochromatic target pattern consisting only of the position detection pattern,
The pattern detection step detects the monochrome target pattern;
The three-dimensional measurement pattern projection method according to claim 13. The pattern storage step stores a measurement pattern including a monochromatic target pattern consisting only of the position detection pattern,
The pattern detection step detects the monochrome target pattern;
The pattern projection method for three-dimensional measurement according to claim 14. In the imaging step, the captured image is a paired stereo image,
An orientation process for orientation of the stereo image;
A three-dimensional measurement step for measuring a three-dimensional shape of the measurement object,
In the orientation step or the three-dimensional measurement step, the marker with color code is projected as a measurement point indicating a reference position for measurement, and the monochromatic target pattern is projected as a reference point;
The pattern projection method for three-dimensional measurement according to claim 15 or 16.