JP2565885B2 - Spatial pattern coding method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION (1) Technical Field of the Invention The present invention relates to a pattern projection method for measuring a three-dimensional shape of an object surface using a TV camera, in which measurement density, reliability,
The present invention relates to a spatial pattern coding method for enhancing simplicity.

(2) Conventional technology As a method of three-dimensionally measuring the surface shape of an object using a TV camera, the spot projection method, slit projection method,
And pattern projection. Among them, the pattern projection method can measure a large number of points with only one projection,
It is famous as a method that can reduce the measurement time.

This method uniquely associates the original pattern image with the projected pattern image, and calculates the three-dimensional distance distribution of the curved surface from the result based on the triangulation principle. It has the advantage of using only one image. Therefore, it is necessary to devise a method of matching.

In order to make a unique correspondence, it is indispensable to use a structured projection pattern and characterize the inside of the pattern to identify each part in the pattern. This characterization is called spatial pattern coding or spatial coding. Hereinafter, for simplicity, it will be abbreviated as coding.

This coding is further divided into pattern shape coding and gradation coding. As the former pattern shape coding, a method using a slit aperture width distribution and a method using an M-sequence code have been proposed, but they have problems in measurement density and measurement stability, and are not practical. The present application belongs to the latter gradation coding.

Heretofore, grayscale coding has been considered to be based on grayscale and color grayscale. Among them, the one based on gray scale is a coding that enables unique correspondence between a pattern original image and a pattern image by projecting a spatial pattern having many gray scales. This is an excellent method in principle. However, in an actual image, it is difficult to correctly identify many gradations due to noise in the image, variations in the reflectance of the object surface, blurring, etc. Usually, there are 3 gradations, and at most 4 or 5 at most. It is said that the degree of smoothness is the limit of discrimination.

There is a similar problem in the method using the color gradation instead of the gradation, and many gradations are used unless the target is an ideal non-measurement surface such as a white or light-colored uniform surface. Can not do it.

As described above, the gradation coding method has a problem that the number of distinguishable gradations is small.

As a solution to the problem of gradation coding, a method has been proposed in which one slit has three color distributions in the vertical direction and the slit number is identified using the color distribution in the entire slit length direction. However, this method is based on the assumption that the slits are continuous and can be observed for a sufficiently long time. If the slits are cut in the middle or there is an error in color identification, the pattern original image and the pattern image should be matched. There were times when it was not possible to set it up uniquely. Therefore, the measurement was limited to a smooth and sufficiently wide surface.

Time-series coding is a method that attempts to cover the above-mentioned drawbacks of pattern coding with the advantages of the slit projection method. In this method, instead of projecting multiple slits at once, a combination of slits to be projected at each time point is prepared in advance and sequentially projected in time series, and for each slit, the on-off at each time point is written in a code. It is a method, belongs to a broad pattern projection method, and is also called a time-series pattern projection method. This method is easy to identify the slit number, but loses the original advantage of the pattern projection method in that many image inputs are required. That is, since the pattern is projected many times, a moving object cannot be measured, and the image processing must be performed on many images, which increases the load on the computer. However, it has disadvantages such as long processing time and large-scaled light projection system.

As described above, it has been impossible in the past to uniquely associate a pattern original image with a pattern image by only one projection and measure a three-dimensional shape of a surface with high density.

(3) Object of the Invention The present invention provides a spatial pattern coding method for obtaining a high measurement density, high measurement stability, and reliability with a single projection in the pattern projection method.

(4) Configuration of the Invention (4-1) Difference between Features of Invention and Prior Art The present invention is characterized by using a lattice plate pattern structure composed of three or more kinds of gradations. In the past, in order to characterize a pattern, it was necessary to prepare a large number of pattern shapes, shades of gray, and color tones, or to project a plurality of slits many times in a time series. The difference is that it provides a pattern structure that creates many feature codes by projecting a pattern of at least three gradations once.

(4-2) Example FIG. 1 is a schematic diagram for explaining the pattern structure of the present invention.

The original pattern image has three shades of gray, black and white, and in the figure, white is represented by 2, black is represented by 0, and gray is represented by 1. When using the gradation of three RGB colors, for example, R corresponds to 2, G corresponds to 1, and B corresponds to 0.

The gradations are arranged so that they are not adjacent except for the corners.

Paying attention to P ₀ in the figure, P ₀ is the corner part of each gradation area and is also the intersection of the edges, and the gradation area of 2,1,0,1 is clockwise from the upper left area. being surrounded. Therefore, a corner portion or an edge intersection of a gradation area like P ₀ is used as a characteristic point, and in particular, it is named a node, and the main node of that node is represented in a ternary 4-digit form using the surrounding gradation. The main code of P ₀ is (2101). Similarly, P _{1 to} P ₄ have codes of (1012), (1020), (1012), and (0212), respectively. There are 18 such codes. This means
This is equivalent to projecting a dot pattern with 18 gradations.

Then give with main code closest node P ₁ to P ₄ of the node P ₀ as supplementary code of the node P _0. In other words, the auxiliary code of P ₀ is 16 digits of ternary (1012,1020,0212). Therefore, if the characteristic code of the node P ₀ is expressed by the main code and the auxiliary code,
It will be 20 decimal digits.

In order to avoid the complexity of increasing the number of digits, the type of code is
Since there are 18 types, if you assign a code number according to the combination of 0, 1, and 2 every 4 digits, the feature code of the node can be expressed by 5 digits in 18 digits. In this way, 1458 kinds of feature codes can be created.
That is, it is equivalent to projecting a 1458 gradation pattern.

Since the pattern structure described above is composed of three values and arranged in a lattice plate shape, it will be referred to as a ternary lattice plate pattern. It is self-evident that the number of equivalent gray scales is increased if the number of gray scales is not limited to three and is four or more, and that the number of equivalent gray scales is further increased if the number of closest nodes is not limited to four and is eight, for example.
Also, the edge does not necessarily have to be a straight line.

This pattern can be easily created by generating random numbers under the condition that the gradations of adjacent grid plates are different from each other. In the example, a 100 × 100 ternary lattice plate pattern was generated by a minicomputer VA × 11/780, displayed on a display device, and a pattern original plate was prepared by a video printer.

FIG. 2 is a first embodiment, and is a partial copy of an example of a projected pattern image when a pattern original image is projected on an object by a slide projector. A CCD camera is used as the camera, and the distance between the camera and the object is approximately 1 m, and the distance between the slide projector and the object is approximately 1.5 m. The image consists of three gradations, and only the edges of each area of the grid plate are shown in the figure.

FIG. 3 is a flowchart showing the flow of overall processing. At 1, the discrimination threshold value for ternarization is obtained, and at 2, each pixel is rewritten with a value of 0, 1 or 2 depending on the gradation.

In 3, the corners of each lattice plate area, that is, the edge intersections are extracted as nodes.

In No. 4, depending on the gradation distribution surrounding the extracted node, 18
One of the code numbers of the types is given as the main code.

In 5, the four main code numbers of the four closest nodes are given as auxiliary codes. However, when a part of the adjacent node cannot be obtained, 0 is added to the code number.

At 6, the result of node identification and code number is recorded in the table.

FIG. 4 is a flowchart showing the flow of the process of associating the pattern original image with the pattern image. The two tables for the pattern original image and the pattern image in which the node and the code identification result in FIG. 3 are recorded are used. For simplicity, the table of the pattern original image will be referred to as table A, and the table of the pattern image will be referred to as table B.

In step 7, for each node in table A, the main code of each node in table B is checked to find a node having the same main code.

Next, in 8, the distance between the node and the projective line determined by the geometrical relationship between the pattern master and the camera is calculated. It is obvious from projective geometry that the correct corresponding points are on this projective line. If the calculated distance is within a predetermined distance (usually 3 pixels), check with 9, and if it is OK, compare the auxiliary code at 10, and if 11 does not include code number 0 and match. Is determined as the corresponding node in 12. If it cannot be determined here, if the auxiliary codes except for 0 match at 13, the corresponding node is registered as a candidate at 14. Same table B
If other nodes in the list are also examined and there are multiple corresponding node candidates that cannot be determined as corresponding nodes,
A node having a large number of matching auxiliary codes is set as a corresponding node.

The above processing is performed for each node in table A. In the figure, it is determined that the node is not a corresponding node and the next corresponding node candidate is searched.

FIG. 5 shows the arrangement of the camera and the light projecting system in the second embodiment. Reference numeral 21 is a pattern projector for projecting a ternary lattice plate pattern, 22, 23 are cameras, and 24 is an object to be measured.

In the above-mentioned first embodiment (FIGS. 2 to 4), the pattern image of one camera is associated with the pattern original image, but in this second embodiment, two patterns obtained from two cameras are used. Nodes are associated between the pattern images. The associating process procedure is the same as in the first embodiment.

FIG. 6 shows a third embodiment using three cameras.
25 is an additional camera, and the three cameras are arranged in three corners. The nodes are associated between the three images based on the principle of three-eye stereoscopic vision.

First, a candidate for the corresponding node is selected between the two images, and the correctness of the corresponding candidate is determined using the third image.

In this method, the feature code can be only the main code, and there is an advantage that the trouble of finding the auxiliary code can be saved. Also, if there is a lot of noise in the image, if there are visible and hidden,
This is especially effective under measurement conditions in which many feature code reading errors are likely to occur, such as when there is highlights or when the object surface is unevenly soiled.

FIG. 7 shows a detailed configuration of an embodiment of the surface shape measuring apparatus by pattern projection. Reference numeral 21 is a pattern projector, which uses a commercially available slide projector. 32 is a ternary lattice plate pattern original plate, 33 is a projection lens, 34 is a lamp, 22, 23 are cameras, 24 is an object to be measured, 35 is an image processing device, 36 is a frame memory, 37 is an image ternarization processing circuit, 38 is a node detection circuit, 39 is a feature code identification circuit, and 40 is a table. Reference numeral 41 is a camera position / azimuth calibration data memory, which is calibrated and recorded in advance before measurement. Also, 4
Reference numeral 2 is a corresponding processing device, 43 is a three-dimensional position calculation circuit, and 44 is a stereoscopic display device.

Gradation can be created by using shades and colors.
The number of gradations may be four or more and the types of feature codes may be increased. Each lattice plate shape of the pattern does not necessarily have to be a square, and may be a rectangle or another polygon. Moreover, the edge of each lattice plate may not necessarily be a straight line but may be a curved line.

Example of measurement data A ternary grid plate pattern was generated using random numbers on a VA × 11/780 minicomputer, displayed on a monitor, and then the monitor image was taken as a slide. This slide pattern was projected on the measurement target using a commercially available cabin slide projector.

The object of measurement is an iron white semi-glossy corrugated plate with a size of about 20 cm x 20 cm, and the corrugated shape has a depth of 3-5 mmp-p (heterogeneous distribution) of peaks and valleys and a cycle of about 30 mm.

On the other hand, place two Sony CCD cameras about 20 cm apart,
Fixed The lens used has a focal length of 16 mm, which is commonly used in TV cameras.

A reference mark plate, which serves as a reference for the spatial coordinate system, was placed approximately 80 cm in front of the camera, and the positions and orientations of the two cameras in the spatial coordinate system were calibrated from the camera image. This calibration method has already been disclosed in Japanese Patent Application No. 60-118756 (Japanese Patent Application Laid-Open No. 61-277012) "Camera position / orientation calibration method and its apparatus". The reference mark plate is disclosed in Japanese Patent Application No. 60-118755 (Japanese Patent Laid-Open No. 61-277011) "Camera position / posture calibration mark data collection method".

FIG. 8 shows an example of the projected pattern image on the corrugated plate described above. After the surface light reflectance correction is applied to this pattern image, the node and the feature code that it has are detected,
Next, the feature codes were used to associate the images, and the three-dimensional position was calculated.

9A and 9B are projection images showing the processing results, where FIG. 9A is a front view and FIG. 9B is an oblique view. The nodes and adjacency relationships are obtained from the identification node numbers, and the measurement node points are directly connected and displayed. Except for some missing measurement points, the correspondence between the images is correct and the positions are determined.

The measurement accuracy is within the maximum absolute accuracy of 0.1 mm, the average variation error within 0.5 mm, and the maximum variation error of 2 mm in the reference coordinate system, that is, the spatial coordinate system. Some of the measurement points were missed because the node position could not be read correctly or the feature code was erroneously read due to a defect in the pattern master plate. If you pay attention to creating the pattern master plate in the future, it will be solved. To be done.

As shown above, the correspondence between nodes between pattern images is 100% correct, and the reliability is high. In addition, the lack of correspondence may occur when there is a strong highlight due to the defect of the pattern original plate or the gloss of the surface, but this does not cause an error in correspondence of other nodes. In this way, the advantages of this method were also confirmed by experiments.

(5) Effects of the Invention As described above, in the pattern coding method of the present application, a grid plate pattern having three or more gradations is used, and the corners or edge intersections of the grid plate are used as feature points (nodes). Since the main code of the node is given by the gradation distribution surrounding the and the main code of the adjacent node is used as the auxiliary code, many feature codes can be created with few gradation patterns. There are 18 codes even if only the feature code is used. The feature code can be easily increased by further increasing the auxiliary code by using adjacent nodes.

Therefore, when used as a pattern original image in the pattern projection method, the correspondence between the pattern original image and the pattern image or the pattern image can be uniquely determined, and stable and high-density measurement can be performed.

[Brief description of drawings]

FIG. 1 is a schematic diagram for explaining the pattern structure of the present invention, and FIG.
FIG. 1 is a diagram showing a first embodiment, a diagram partially showing an example of a projected pattern image, FIG. 3 is a flow chart showing a flow of overall processing, FIG. 4 is a flow chart showing a flow of association processing, FIG. 5 is a diagram showing the arrangement of the camera and the projection system of the second embodiment using two cameras, and FIG. 6 is the arrangement of the camera and the projection system of the third embodiment using three cameras. FIG. 7 is a diagram showing an embodiment configuration of a surface shape measuring device by pattern projection, FIG. 8 is an oscillographic wavy photograph showing an example of a projected pattern image on a corrugated plate, and FIG. 9 is a processing result. It is the oscillographic wave-like photograph of the projection image shown, (a) is seen from the front direction, (b) is seen from the oblique direction. 3 to 7, 1 to 14 ... Flowchart processing, 21 ... Pattern projector, 22, 23, 25 ... Camera, 24 ...
Object to be measured, 32 …… 3-value lattice plate pattern original plate, 33 …… Projection lens, 34 …… Lamp, 35 …… Image processing device, 36 …… Frame memory, 37 …… Image ternarization processing circuit, 38… … Node detection circuit, 39 …… Feature code identification circuit, 40 …… Table, 41
...... Camera position / azimuth calibration data memory, 42 …… Corresponding processing device, 43 …… 3D position calculation circuit, 44 …… Stereoscopic display device.

Continuation of front page (56) Reference JP-A-60-126775 (JP, A) JP-A-61-277012 (JP, A) JP-A-61-277011 (JP, A)

Claims (1)

(57) [Claims] 1. A gray scale region having three or more shades of three or more values, or three or more colors, or a combination of shades and colors, and at least three gray scale regions at intersections of the boundary lines of the gray scale regions.
A multi-valued grid plate pattern arranged so that the gradation regions of different types are in contact with each other, and the kind and order of the gradations which are in contact with the intersection of the projected image generated by projecting the pattern on the object to be measured. A main code corresponding to the main code is added, or a combination code obtained by combining the main code of the intersection and the main code of the surrounding intersection is added as a characteristic code for identifying the intersection. Spatial pattern encoding method.