JP2019158351A - Object recognition method, height measurement method, and three-dimensional measurement method - Google Patents

Abstract

To provide an object recognition method, a height measurement method, and a three-dimensional measurement method capable of reducing measurement time while improving measurement accuracy.SOLUTION: An object recognition method includes: a first step of projecting a predetermined pattern from a projector (10) toward an object (2); a second step of imaging the reflected light from the object (2) multiple times with an imager (9); and a third step of averaging for each pixel the luminance value of the captured image captured multiple times, and executes for different patterns the first, second and third steps. The shape of the object (2) is recognized based on the obtained multiple averaged image data.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an object recognition method, a height measurement method, and a three-dimensional measurement method for imaging an object on which a specific pattern is projected and recognizing or measuring the shape of the object based on the captured image. It is.

  As a method for recognizing or measuring the shape (for example, height) of an object, a three-dimensional measurement method based on a phase shift method is known (see, for example, Patent Document 1). In the measurement method disclosed in Patent Document 1, a stripe pattern is formed on an object using an illumination device that includes a light source that emits predetermined light and a lattice plate that converts light from the light source into a stripe pattern. Project. Then, by moving the lattice plate, the object is imaged by the camera while moving the fringe pattern projected on the object, and a plurality of captured image data is acquired. The shape (specifically, height) of the object is measured from the plurality of captured image data by the phase shift method.

JP 2017-116303 A

  However, for example, the variation in the amount of light in the light source, the variation in the light receiving sensitivity in the light receiving unit (that is, the sensor) of the camera, and the position when the lattice plate is moved when the lattice plate is moved as in Patent Document 1. There is a possibility that the acquired captured image data (specifically, the luminance value) varies due to variations or the like. For this reason, there is a possibility that the shape of the object cannot be accurately measured. In order to suppress such measurement variations, it may be possible to measure the shape of the object multiple times by repeating the above measurement procedure and average the obtained measurement data, but the measurement time becomes longer. End up.

  The present invention has been made in view of such circumstances, and an object thereof is to provide an object recognition method, a height measurement method, and a three-dimensional measurement method capable of reducing measurement time while improving measurement accuracy. There is.

The object recognition method of the present invention is proposed to achieve the above object, and includes a first step of projecting a predetermined pattern from a projector toward an object,
A second step of imaging the reflected light from the object multiple times with an imaging device;
A third step of averaging the luminance value of the captured image captured a plurality of times for each pixel;
Including
In different patterns, the first step, the second step, and the third step are respectively performed, and the shape of the object is recognized based on a plurality of obtained averaged image data. It is characterized by that.

Moreover, the height measurement method of the present invention includes a first step of projecting a predetermined pattern from a projector toward an object,
A second step of imaging the reflected light from the object multiple times with an imaging device;
A third step of averaging the luminance value of the captured image captured a plurality of times for each pixel;
Including
In different patterns, the first step, the second step, and the third step are respectively performed, and the height of the object is measured based on the obtained plurality of averaged image data. It is characterized by doing.

Furthermore, the three-dimensional measurement method of the present invention includes a first step of projecting a predetermined pattern from a projector toward an object,
A second step of imaging the reflected light from the object multiple times with an imaging device;
A third step of averaging the luminance value of the captured image captured a plurality of times for each pixel;
Including
In a different pattern, the first step, the second step, and the third step are respectively performed, and the three-dimensional shape of the object is obtained based on the plurality of averaged image data obtained. It is characterized by measuring.

  According to the present invention, it is possible to suppress variations in measurement results while shortening the measurement time.

It is a block diagram explaining the structure of one form of the three-dimensional measuring apparatus of this invention. It is a graph explaining a sine wave pattern. It is a flowchart explaining a measuring method. It is a flowchart explaining the conventional measuring method. It is the graph which compared the repetition precision and measurement time of this invention and a comparative example.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described with reference to the accompanying drawings. In the embodiments described below, various limitations are made as preferred specific examples of the present invention. However, the scope of the present invention is not limited to the following description unless otherwise specified. However, the present invention is not limited to these embodiments.

  FIG. 1 is a block diagram showing a configuration of an embodiment of a three-dimensional measuring apparatus 1 according to the present invention. The three-dimensional measuring apparatus 1 in this embodiment includes a stage 3, a measuring instrument group 5, a computer 6, a display device 7 such as a liquid crystal display, and the like, and the three-dimensional shape of the object 2 (high from the reference plane). Are measured using the phase shift method. The object 2 is placed on the placement plate 12 set on the stage 3. The mounting surface 13 which is the upper surface (surface facing the measuring instrument group 5) of the mounting plate 12 also functions as a reference surface serving as a height reference. The mounting plate 12 is configured to be movable in the X direction, the Y direction, and the Z direction by a motor (not shown) incorporated in the stage 3.

  The measuring instrument group 5 in the present embodiment includes an imaging device 9 and a projector 10. The imaging device 9 receives, for example, an optical system composed of a lens or the like, or reflected light of light projected from the projector 10 onto the placement surface 13 or the object 2 placed on the placement surface 13. And a light receiving element for converting it into an electrical signal. The imaging device 9 captures the mounting surface 13 or the target object 2 (hereinafter also referred to as imaging), converts it into image data, and outputs the image data to the computer 6.

  The projector 10 is a projector that projects an image corresponding to the image signal when the image signal from the computer 6 is input. The projector 10 projects a striped sine wave pattern (corresponding to a pattern in the present invention) with brightness values dark and dark toward the placement surface 13 or the object 2 placed on the placement surface 13. In the present embodiment, a plurality of image data corresponding to sine wave patterns whose phases are different from each other are prepared in advance, and one of these is selected by the computer 6 and the sine wave corresponding to the selected image is selected. A pattern is projected from the projector 10. In addition, as a projector, the structure which is equipped with a light source, a pattern grating | lattice, a lens, etc. and moves a pattern grating | lattice to the direction which cross | intersects an optical axis can also be employ | adopted.

  FIG. 2 is a graph showing an example of a sine wave pattern projected from the projector 10. The horizontal axis in FIG. 2 is the coordinate, and the vertical axis is the luminance value. As shown in FIG. 2, the sine wave pattern is a pattern in which luminance (or contrast) changes with a sine wave with respect to coordinates. In the present embodiment, 16 types of sinusoidal patterns with different phases can be projected from the projector 10 without changing the period and amplitude. For example, 16 types of sine wave patterns with phases shifted by π / 8 are prepared. Note that the number of sine wave patterns to be projected is not limited to 16, but it is only necessary to project the types of sine wave patterns necessary for three-dimensional measurement. For example, when three-dimensional measurement is performed using the phase shift method, at least three types of sine wave patterns with different phases may be prepared.

  As shown in FIG. 1, the computer 6 in the present embodiment includes a calculation unit 15 and a storage unit 16. The storage unit 16 is configured by, for example, a hard disk drive or a flash memory. In addition to the operation system, various application programs, calculation data, and the like, the storage unit 16 stores image data of a sine wave pattern to be projected by the projector 10. The calculation unit 15 includes a CPU, a ROM, a RAM, and the like (not shown), and performs various processes such as image processing according to the operation system stored in the storage unit 16. In addition, captured image data (hereinafter simply referred to as a captured image) of the placement surface 13 or the object 2 input from the imaging device 9 is stored in the storage unit 16.

  Next, a measurement method for measuring the object 2 in the three-dimensional measurement apparatus 1 having the above configuration will be described. FIG. 3 is a flowchart for explaining a measurement method according to the present invention. First, one sine wave pattern is selected from 16 types of sine wave patterns (step S1), and the sine wave pattern is projected from the projector 10 toward the object 2 placed on the placement surface 13 (step S1). S2, corresponding to the first step in the present invention). Next, in this state, that is, in the state in which one sine wave pattern is projected, the reflected light from the object 2 is continuously imaged n times (in this embodiment, 8 times) by the imaging device 9 (in other words, in other words, in other words) Then, the image is taken) (step S3, corresponding to the second step in the present invention). As a result, n captured images, that is, eight captured images in the present embodiment are obtained. Here, even if continuous imaging is performed without changing the projected sine wave pattern, the influence of variations in the amount of light in the light source in the projector 10 and variations in the light receiving sensitivity in the light receiving element in the imaging device 9 is affected. As a result, the captured images may vary. For this reason, the obtained eight captured images are averaged by image processing to suppress this variation. That is, in step S4, the luminance values of the eight captured images are averaged for each pixel (corresponding to the third step in the present invention). The averaged image data (hereinafter referred to as an averaged image) obtained in this way is stored in the storage unit 16.

  The step of obtaining the averaged image (that is, the first step, the second step, and the third step in the present invention) is executed in all 16 types of sine wave patterns. Specifically, it is determined whether averaged images have been acquired for all 16 types of sine wave patterns (step S5). When averaged images are not acquired in all 16 types of sine wave patterns (NO in step S5), sine wave patterns for which averaged images have not been acquired, that is, sine waves for which averaged images have already been acquired A sine wave pattern having a phase different from that of the pattern is selected (step S6), and the sine wave pattern is projected from the projector 10 onto the object 2 placed on the placement surface 13 (step S2). And as above mentioned, the reflected light from the target object 2 is continuously imaged n times with the imaging device 9 (step S3), and a captured image is averaged by image processing (step S4). In this way, if averaged images are obtained for all 16 types of sine wave patterns (YES in step S5), the height is increased by a known phase shift method based on these 16 averaged images. The required calculation process is performed (step S7). And the obtained measurement result data is memorize | stored in the memory | storage part 16, and a measurement is complete | finished. As described above, the three-dimensional shape of the object 2 can be accurately measured.

  As described above, since a plurality of captured images (eight captured images in the present embodiment) are averaged to produce an averaged image, variations in measurement results can be suppressed. Further, since the object 2 is continuously imaged without changing the sine wave pattern, the switching time of the sine wave pattern can be shortened. That is, 16 types of sine wave patterns are imaged once, a series of steps for obtaining the height from the 16 types of captured images by the phase shift method are performed a plurality of times (for example, 8 times), and finally the measurement result data is averaged Compared with the measuring method to be changed, the number of times of switching the sine wave pattern can be reduced. In particular, when the projector 10 is a projector using a liquid crystal panel or the like, it takes one frame to switch images, so that a remarkable effect can be obtained by reducing the number of switching of the sine wave pattern. Furthermore, 16 types of sine wave patterns are imaged once, a series of steps for obtaining the height from the 16 types of captured images by the phase shift method are performed a plurality of times (for example, 8 times), and finally the measurement result data is averaged. The number of times of performing the calculation process for obtaining the height is reduced as compared with the measuring method to be realized. As a result, measurement time can be shortened while suppressing variations in measurement results.

  Next, as a comparative example, 16 types of sine wave patterns are imaged once, a series of steps for obtaining the height from the 16 types of captured images by the phase shift method are performed a plurality of times (for example, 8 times), and finally A measurement method for averaging the measurement result data will be described. FIG. 4 is a flowchart for explaining the measurement method. In the case of this measurement method, first, one sine wave pattern is selected from the 16 types of sine wave patterns (step S11), and the sine wave pattern is applied from the projector 10 to the object 2 placed on the placement surface 13. Projecting toward the screen (step S12). Next, in this state, that is, in a state where one sine wave pattern is projected, the reflected light from the object 2 is imaged once by the imaging device 9 (step S13). This series of steps is executed for all 16 types of sine wave patterns. Specifically, it is determined whether or not captured images have been acquired in all 16 types of sine wave patterns (step S14). When captured images are not acquired for all 16 types of sine wave patterns (NO in step S14), a sine wave pattern for which captured images are not acquired is selected (step S15), and the sine wave pattern is projected. Projection is performed from the machine 10 toward the object 2 placed on the placement surface 13 (step S12). And the reflected light from the target object 2 is imaged once with the imaging device 9, and a captured image is acquired (step S13). If such steps are repeated and captured images are obtained for all 16 types of sine wave patterns (YES in step S14), the height is determined by a known phase shift method based on these 16 captured images. Is calculated (step S16).

  Thereafter, the same processing is performed a plurality of times. Specifically, it is determined whether or not the calculation process for obtaining the height has been performed n times (for example, 8 times) (step S17). If n times (for example, 8 times) have not been performed (NO in step S17), the above-described series of steps from step S11 to step S16 is executed again. That is, switching of 16 types of sine wave patterns is performed again. Then, the same processing is performed n times (for example, 8 times), and if a plurality of (for example, 8) height data corresponding to this is acquired (YES in step S17), these multiple (for example, for example) Eight) height data is averaged (step S18). The obtained measurement result data is stored in the storage unit 16 and the measurement ends. Thereby, the three-dimensional three-dimensional shape of the target object 2 can be accurately measured.

  In short, in the case of the above-described comparative example, switching of the sine wave pattern is 16 times × n times (8 times), whereas in the present invention, imaging is performed n times (8 times) without changing the sine wave pattern. Therefore, the sine wave pattern is switched 16 times. Further, in the case of the comparative example described above, the calculation process by the phase shift method is n times (8 times), whereas in the case of the present invention, the calculation process is performed using the averaged image. It becomes. For this reason, in this invention, it becomes possible to shorten measurement time, suppressing the dispersion | variation in a measurement result.

  FIG. 5 is a graph comparing the measurement method in the present invention and the measurement method in the comparative example. “◯” shown in FIG. 5 represents a value of repetition accuracy (that is, measurement accuracy), and “x” represents measurement time. The left vertical axis is the repeatability, and the right vertical axis is the measurement time [s]. The repeatability indicates that the higher the numerical value, the higher the measurement accuracy. That is, in the example of FIG. 5, the measurement accuracy is the highest when the repetition accuracy is “1”, and the accuracy decreases as the value becomes smaller (that is, the closer to “0.4”). As shown in FIG. 5, it can be seen that there is almost no difference in measurement accuracy between the measurement method of the present invention and the measurement method of the comparative example. On the other hand, in the measurement time, it can be seen that the measurement method in the present invention is shortened by about 0.3 [s] than the measurement method in the comparative example. Thus, by using the measurement method of the present invention, the measurement time can be shortened while maintaining measurement accuracy.

  By the way, the present invention is not limited to the above-described embodiment, and various modifications can be made based on the description of the scope of claims. For example, the number of times of capturing one sine wave pattern is not limited to eight, and can be arbitrarily set according to measurement accuracy. Further, the types of sine wave patterns are not limited to 16 types, and can be set to any type according to measurement accuracy. Furthermore, the calculation process for obtaining the height of the object 2 is not limited to the phase shift method, and other methods may be employed. For example, the height of the object 2 can be obtained by a calculation process using a spatial coding method. Note that, when performing a calculation process using the spatial coding method, the projected pattern is, for example, a black and white binarized stripe pattern. In this case, stripe patterns with different stripe widths (for example, stripe patterns with stripe widths that become narrower in stages such as half the screen, 1/4, 1/8, 1/16, etc.) are sequentially projected. Is done.

  In the above description, the three-dimensional measurement method for measuring the three-dimensional shape of the object 2 in the three-dimensional measurement apparatus 1 has been described. However, the present invention can also be applied to other apparatuses. In the object recognition apparatus, the present invention can also be applied to an object recognition method for recognizing the shape of an object (for example, what pattern is formed at which position) by a phase shift method or the like. Further, the present invention can also be applied to a height measurement method in which the height of an object (for example, the height or thickness from a reference surface at a predetermined position) is measured by a phase shift method or the like in a height measurement apparatus. In short, the present invention can be applied to a method in which a plurality of patterns are projected onto an object, an image is taken for each projected pattern, and image processing is performed from the obtained plurality of captured images.

  DESCRIPTION OF SYMBOLS 1 ... Three-dimensional measuring device, 2 ... Object, 3 ... Stage, 5 ... Measuring machine group, 6 ... Computer, 7 ... Display apparatus, 9 ... Imaging device, 10 ... Projector, 12 ... Mounting plate, 13 ... Mounting Placement surface, 15 ... calculation unit, 16 ... storage unit

Claims (3)

A first step of projecting a predetermined pattern from a projector toward an object;
A second step of imaging the reflected light from the object multiple times with an imaging device;
A third step of averaging the luminance value of the captured image captured a plurality of times for each pixel;
Including
In different patterns, the first step, the second step, and the third step are respectively performed, and the shape of the object is recognized based on a plurality of obtained averaged image data. An object recognition method characterized by the above. A first step of projecting a predetermined pattern from a projector toward an object;
A second step of imaging the reflected light from the object multiple times with an imaging device;
A third step of averaging the luminance value of the captured image captured a plurality of times for each pixel;
Including
In different patterns, the first step, the second step, and the third step are respectively performed, and the height of the object is measured based on the obtained plurality of averaged image data. A height measurement method characterized by: A first step of projecting a predetermined pattern from a projector toward an object;
A second step of imaging the reflected light from the object multiple times with an imaging device;
A third step of averaging the luminance value of the captured image captured a plurality of times for each pixel;
Including
In a different pattern, the first step, the second step, and the third step are respectively performed, and the three-dimensional shape of the object is obtained based on the plurality of averaged image data obtained. A three-dimensional measuring method characterized by measuring.

Images