JP7247031B2 - inspection equipment - Google Patents

Description

The present invention relates to an inspection device for inspecting the height of an object to be measured.

In a triangulation type inspection apparatus, for example, a light projecting section irradiates a surface of an object to be measured with structured light, and the reflected light is received by a light receiving section having pixels arranged one-dimensionally or two-dimensionally. . Height image data representing a height image of the object to be measured is generated based on the data of the light-receiving amount distribution obtained by the light-receiving unit.

Regarding the generation of height image data, depending on the irradiation direction of the structured light to the measurement object, the light reception direction of the structured light reflected from the measurement object, or the shape of the measurement object, there may be an unmeasurable area on the surface of the measurement object. Possible parts may occur.

Therefore, by sequentially irradiating the object to be measured with structured light in a plurality of directions and by sequentially receiving the structured light reflected in one direction by the object to be measured, a plurality of heights corresponding to a plurality of irradiation directions are obtained. There is a method to generate the image data and synthesize them. Alternatively, a plurality of height image data corresponding to a plurality of light receiving directions can be obtained by irradiating structured light in one direction toward the measurement object and receiving structured light reflected from the measurement object in a plurality of directions. , and there is a way to compose them. By synthesizing a plurality of height image data respectively corresponding to a plurality of directions, height image data with reduced unmeasurable portions is generated.

For example, in the three-dimensional measurement apparatus described in Patent Document 1, one projector irradiates a measurement object with structured light. The object to be measured irradiated with structured light is imaged by a plurality of imaging means arranged at a plurality of positions. A three-dimensional point cloud (height image data) of the object to be measured is measured using the images obtained by each imaging means. Measurement results of a plurality of three-dimensional point groups respectively corresponding to a plurality of imaging means are integrated.

Patent No. 5633058

When integrating the measurement results of multiple 3D point clouds, the reliability is compared for each corresponding pixel among the multiple 3D point clouds, and the highly reliable measurement result is the final measurement for that pixel. Retained as a candidate result. A final measurement result is obtained from the measurement results left for each pixel.

In the technique described in Patent Document 1, reliability comparison is performed by comparing the amplitude of luminance of each pixel obtained during phase analysis using structured light. Specifically, for a certain pixel, a measurement result corresponding to an imaging means determined to have a large luminance amplitude among a plurality of imaging means is left.

In this case, for example, adoption of a saturated measurement value due to the occurrence of halation and adoption of a noise level measurement value due to the occurrence of shadows is reduced. However, even if the luminance amplitude for each pixel is used as described above, it may not be possible to perform an appropriate reliability comparison.

For example, depending on the shape, surface condition and material of the measurement object, multiple reflections may occur on the surface of the measurement object when structured light is irradiated onto the measurement object. In this case, there is a possibility that the amplitude of luminance will not decrease (will not be attenuated) at the portion where multiple reflection occurs. Therefore, if a measurement result containing a component of multiple reflection is left as a candidate for the final measurement result for that portion, the accuracy of the inspection of the object to be measured decreases.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an inspection apparatus capable of improving the accuracy of inspection of the height of an object to be measured.

(1) An inspection apparatus according to a first invention includes a first illumination unit, a second illumination unit, and a third illumination unit that irradiate a measurement object with structured light from four different directions while changing the patterns into a plurality of patterns. A plurality of first pattern image data corresponding to the first illumination unit are generated by receiving the structured light reflected from the object to be measured by the illumination unit and the fourth illumination unit, and the second illumination unit generating a plurality of second pattern image data corresponding to the illumination portion; generating a plurality of third pattern image data corresponding to the third illumination portion; generating a plurality of fourth pattern image data corresponding to the fourth illumination portion; an imaging unit that generates image data; a first height image data of the measurement object based on the plurality of first pattern image data; and a measurement object based on the plurality of second pattern image data. generating second height image data; generating third height image data of the measurement object based on the plurality of third pattern image data; and generating measurement object based on the plurality of fourth pattern image data; An image processing unit comprising: a data generation unit that generates fourth height image data of an object; and an image processing unit that generates synthesized height image data by synthesizing the first to fourth height image data. extracts two height data according to a predetermined condition from the height data of the same pixel positions corresponding to each other in the first to fourth height image data, calculates the difference between them, and calculates determines whether the calculated difference is within a predetermined allowable range, and if the calculated difference is within the allowable range, at least one of the two height data used to calculate the difference is determined. The height data at the pixel position of the composite height image data is determined based on the height data, and if the calculated difference is not within the allowable range, the height data at the pixel position of the composite height image data is set to an invalid value. , and the predetermined condition is the minimum difference among the plurality of calculated differences when the difference between each two height data among the height data at the same pixel position is calculated. It is to extract the two height data used for the calculation.

In the inspection apparatus, the object to be measured is irradiated a plurality of times with structured light from each of the first to fourth illumination units while being changed into a plurality of patterns. The imaging unit receives structured light reflected from the object to be measured. Thereby, a plurality of first pattern image data corresponding to the first illumination section are generated, and the first height image data are generated based on the plurality of first pattern image data. Also, a plurality of second pattern image data corresponding to the second illumination units are generated, and second height image data is generated based on the plurality of second pattern image data. Also, a plurality of third pattern image data corresponding to the third illumination unit are generated, and third height image data is generated based on the plurality of third pattern image data. Also, a plurality of fourth pattern image data corresponding to the fourth illumination section are generated, and fourth height image data is generated based on the plurality of fourth pattern image data.

The first to fourth height image data are synthesized to generate synthesized height image data. At the time of this synthesis, two height data according to a predetermined condition are extracted from the height data of the same pixel positions corresponding to each other in the first to fourth height image data, and the difference between them is calculated. . Also, it is determined whether or not the calculated difference is within the allowable range.

Here, when the calculated difference is within the allowable range, the two height data used for calculating the difference are within the allowable error range with respect to the true height of the portion of the measurement object corresponding to the pixel position. This is highly accurate data that indicates the height within. Therefore, when the calculated difference is within the allowable range, based on at least one height data of the two height data used to calculate the difference, the pixel position of the synthetic height image data is calculated. Height data is determined.

On the other hand, if the calculated difference is not within the allowable range, the two height data used to calculate the difference are low-accuracy data indicating a height outside the allowable error range with respect to the true height. Very likely. Therefore, if the calculated difference is not within the allowable range, the height data at the pixel position is set to an invalid value in the combined height image data.

As a result, at each pixel position of the synthetic height image data, the value of height data with high accuracy is indicated, but the value of height data with low accuracy is not indicated. Therefore, by using the generated synthetic height image data, it is possible to improve the accuracy of inspection of the height of the object to be measured.

Further, the predetermined condition is to calculate the minimum difference among the plurality of calculated differences when the difference between each two height data among the height data at the same pixel position is calculated. It is to extract the two height data used.

The two height data extracted according to the above conditions are considered to have particularly high accuracy among the height data at the same pixel position. Therefore, the accuracy of synthetic height image data is further improved.

( 2 ) If the difference between the two height data in accordance with a predetermined condition among the height data at the same pixel position is within the allowable range, the image processing unit The average value of the height data or one of the two height data values may be determined as the height data value at the pixel position of the composite height image data.

In this case, the value of the height data at each pixel position of the synthetic height image data is calculated easily and appropriately.

(3) An inspection apparatus according to a second aspect of the present invention includes a first illumination section, a second illumination section, and a third illumination section that irradiate an object to be measured while changing structured light from four different directions into a plurality of patterns. A plurality of first pattern image data corresponding to the first illumination unit are generated by receiving the structured light reflected from the object to be measured by the illumination unit and the fourth illumination unit, and the second illumination unit generating a plurality of second pattern image data corresponding to the illumination portion; generating a plurality of third pattern image data corresponding to the third illumination portion; generating a plurality of fourth pattern image data corresponding to the fourth illumination portion; an imaging unit that generates image data; a first height image data of the measurement object based on the plurality of first pattern image data; and a measurement object based on the plurality of second pattern image data. generating second height image data; generating third height image data of the measurement object based on the plurality of third pattern image data; and generating measurement object based on the plurality of fourth pattern image data; An image processing unit comprising: a data generation unit that generates fourth height image data of an object; and an image processing unit that generates synthesized height image data by synthesizing the first to fourth height image data. extracts two height data according to a predetermined condition from the height data of the same pixel positions corresponding to each other in the first to fourth height image data, calculates the difference between them, and calculates It is determined whether the calculated difference is within a predetermined allowable range, and if the calculated difference is within the allowable range, the average value of the two height data used to calculate the difference, or 2 One of the two height data is determined as the value of the height data at the pixel position of the composite height image data, and if the calculated difference is not within the allowable range, the pixel position of the composite height image data set the height data to an invalid value.

( 4 ) An inspection apparatus according to a third aspect of the present invention includes a first illumination section, a second illumination section, and a third illumination section that irradiate an object to be measured while changing structured light from four different directions into a plurality of patterns. A plurality of first pattern image data corresponding to the first illumination unit are generated by receiving the structured light reflected from the object to be measured by the illumination unit and the fourth illumination unit, and the second illumination unit generating a plurality of second pattern image data corresponding to the illumination portion; generating a plurality of third pattern image data corresponding to the third illumination portion; generating a plurality of fourth pattern image data corresponding to the fourth illumination portion; an imaging unit that generates image data; a first height image data of the measurement object based on the plurality of first pattern image data; and a measurement object based on the plurality of second pattern image data. generating second height image data; generating third height image data of the measurement object based on the plurality of third pattern image data; and generating measurement object based on the plurality of fourth pattern image data; An image processing unit comprising: a data generation unit that generates fourth height image data of an object; and an image processing unit that generates synthesized height image data by synthesizing the first to fourth height image data. extracts two height data according to a predetermined condition from the height data of the same pixel positions corresponding to each other in the first to fourth height image data, calculates the difference between them, and calculates determines whether the calculated difference is within a predetermined allowable range, and if the calculated difference is within the allowable range, at least one of the two height data used to calculate the difference is determined. The height data at the pixel position of the composite height image data is determined based on the height data, and if the calculated difference is not within the allowable range, the height data at the pixel position of the composite height image data is set to an invalid value. , and the data generation unit, when generating each of the first to fourth height image data, sets the plurality of pattern image data generated corresponding to the illumination unit and the predetermined light reception determination condition to Based on this, it is determined whether or not the portion of the measurement object corresponding to each pixel position of the height image data to be generated is an unilluminatable portion that cannot reflect the structured light to the imaging unit. , the height data of the pixel position corresponding to the non-illumination portion is set as the illumination invalid value, and the image processing unit does not use the illumination invalid value for calculating the difference between the two height data according to the predetermined condition. .

In this case, the height data values of the pixel positions indicating the unilluminated portion in the first to fourth height image data are not used for generating the synthetic height image data. This prevents unstable height components due to non-illumination areas from remaining in the synthetic height image data.

( 5 ) When only one height data out of four height data at each pixel position indicates a value other than an illumination invalid value and the other height data is set to an illumination invalid value In addition, it may be possible to further select whether to set one height data as the height data of the pixel position or set the value of the height data of the pixel position to an invalid value.

In this case, appropriate synthetic height image data can be generated according to the shape, material, surface condition, etc. of the object to be measured.
(6) The first illumination unit and the second illumination unit are arranged to face each other with the imaging unit interposed in the first direction, and the third illumination unit and the fourth illumination unit are arranged in the first direction. They may be arranged so as to face each other across the imaging unit in a second direction that intersects with the first direction.
In this case, it is possible to irradiate the structured light over a wide range of the surface of the object to be measured. Therefore, synthetic height image data with high accuracy can be acquired over a wide range of the surface of the object to be measured.

According to the present invention, the accuracy of inspection for the height of the object to be measured is improved.

1 is a block diagram showing the configuration of an inspection device according to one embodiment of the present invention; FIG. 1 is a block diagram showing the configuration of an inspection device according to one embodiment of the present invention; FIG. FIG. 2 is a diagram showing an example of a configuration of each illumination unit in FIG. 1; FIG. It is a figure for demonstrating the principle of a triangulation ranging method. 2 is a flowchart showing an example of inspection processing executed by the inspection apparatus of FIG. 1; (a) is an external perspective view showing an example of a measurement object for explaining the accuracy of height image data, (b) is a plan view of the measurement object S in (a), and (c). is an end view of the object to be measured of (a). FIG. 7 is a schematic perspective view showing an example of a positional relationship between a plurality of illumination units and imaging units shown in FIG. 2 and a measurement object during inspection of the measurement object shown in FIG. 6; (a) is a side view showing a state in which the object to be measured is irradiated with structured light from one illumination unit in FIG. 7, and (b) is the height obtained by irradiation of the structured light in (a). FIG. 4 is a diagram showing an image; (a) is a side view showing a state in which the object to be measured is irradiated with structured light from another illumination unit in FIG. 7, and (b) is the height obtained by irradiation of the structured light in (a). FIG. 4 is a diagram showing an image; (a) is a side view showing a state in which the object to be measured is irradiated with structured light from still another illumination unit in FIG. 7; FIG. 10 is a diagram showing a sparse image; (a) is a side view showing a state in which the object to be measured is irradiated with structured light from still another illumination unit in FIG. 7; FIG. 10 is a diagram showing a sparse image; It is a figure which shows the concept of the compositing process which concerns on one embodiment of this invention. It is a figure which shows an example of the height image based on the synthetic|combination height image data produced|generated by the synthetic|combination process which concerns on one embodiment of this invention. 4 is a flow chart of synthesis processing according to an embodiment of the present invention; 10 is a flow chart of composition processing according to another embodiment.

An inspection apparatus according to an embodiment of the present invention will be described below with reference to the drawings.

[1] Configuration of Inspection Apparatus FIGS. 1 and 2 are block diagrams showing the configuration of an inspection apparatus according to an embodiment of the present invention. As shown in FIGS. 1 and 2, the inspection apparatus 300 includes a head section 100, a controller section 200, an operation section 310 and a display section 320. FIG. The controller section 200 is connected to an external device 400 such as a programmable logic controller.

As indicated by thick arrows in FIG. 1, a plurality of objects S to be measured are sequentially conveyed by the belt conveyor 301 so as to pass through the space below the head section 100 . When each measurement object S passes through the space below the head unit 100, the belt conveyor 301 is moved for a certain period of time so that the measurement object S temporarily stops at a predetermined position below the head unit 100. Stop.

The head unit 100 is, for example, an imaging device that integrates light emitting and receiving, and includes a plurality of lighting units 110 , an imaging unit 120 and a computing unit 130 . 2, illustration of the calculation unit 130 is omitted. In this embodiment, four illumination units 110 are provided so as to surround the imaging unit 120 at intervals of 90 degrees.

Each illumination unit 110 is configured to be able to irradiate the measurement object S with red, blue, green, or white light having an arbitrary pattern from obliquely above. In addition, each lighting unit 110 is configured to be able to irradiate the measurement object S with uniform red, blue, green, or white light having no pattern from obliquely above. Hereinafter, light having an arbitrary pattern is called structured light, and uniform light is called uniform light.

Moreover, when distinguishing the four illumination parts 110, the four illumination parts 110 are each called illumination part 110A, 110B, 110C, and 110D. The illumination unit 110A and the illumination unit 110B face each other with the imaging unit 120 interposed therebetween. Also, the illumination unit 110C and the illumination unit 110D face each other with the imaging unit 120 interposed therebetween. A configuration of the illumination unit 110 will be described later.

The imaging unit 120 includes an imaging element 121 and light receiving lenses 122 and 123, as shown in FIG. At least the light receiving lens 122 of the light receiving lenses 122 and 123 is a telecentric lens. The structured light reflected upward by the measurement object S is collected and imaged by the light receiving lenses 122 and 123 of the imaging unit 120 and then received by the imaging device 121 . The imaging element 121 is, for example, a monochrome CCD (charge-coupled device), and generates image data by outputting an analog electrical signal corresponding to the amount of light received from each pixel. The imaging device 121 may be another imaging device such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

In the following description, image data representing an image of the measurement object S generated by the imaging unit 120 while the measurement object S is irradiated with structured light will be referred to as pattern image data. On the other hand, image data representing an image of the measurement object S generated by the imaging unit 120 while the measurement object S is irradiated with uniform light is called texture image data.

The calculation unit 130 is implemented by, for example, an FPGA (Field Programmable Gate Array), and includes an imaging processing unit 131 , a calculation processing unit 132 , an image processing unit 133 , a storage unit 134 and an output processing unit 135 . In the present embodiment, arithmetic unit 130 is realized by FPGA, but the present invention is not limited to this. Arithmetic unit 130 may be realized by a CPU (central processing unit) and RAM (random access memory), or may be realized by a microcomputer.

The imaging processing unit 131 controls operations of the four lighting units 110 and the imaging units 120 . The arithmetic processing unit 132 generates height image data representing a height image of the measurement object S based on the plurality of pattern image data for each of the four illumination units 110 . The height image data is composed of a plurality of height data indicating heights of a plurality of portions of the measuring object S respectively corresponding to a plurality of pixel positions.

The image processing unit 133 generates synthetic height image data by synthesizing a plurality of height image data generated corresponding to the plurality of illumination units 110 by the arithmetic processing unit 132 . Further, the image processing unit 133 generates synthesized texture image data by synthesizing a plurality of pieces of texture image data generated by the imaging unit 120 corresponding to the plurality of illumination units 110 respectively. Details of the image processing unit 133 will be described later.

The storage unit 134 is used as a work area in the calculation unit 130, and stores pattern image data, height image data, synthetic height image data, and texture images generated by the imaging unit 120, the calculation processing unit 132, and the image processing unit 133. Temporarily store data and synthetic texture image data. The output processing unit 135 outputs the synthetic height image data or synthetic texture image data stored in the storage unit 134 .

Controller section 200 includes head control section 210 , image memory 220 and inspection section 230 . The head control section 210 controls the operation of the head section 100 based on commands given by the external device 400 . The image memory 220 stores the synthetic height image data or the synthetic texture image data output by the calculation unit 130 .

The inspection unit 230 executes processing such as edge detection or dimension measurement based on the inspection content specified by the user and the synthetic height image data or synthetic texture image data stored in the image memory 220 . The inspection unit 230 also compares the measured value with a predetermined threshold value to determine whether the measurement object S is good or bad, and gives the determination result to the external device 400 .

An operation unit 310 and a display unit 320 are connected to the controller unit 200 . Operation unit 310 includes a keyboard, pointing device, or dedicated console. A mouse, a joystick, or the like is used as the pointing device. The user can designate desired examination contents to the controller unit 200 by operating the operation unit 310 .

The display unit 320 is configured by, for example, an LCD (liquid crystal display) panel or an organic EL (electroluminescence) panel. The display unit 320 displays a height image or the like based on the synthetic height image data stored in the image memory 220 . The display unit 320 also displays the determination result of the measurement object S by the inspection unit 230 .

FIG. 3 is a diagram showing an example of the configuration of each illumination unit 110 in FIG. 1. As shown in FIG. As shown in FIG. 3, each illumination section 110 includes light sources 111 , 112 and 113 , dichroic mirrors 114 and 115 , illumination lens 116 , mirror 117 , pattern generator 118 and projection lens 119 . The light sources 111, 112, and 113 are LEDs (light emitting diodes), for example, and emit green light, blue light, and red light, respectively. Each of the light sources 111-113 may be light sources other than LEDs.

The dichroic mirror 114 is arranged so that the green light emitted by the light source 111 and the blue light emitted by the light source 112 can be superimposed. The dichroic mirror 115 is arranged so that the light superimposed by the dichroic mirror 114 and the red light emitted by the light source 113 can be superimposed. Thereby, the lights emitted by the light sources 111 to 113 are superimposed on a common optical path, and white light can be generated.

The illumination lens 116 collects light that has passed through or is reflected by the dichroic mirror 115 . The mirror 117 reflects the light condensed by the illumination lens 116 to the pattern generator 118 . The pattern generation unit 118 is, for example, a DMD (digital micromirror device), and imparts an arbitrary pattern to incident light. The pattern generator 118 may be an LCD or LCOS (reflective liquid crystal element). The light projection lens 119 magnifies the light from the light generation surface of the pattern generation unit 118 and irradiates the measurement object S in FIG. Note that the light projection lens 119 may be configured to collimate the light from the pattern generation unit 118 and irradiate the measurement object S with the light.

The imaging processing unit 131 in FIG. 1 controls the emission of light from the light sources 111 to 113 and also controls the pattern given to the light by the pattern generation unit 118 . Accordingly, it is possible to selectively emit red, blue, green, or white structured light and red, blue, green, or white uniform light from illumination section 110 . Note that, in the present embodiment, white structured light or uniform light is selectively emitted from each illumination unit 110 to the measurement object S. As shown in FIG.

[2] Generation of Height Image Data In the inspection device 300, a three-dimensional coordinate system (hereinafter referred to as device coordinate system) unique to the head unit 100 is defined. The device coordinate system of this example includes an origin and mutually orthogonal X, Y, and Z axes. In the following description, the direction parallel to the X axis of the apparatus coordinate system is called the X direction, the direction parallel to the Y axis is called the Y direction, and the direction parallel to the Z axis is called the Z direction. The X direction and the Y direction are orthogonal to each other within a plane parallel to the upper surface of the belt conveyor 301 (hereinafter referred to as a reference plane). The Z direction is orthogonal to the reference plane.

In the head unit 100, height image data representing a height image of the measurement object S is generated by a triangulation method. FIG. 4 is a diagram for explaining the principle of the triangulation method. The X, Y and Z directions are indicated by arrows in FIG. As shown in FIG. 4, an angle α between the optical axis of the light emitted from the illumination section 110 and the optical axis of the light incident on the imaging section 120 is set in advance. The angle α is greater than 0 degrees and less than 90 degrees.

When the measurement object S does not exist below the head unit 100 , the light emitted from the illumination unit 110 is reflected by the point O on the reference plane R and enters the imaging unit 120 . On the other hand, when the measurement object S exists below the head unit 100 , the light emitted from the illumination unit 110 is reflected by the point A on the surface of the measurement object S and enters the imaging unit 120 . As a result, the measurement object S is imaged, and image data representing an image of the measurement object S is generated.

Assuming that the distance in the X direction between the points O and A is d, the height h of the point A of the measuring object S with respect to the reference plane R is given by h=d/tan(α). The calculator 130 calculates the distance d based on the image data generated by the imaging unit 120 . Further, the calculation unit 130 calculates the height h of the point A on the surface of the measurement object S based on the calculated distance d. By calculating the heights of all the points on the surface of the measurement object S, it is possible to specify the coordinates expressed in the apparatus coordinate system for all the points irradiated with light. Thereby, height image data of the measuring object S is generated.

In order to illuminate all points on the surface of the measurement object S, structured light having various patterns is sequentially emitted from each illumination unit 110 . In the present embodiment, striped structured light (hereinafter referred to as striped light) having a linear cross section parallel to the Y direction and aligned in the X direction is changed in spatial phase. The light is emitted from each illumination unit 110 multiple times. In addition, a code-like structured light (hereinafter referred to as code-like light) having a linear cross section parallel to the Y direction and having a bright portion and a dark portion aligned in the X direction is composed of the bright portion and the dark portion. is emitted from each lighting unit 110 a plurality of times while being changed in a gray code manner.

[3] Basic Flow of Inspection Processing FIG. 5 is a flowchart showing an example of inspection processing executed by the inspection apparatus 300 of FIG. In the inspection process of this example, synthetic height image data and synthetic texture image data are generated in this order. As shown in FIG. 5, the imaging processing unit 131 first selects one of the illumination units 110A to 110D (step S11). Next, the imaging processing unit 131 controls the illumination unit 110 selected in step S11 or step S19 described later so as to emit white structured light having a predetermined pattern (step S12).

In addition, the imaging processing unit 131 controls the imaging unit 120 so as to capture an image of the measurement object S in synchronization with emission of the structured light in step S12 (step S13). As a result, pattern image data of the measuring object S is generated by the imaging unit 120 . After that, the imaging processing unit 131 causes the storage unit 134 to store the pattern image data generated in step S13 (step S14). Subsequently, the imaging processing unit 131 determines whether or not imaging has been performed a predetermined number of times (step S15).

In step S15, if imaging has not been performed a predetermined number of times, the imaging processing unit 131 controls the pattern generation unit 118 in FIG. 3 to change the pattern of structured light (step S16), and returns to step S12. . Steps S12 to S16 are repeated until imaging is performed a predetermined number of times. As a result, a plurality of pattern image data are stored in the storage unit 134 when the measuring object S is sequentially irradiated with the striped light and the coded light while the pattern is changed. Either the striped light or the coded light may be emitted first.

In step S15, when the predetermined number of times of imaging is performed, the arithmetic processing unit 132 generates height image data by performing arithmetic operations on the plurality of pattern image data stored in the storage unit 134 (step S17). .

Here, the magnitude of variations in the values (luminance values) of a plurality of pixels at the same pixel position corresponding to each other in the plurality of pattern image data generated for one illumination unit 110 is equal to or less than a predetermined threshold. means that light does not reach portions of the measuring object S corresponding to the plurality of pixels, and shadows are generated. Alternatively, the variation in the values of the plurality of pixels at the same pixel position corresponding to each other in the plurality of pattern image data generated for one lighting unit 110 is equal to or less than a predetermined threshold value. It means that the reflected light from the portion of the measurement object S corresponding to the pixel does not enter the imaging section 120 .

Therefore, when generating the height image data, the arithmetic processing unit 132 corresponds to a portion of the measurement object S that cannot reflect the structured light to the imaging unit 120 (hereinafter referred to as a non-illumination portion). The value of the height data of the pixel position to be detected is set to a predetermined illumination invalid value.

Specifically, the arithmetic processing unit 132 determines the degree of variation in the values of the plurality of pixel data for the plurality of pixels at the same pixel position corresponding to each other in the plurality of pattern image data generated in steps S12 to S16. It is determined whether or not it is equal to or less than a predetermined threshold. Therefore, when the magnitude of variation in the plurality of values is equal to or less than a predetermined threshold value, the arithmetic processing unit 132 calculates a plurality of pixels at the same pixel positions corresponding to each other in the plurality of pattern image data in the height image data. Set the height data value corresponding to the lighting invalid value. In this way, the value of the height data of the pixel position corresponding to the non-illumination portion in the height image data is set as the illumination invalid value. The lighting disabled value is 0, for example. The above-mentioned threshold value for determining the magnitude of variation of a plurality of values is stored in advance in the storage unit 134 of FIG.

Next, the arithmetic processing unit 132 determines whether or not a predetermined number (four in this example) of height image data corresponding to each of the plurality of illumination units 110 has been generated (step S18). If the predetermined number of height image data has not been generated, the imaging processing unit 131 selects another lighting unit 110 (step S19), and returns to step S12. Steps S12 to S19 are repeated until a predetermined number of height image data are generated.

When the predetermined number of height image data is generated in step S18, the image processing unit 133 performs synthesis processing for synthesizing the generated predetermined number of height image data (step S20). Thereby, synthetic height image data is generated. Details of the combining process will be described later. Also, the output processing unit 135 outputs the synthetic height image data generated in step S20 to the controller unit 200 (step S21). Thereby, the synthetic height image data is accumulated in the image memory 220 of the controller section 200 .

Subsequently, texture generation processing is executed in the head unit 100 (step S22). In texture generation processing, the imaging processing unit 131 controls the plurality of lighting units 110 so that white uniform light is sequentially emitted. In addition, the imaging processing unit 131 controls the imaging unit 120 so as to image the measurement object S in synchronization with the emission of each uniform light. Thereby, a plurality of texture image data corresponding to the plurality of lighting units 110 are generated. Furthermore, the image processing unit 133 synthesizes a plurality of generated texture image data. Thereby, synthetic texture image data is generated. The output processing unit 135 also outputs the generated synthetic texture image data to the controller unit 200 . As a result, the synthetic texture image data is accumulated in the image memory 220 of the controller unit 200 .

Thereafter, determination processing is executed in the controller unit 200 (step S23). In the determination process, the inspection unit 230 determines the object to be measured based on at least one of the synthesized height image data and synthesized texture image data accumulated in the image memory 220 and various threshold values preset in the image memory 220. Determining (inspecting) whether the object S is good or bad. Note that the inspection unit 230 may display the determination result in step S<b>23 on the display unit 320 or may provide it to the external device 400 .

Either of the series of processes from steps S11 to S21 and the texture generation process of step S22 may be executed first, or may be partially executed in parallel.

[4] Accuracy of height image data In the present embodiment, for each of a plurality of portions of the measurement object S, pixels of the height image data corresponding to the portion with respect to the true height of the portion indicate The degree of matching of values (values indicated by height data) is called accuracy of height image data. Height image data is said to be highly accurate if the values of the height image data match or nearly match the true height. On the other hand, when the value of the height image data deviates greatly from the true height, the accuracy of the height image data is considered to be low.

The accuracy of height image data will be described with reference to specific examples. FIG. 6(a) is an external perspective view showing an example of the measurement object S for explaining the accuracy of the height image data, and FIG. 6(b) is a plane of the measurement object S in FIG. 6(a). 6(c) is an end view of the measurement object S of FIG. 6(a).

As shown in FIGS. 6A to 6C, the measurement object S of this example is made of, for example, a metallic material with a glossy surface, and has a base portion s0 and a wall portion W. As shown in FIG. The base portion s0 is formed in a rectangular plate shape extending in one direction and has one end surface s01, the other end surface s02, an upper surface and a lower surface. The wall portion W is formed to protrude upward from a portion of the upper surface of the base portion s0, and has a substantially U-shaped cross section parallel to the upper surface of the base portion s0. A space GS inside the wall portion W is open in a direction from the other end surface s02 to the one end surface s01 of the base portion s0. The top surface of the base part s0 has a uniform height with respect to the bottom surface of the base part s0. Moreover, the upper surface of the wall portion W also has a uniform height with the lower surface of the base portion s0 as a reference.

FIG. 7 is a schematic perspective view showing an example of the positional relationship between the plurality of illumination units 110 and imaging units 120 shown in FIG. 2 and the measurement object S when the measurement object S shown in FIG. 6 is inspected. In FIG. 7, the four illumination units 110A, 110B, 110C, and 110D and the imaging unit 120 shown in FIG. be Also, two straight lines extending parallel to each other in the X direction and the Y direction through the imaging unit 120 are indicated by one-dot chain lines.

The imaging unit 120 is arranged above the belt conveyor 301 so that the imaging field of view faces downward and the optical axis of the optical system of the imaging unit 120 (the light receiving lenses 122 and 123 in FIG. 1) is parallel to the Z direction. ing. On the other hand, the illumination units 110A and 110B are arranged symmetrically so as to sandwich the imaging unit 120 in the X direction at the same or substantially the same height position as the imaging unit 120 in the Z direction. The illumination units 110C and 110D are arranged symmetrically so as to sandwich the imaging unit 120 in the Y direction at the same or approximately the same height position as the imaging unit 120 in the Z direction.

The measurement object S is positioned below the imaging unit 120 and on the optical axis of the optical system of the imaging unit 120 . The measurement object S is arranged parallel to the X direction on the belt conveyor 301 so that one end surface s01 faces the direction of the illumination unit 110A and the other end surface s02 faces the direction of the illumination unit 110B.

When inspecting the measuring object S, the belt conveyor 301 is temporarily stopped while the center of the measuring object S is positioned on the optical axis of the optical system of the imaging unit 120, as shown in FIG. In addition, by the processing of steps S11 to S19 in the above inspection processing, as indicated by thick solid line arrows in FIG. Structured light is applied sequentially. At this time, each structured light reflected from the measurement object S and traveling upward is received by the imaging device 121 of the imaging unit 120 shown in FIG. Thereby, a series of pattern image data corresponding to a series of structured light is generated for each illumination section.

Here, the traveling direction of the structured light emitted from each illumination unit 110 to the measurement object S is inclined with respect to the optical axis of the optical system of the imaging unit 120 . Therefore, depending on the shape of the object S to be measured, there is a possibility that the above non-illuminatable portion may occur for each irradiation direction of the structured light. Alternatively, depending on the surface state of the measurement object S, there is a portion where multiple reflection of structured light occurs (hereinafter referred to as a multiple reflection portion) within the imaging field of view of the imaging unit 120 for each irradiation direction of the structured light. there is a possibility. Height image data with high accuracy cannot be obtained for these portions.

8A is a side view showing a state in which the measurement object S is irradiated with the structured light from one illumination unit 110A in FIG. 7, and FIG. 8B is the structured light of FIG. 8A. It is a figure which shows the height image acquired by irradiation of.

In FIG. 8(a) and FIGS. 9(a), 10(a), and 11(a), which will be described later, the irradiation range of the structured light irradiated from each of the illumination units 110A to 110D onto the measurement object S is hatched. is indicated by In the height images shown in FIG. 8(b) and FIG. 9(b), FIG. 10(b) and FIG. It is represented by the density of the pattern. In this example, the higher the density of the dot pattern, the lower the height of the corresponding portion, and the lower the density of the dot pattern, the higher the height of the corresponding portion. Furthermore, in the height images shown in FIG. 8(b) and later-described FIGS. 9(b), 10(b) and 11(b), portions corresponding to the non-illumination portions are indicated by dark hatching. , and the portions corresponding to the above multiple reflection portions are represented by blanks. The unilluminatable portion in this example is a shadowed portion.

As shown in FIG. 8A, the wall portion W blocks the structured light traveling from the illumination portion 110A toward the other end face s02 of the base portion s0. As a result, as indicated by a thick solid line in FIG. 8A, a non-illumination portion ss is generated in the vicinity of the other end surface s02 of the base portion s0.

Further, as indicated by a two-dot chain line arrow a1 in FIG. 8(a), the structured light traveling from the illumination unit 110A toward one end surface s01 of the base unit s0 is reflected by the one end surface s01 and conveyed to the belt conveyor. It is incident on a portion of the upper surface of 301 in the vicinity of the one end surface s01. Furthermore, as indicated by a two-dot chain line arrow a2 in FIG. and is incident on a portion of the upper surface of the base portion s0. As a result, the belt conveyor 301 and a portion of the upper surface of the base portion s0 generate a multiple reflection portion rr.

As a result, as shown in FIG. 8B, the height image of the illumination unit 110A includes a wall image wi and a base image s0i respectively indicating the heights of the wall W, the base s0, and the top surface of the belt conveyor 301. and the conveyor image 301i, an unilluminated image ssi and a multiple reflected image rri showing the unilluminated portion ss and the multiple reflected portion rr, respectively. The value of each pixel constituting the non-illumination image ssi is set to the illumination invalid value. The display portions of the non-illumination image ssi and the multiple reflection image rri are originally portions where the height of any one of the wall portion W, the base portion s0, and the top surface of the belt conveyor 301 should be displayed. Therefore, the height image data acquired for the illumination section 110A has locally low accuracy.

9A is a side view showing a state in which the measurement object S is irradiated with structured light from another illumination unit 110B in FIG. 7, and FIG. 9B is a side view showing the structured light in FIG. It is a figure which shows the height image acquired by irradiation of. FIG. 10(a) is a side view showing a state in which the measurement object S is irradiated with structured light from still another illumination unit 110C in FIG. 7, and FIG. 10(b) is a structured light shown in FIG. It is a figure which shows the height image acquired by irradiation of light. FIG. 11(a) is a side view showing a state in which the measurement object S is irradiated with structured light from still another illumination unit 110D in FIG. 7, and FIG. 11(b) is a structured light shown in FIG. It is a figure which shows the height image acquired by irradiation of light.

In the examples of FIGS. 9A, 10A, and 11A, similarly to the example of FIG. 8A, structured light is emitted from each of the illumination units 110B, 110C, and 110D to the measurement object S. is irradiated, a non-illumination portion ss and a multiple reflection portion rr are generated on a part of the upper surface of the belt conveyor 301 and the base portion s0. In FIGS. 9(a), 10(a) and 11(a), the unilluminatable portion ss is indicated by a thick line or a thick dotted line. Also, the traveling path of the structured light that causes the multiple reflection portion rr is indicated by a two-dot chain line arrow.

As a result, as shown in FIGS. 9B, 10B and 11B, the height images of the lighting units 110B, 110C and 110D include the wall image wi, the base image s0i and the conveyor Along with image 301i, non-illumination image ssi and multiple reflection image rri are included. Therefore, the height image data acquired for each of the illumination units 110B, 110C, and 110D has locally low accuracy, as is the height image data acquired for the illumination unit 110A.

Here, the positions where the non-illumination portion ss and the multiple reflection portion rr are generated differ according to the irradiation direction of the structured light. Therefore, when the four height image data corresponding to the four illumination units 110A, 110B, 110C, and 110D are compared, the pixel positions of the non-illumination image ssi and the multiple reflection image rri are different from each other.

When combining a plurality of height image data generated for a plurality of lighting units 110, if a component of the height image data with low accuracy remains in the combined height image data, the overall height image data of the combined height image data Less accurate. That is, if the values of a plurality of pixels forming the non-illumination image ssi and the multiple reflection image rri are used to generate the synthetic height image data, the overall accuracy of the synthetic height image data is reduced. Therefore, in the present embodiment, synthesis processing is performed to obtain synthetic height image data with higher accuracy. Details of the synthesizing process executed in step S20 of FIG. 5 will be described below.

[5] Synthesis Processing of Height Image Data FIG. 12 is a diagram showing the concept of synthesis processing according to an embodiment of the present invention. As shown in FIG. 12, in the synthesizing process according to the present embodiment, first, four height image data generated corresponding to the four lighting units 110A to 110D are prepared.

The four height image data are generated using one common imaging unit 120 . Therefore, the four height image data have a plurality of sets of four pixels pi at the same pixel positions corresponding to each other. In the upper part of FIG. 12, partial sets pg1, pg2, pg3, . It is indicated by a thick dashed line and a white arrow.

Next, two values according to a predetermined combination condition are extracted from the values of the four pixels pi at the same corresponding pixel position in each of the sets pg1 to pgn. The combination conditions of this example include the first extraction condition and the second extraction condition, and are stored in advance in the storage unit 134 of FIG. 1 before the measurement object S is inspected.

The first extraction condition is to extract height data excluding illumination invalid values from the values of a plurality of pixels pi in each set pg1 to pgn. As a result, the illumination invalid value component can be eliminated from the values of the plurality of pixels pi.

The second extraction condition is a condition used when a plurality of height data values are extracted under the first extraction condition, and is a difference between two heights among the plurality of extracted height data. are calculated, two height data used for calculating the minimum difference among the plurality of calculated differences are extracted. As a result, two pieces of height data that are considered to be highly accurate can be extracted from the height data of a plurality of pixels pi.

For example, as shown in the middle part of FIG. 12, four height data values va, If vb, vc, vd are set, their values va, vb, vc, vd are sorted in ascending or descending order.

Next, according to the first extraction condition, the values va, vb, vc, and vd excluding the invalid illumination value are extracted as indicating the height for each of the sets pg1 to pgn. Assuming that the illumination invalid value is 0, in the middle example of FIG. vb, vc, vd are extracted.

Next, according to the second extraction condition, the difference di between two heights adjacent to each other is calculated in ascending or descending order for each pair pg1 to pgn, and the two height data used to calculate the smallest difference di. is extracted. In the middle example of FIG. 12, as indicated by the thick dotted lines, values va and vb are extracted for the set pg1, values vc and vd are extracted for the set pg2, and values vc and vd are extracted for the set pg3.

Subsequently, it is determined whether or not the difference between the two height data extracted for each of the sets pg1 to pgn is within a predetermined allowable range. The permissible range is determined in consideration of the accuracy required for inspection of the measurement object S, and is stored in advance in the storage unit 134 of FIG. 1 before the measurement object S is inspected.

In this case, when the difference between the two height data extracted for one set is within the allowable range, the value indicated by the two height data is the portion of the measurement object S corresponding to the set. It is considered to be highly accurate as it is within the allowable error range for the true height. Therefore, when the difference between the two height data extracted for each of the sets pg1 to pgn is a value within the allowable range, height data obtained from the two height data according to a predetermined method is determined. be. Also, the determined height data value is set as a synthetic effective value indicated by the target pixel corresponding to the plurality of pixels of the set in the synthetic height image data. The predetermined method may be a method of calculating an average height from two pieces of height data, or a method of selecting one height from two pieces of height data.

On the other hand, if the difference between the two height data extracted for one set is not within the allowable range, at least one of the two height data is the portion of the measurement object S corresponding to the set. It is considered to be out of the allowable error range for the true height and the accuracy is low. Therefore, if the difference between the two height data extracted for each set pg1 to pgn is not within the allowable range, the value of the height data of the target pixel corresponding to the set in the synthesized height image data is Set to the defined synthetic invalid value. This reduces the amount of low-accuracy components remaining in the combined height image data due to the multiple reflection portions rr. The synthetic invalid value is 0, for example.

In the example in the lower part of FIG. 12, as indicated by the thick solid arrow, the difference between the values va and vb of the pair pg1 is within the allowable range, so that the synthetic effective value based on the values va and vb corresponds to the pair pg1. It is set as the height data value of the target pixel tpi of the composite height image data. In addition, as indicated by the thick two-dot chain line arrow, the difference between the values vc and vd of the group pg2 is outside the allowable range, so that the synthesis invalid value is the target pixel tpi of the synthetic height image data corresponding to the group pg2. Set as height data value. Furthermore, as indicated by the thick dotted arrow, the difference between the values vc and vd of the set pg3 is within the allowable range, so that the composite effective value based on the values vc and vd is the composite height image data corresponding to the set pg3. It is set as the height data value of the target pixel tpi.

By setting the values of all the target pixels tpi of the synthetic height image data corresponding to all the sets pg1 to pgn as described above, the generation of the synthetic height image data is completed, and the synthesis process is started. finish.

In the example of FIG. 12, two height data are extracted according to a predetermined combination condition among the heights indicated by the pixels pi of each set pg1 to pgn. However, if, for example, all the height data of a set of multiple pixels pi are set to the illumination invalid value, the above difference cannot be calculated for the height data. Also, when only one height data among the height data of a plurality of pixels pi in one set indicates a value (effective height value) other than an illumination invalid value, the above-mentioned height data Difference calculation cannot be performed. Therefore, in the present embodiment, when all of the height data of the plurality of pixels pi in one set are set to the lighting invalid value, and when one of the height data of the plurality of pixels pi in the one set Even when only one height data indicates a value other than the illumination invalid value, the value of the target pixel corresponding to the one set in the combined height image data is set to the predetermined combined invalid value.

FIG. 13 is a diagram showing an example of a height image based on synthetic height image data generated by synthesizing processing according to one embodiment of the present invention. The height image of FIG. 13 is obtained by synthesizing a plurality of height image data representing the height images of FIGS. 8(b), 9(b), 10(b) and 11(b). 10 is a height image indicated by combined height image data. As shown in FIG. 13, in the height image generated by the synthesizing process according to the present embodiment, the heights of FIGS. 8B, 9B, 10B and 11B are shown. The multiple reflection image rri that appears in the s image is eliminated. Also, the ratio of the non-illumination image ssi to the entire height image is sufficiently reduced.

FIG. 14 is a flowchart of synthesis processing according to one embodiment of the present invention. At the start of the synthesizing process, a plurality (four in this example) of the lighting units 110A to 110D are obtained in advance in the arithmetic processing unit 132 by the processing of steps S11 to S19 in FIG. It is assumed that image data has been generated.

As shown in FIG. 14, when the synthesizing process is started, the image processing unit 133 of FIG. 1 generates a plurality of sets pg1 to pgn of a plurality of pixels pi at the same pixel positions corresponding to each other in the height image data. , is selected as a set of interest (step S31).

Next, the image processing unit 133 extracts two pieces of height data according to a predetermined combination condition among the pieces of height data of the plurality of (four in this example) pixels pi in the set of interest (step S32). . In this case, as described above, when all of the height data of a plurality of pixels pi are set to illumination invalid values, and when only the height data of one pixel out of a set of a plurality of pixels pi is If a value (effective height value) other than the lighting invalid value is indicated, two heights cannot be extracted.

Therefore, the image processing unit 133 determines whether or not two pieces of height data are extracted in the process of step S32 (step S33). When two height data are extracted, the image processing unit 133 determines whether the difference between the two extracted height data is within a predetermined allowable range (step S34).

If the difference between the two extracted height data is within the allowable range, the image processing unit 133 determines a combined effective value from the two detected height data according to a predetermined method (step S35). . Further, the image processing unit 133 sets the determined combined effective value as the height data value of the target pixel corresponding to the set of interest (step S36).

If the two pieces of height data are not extracted in step S33 above, or if the difference between the two pieces of height data extracted in step S34 above is not within the allowable range, the image processing unit 133 selects The value of the height data of the target pixel is set to the synthesis invalid value (step S37).

After the processing of either step S36 or step S37, the image processing unit 133 determines whether or not all of the plurality of sets pg1 to pgn have been selected as the set of interest (step S38). The fact that all of the plurality of groups pg1 to pgn have been selected as the group of interest means that the values of the height data of all the target pixels in the synthetic height image data have been set, that is, the generation of the synthetic height image data has been completed. means that This completes the compositing process. On the other hand, when all of the plurality of pairs pg1 to pgn have not been selected as the group of interest, the image processing unit 133 newly selects another pair of the plurality of pairs pg1 to pgn that is not the group of interest as the group of interest. (step S39), and the process proceeds to step S32.

[6] Effects of the Embodiments (1) In the inspection apparatus 300 described above, the structured light from each of the plurality of illumination units 110 is irradiated onto the measurement object S a plurality of times while being changed into a plurality of patterns. In the imaging unit 120 , a plurality of pattern image data corresponding to each illumination unit 110 are generated by receiving the structured light reflected from the measurement object S. Height image data is generated for each illumination unit 110 based on a plurality of pattern images corresponding to the illumination unit 110 .

A plurality of height image data for a plurality of lighting units 110 are synthesized to generate synthetic height image data. At the time of this synthesis, for each set pg1 to pgn of a plurality of pixels pi at the same pixel position corresponding to each other in the plurality of height image data, two height data of the plurality of height data indicated by the plurality of pixels pi are selected according to the combination condition. The difference of the height data is calculated. Also, it is determined whether or not the calculated difference is within the allowable range.

If the calculated difference is within the allowable range, the two height data used to calculate the difference are within the allowable error range with respect to the true height of the portion of the measurement object S corresponding to the pixel position. This is highly accurate data indicating a certain height. Therefore, when the calculated difference is within the allowable range, the height data obtained according to a predetermined method is determined from the two height data used to calculate the difference. Also, the determined height data is set as a synthesized effective value indicated by the target pixel corresponding to the pixel position in the synthesized height image data.

On the other hand, if the calculated difference is not within the allowable range, the two height data used to calculate the difference are low-accuracy data indicating a height outside the allowable error range with respect to the true height. Very likely. Therefore, when the calculated difference is not within the allowable range, the value of the height data of the target pixel corresponding to the pixel position in the combined height image data is set to a predetermined combined invalid value.

As a result, in the synthesized height image data finally generated, the value of height data with high accuracy is indicated by the synthetic effective value for each target pixel, but the value of height data with low accuracy is indicated. not. Therefore, by using the generated synthetic height image data, it is possible to improve the accuracy of inspection of the height of the object to be measured.

(2) In the present embodiment, two values are extracted according to a predetermined combination condition in the process of step S32 in the synthesizing process of FIG. The above combination conditions are estimated to be highly accurate from the height data of the plurality of pixels pi so that the illumination invalid value component is excluded from the values of the plurality of pixels pi corresponding to each of the sets pg1 to pgn. Two height data are defined to be extracted. Therefore, by determining the composite effective value based on the two extracted height data, the accuracy of the composite height image data is further improved.

(3) In the present embodiment, illumination units 110A and 110B are arranged to face each other with imaging unit 120 interposed therebetween in the X direction. Also, the illumination units 110C and 110D are arranged so as to face each other with the imaging unit 120 interposed therebetween in the Y direction.

In this case, a wide range of the surface of the measurement object S can be irradiated with the structured light. Therefore, synthetic height image data with high accuracy over a wide range of the surface of the measurement object S can be obtained.

[7] Other Embodiments (1) In the synthesizing process according to the above embodiments, when two height data complying with the combination condition are not extracted from the plurality of heights indicated by the plurality of pixels pi of the set of interest. In addition, the value of the target pixel (height data value) of the synthesized height image data corresponding to the set of interest is set to the invalid synthesis value, but the present invention is not limited to this.

Even if two height data complying with the combination condition are not extracted from the plurality of height data of the plurality of pixels pi in the set of interest, the value of one height data out of the plurality of pixels pi of the set of interest is highly accurate. If it indicates a valid value, the height data value of that one pixel may be determined as the combined valid value.

FIG. 15 is a flowchart of composition processing according to another embodiment. A description will be given of the difference between the composition processing according to another embodiment shown in FIG. 15 and the composition processing shown in FIG. 14 according to the above-described embodiment.

In the synthesizing process of FIG. 15, if two pieces of height data are not extracted in the process of step S33, the image processing unit 133 sets the height data values of all the pixels in the group of interest to the lighting invalid value. It is determined whether or not there is (step S41).

In step S41, when the height data values of all the pixels in the group of interest are set to the illumination invalid value, the image processing unit 133 proceeds to step S37, as in the above embodiment. As a result, the value of the height data of the target pixel corresponding to the group of interest is set to the synthesis invalid value (step S37).

On the other hand, in step S41, if the height data values of all the pixels in the group of interest are not set to the lighting invalid value, the value of the height data of one of the plurality of pixels in the group of interest is There is a possibility that the height of the portion of the measurement object S corresponding to the set of interest is indicated accurately. For example, the measurement object S made of rubber or the like may have a surface condition that makes it difficult for multiple reflections to occur. Regarding such a measurement object S, the value of each pixel in the set of interest is highly likely to accurately indicate the height of the portion of the measurement object S corresponding to the set of interest unless it is an illumination invalid value. .

Therefore, in this example, if the values of all the pixels in the group of interest are not set to the illumination invalid value in step S41, the image processing unit 133 calculates the height data of one pixel among the plurality of pixels in the group of interest. is determined as a synthesized effective value (step S42). After that, the image processing unit 133 proceeds to the process of step S36.

According to the synthesizing process of FIG. 15, the number of target pixels for which the invalid synthesizing value is set can be reduced in the synthetic height image data.

The inspection apparatus 300 of this example may be configured such that the user can select the combining process of FIG. 14 and the combining process of FIG. In this case, for example, the controller unit 200 in FIG. 1 accepts selection of the combining process to be executed from among the combining processes in FIGS. 14 and 15 in response to the operation of the operation unit 310 by the user. In addition, the image processing unit 133 executes a synthesis process selected by the user when the measurement object S is inspected. This makes it possible to generate appropriate synthetic height image data according to the shape, material, surface state, and the like of the object S to be measured.

(2) In the synthesizing process according to the above embodiment, two height data values according to the combination condition are extracted from the values of the plurality of pixels pi in each of the sets pg1 to pgn. Here, the combination condition may be to extract two height data values corresponding to two specific lighting units 110 from the values of a plurality of pixels pi in each set pg1 to pgn.

For example, when a plurality of pairs of illumination units 110 facing each other are provided in the inspection apparatus 300, the height data values of two pixels pi corresponding to the two illumination units 110 facing each other in one predetermined direction are may be extracted. Alternatively, when a plurality of pairs of illumination units 110 facing each other are provided in the inspection apparatus 300, the difference between the height data values of two pixels pi corresponding to each pair of illumination units 110 is calculated, and the difference is Values of height data of two pixels pi that are the smallest may be extracted.

In such a combination condition, if there is another height data value belonging to the group between the two extracted height data values, the two extracted height data values Additionally, the composite effective value may be determined by using other height data values. In this case, the composite valid value is determined based on the three height data values. In this way, the composite effective value may be determined based on two or more height data values.

(3) Although inspection apparatus 300 according to the above embodiment has four illumination units 110 for one imaging unit 120, the present invention is not limited to this. The inspection apparatus 300 may have four or more illumination units 110 for one imaging unit 120 . Therefore, inspection device 300 may have five or six illumination units 110 . Even in this case, by performing the synthesis process in the same manner as in the above embodiment, it is possible to improve the accuracy of the inspection of the height of the object to be measured.

[8] Correspondence between each component of the claims and each part of the embodiment An example of correspondence between each component of the claims and each part of the embodiment will be described. In the above embodiment, the measurement object S is an example of the measurement object, the illumination unit 110A is an example of the first illumination unit, the illumination unit 110B is an example of the second illumination unit, and the illumination unit 110C is an example of a third lighting section, and lighting section 110D is an example of a fourth lighting section.

The imaging unit 120 is an example of an imaging unit, the imaging processing unit 131 and the arithmetic processing unit 132 are examples of a data generation unit, the image processing unit 133 is an example of an image processing unit, and the X direction (or Y direction) is an example of an image processing unit. ) is an example of the first direction, the Y direction (or X direction) is an example of the second direction, the combination condition is an example of a predetermined condition, and the combined invalid value is an example of an invalid value be.

Various other elements having the structure or function described in the claims can be used as each component of the claims.
[9] Reference form
(1) An inspection apparatus according to a reference embodiment includes a first illumination unit, a second illumination unit, and a third illumination unit that irradiate an object to be measured while changing structured light from four different directions into a plurality of patterns. and a fourth illumination unit, and receive structured light reflected from the object to be measured to generate a plurality of first pattern image data corresponding to the first illumination unit, and to the second illumination unit generating a plurality of corresponding second pattern image data, generating a plurality of third pattern image data corresponding to the third illumination section, and generating a plurality of fourth pattern image data corresponding to the fourth illumination section; an imaging unit that generates first height image data of the measurement object based on the plurality of first pattern image data; and generates a second height image data of the measurement object based on the plurality of second pattern image data; generating height image data of the measurement object based on the plurality of third pattern image data; generating third height image data of the measurement object based on the plurality of fourth pattern image data; a data generation unit that generates fourth height image data; and an image processing unit that generates combined height image data by combining the first to fourth height image data, the image processing unit comprising: Two height data according to a predetermined condition are extracted from the height data of the same pixel positions corresponding to each other in the first to fourth height image data, and the difference between them is calculated. Determining whether the difference is within a predetermined allowable range, and height data of at least one of the two height data used to calculate the difference if the calculated difference is within the allowable range Determine the height data at the pixel position of the composite height image data based on the above, and set the height data at the pixel position of the composite height image data to an invalid value if the calculated difference is not within the allowable range. do.
In the inspection apparatus, the object to be measured is irradiated a plurality of times with structured light from each of the first to fourth illumination units while being changed into a plurality of patterns. The imaging unit receives structured light reflected from the object to be measured. Thereby, a plurality of first pattern image data corresponding to the first illumination section are generated, and the first height image data are generated based on the plurality of first pattern image data. Also, a plurality of second pattern image data corresponding to the second illumination units are generated, and second height image data is generated based on the plurality of second pattern image data. Also, a plurality of third pattern image data corresponding to the third illumination unit are generated, and third height image data is generated based on the plurality of third pattern image data. Also, a plurality of fourth pattern image data corresponding to the fourth illumination section are generated, and fourth height image data is generated based on the plurality of fourth pattern image data.
The first to fourth height image data are synthesized to generate synthesized height image data. At the time of this synthesis, two height data according to a predetermined condition are extracted from the height data of the same pixel positions corresponding to each other in the first to fourth height image data, and the difference between them is calculated. . Also, it is determined whether or not the calculated difference is within the allowable range.
Here, when the calculated difference is within the allowable range, the two height data used for calculating the difference are within the allowable error range with respect to the true height of the portion of the measurement object corresponding to the pixel position. This is highly accurate data that indicates the height within. Therefore, when the calculated difference is within the allowable range, based on at least one height data of the two height data used to calculate the difference, the pixel position of the synthetic height image data is calculated. Height data is determined.
On the other hand, if the calculated difference is not within the allowable range, the two height data used to calculate the difference are low-accuracy data indicating a height outside the allowable error range with respect to the true height. Very likely. Therefore, if the calculated difference is not within the allowable range, the height data at the pixel position is set to an invalid value in the combined height image data.
As a result, at each pixel position of the synthetic height image data, the value of height data with high accuracy is indicated, but the value of height data with low accuracy is not indicated. Therefore, by using the generated synthetic height image data, it is possible to improve the accuracy of inspection of the height of the object to be measured.
(2) The predetermined condition is the calculation of the minimum difference among the plurality of calculated differences when the difference between each two height data of the height data at the same pixel position is calculated. It may be to extract the two height data used for .
The two height data extracted according to the above conditions are considered to have particularly high accuracy among the height data at the same pixel position. Therefore, the accuracy of synthetic height image data is further improved.
(3) If the difference between the two height data in accordance with a predetermined condition among the height data at the same pixel position is within the allowable range, the image processing unit The average value of the height data or one of the two height data values may be determined as the height data value at the pixel position of the composite height image data.
In this case, the value of the height data at each pixel position of the synthetic height image data is calculated easily and appropriately.
(4) The first illumination unit and the second illumination unit are arranged to face each other with the imaging unit interposed in the first direction, and the third illumination unit and the fourth illumination unit are arranged in the first direction. They may be arranged so as to face each other across the imaging unit in a second direction that intersects with the first direction.
In this case, it is possible to irradiate the structured light over a wide range of the surface of the object to be measured. Therefore, synthetic height image data with high accuracy can be acquired over a wide range of the surface of the object to be measured.
(5) The data generation unit, when generating each of the first to fourth height image data, based on a plurality of pattern image data generated corresponding to the illumination unit and a predetermined light reception determination condition. determining whether or not a portion of the measurement object corresponding to each pixel position of the height image data to be generated is an unilluminatable portion that cannot reflect the structured light to the imaging unit; The height data of the pixel position corresponding to the non-illumination portion is set as the illumination invalid value, and the image processing unit calculates the difference between the two height data according to the predetermined condition without using the illumination invalid value. good.
In this case, the height data values of the pixel positions indicating the unilluminated portion in the first to fourth height image data are not used for generating the synthetic height image data. This prevents unstable height components due to non-illumination areas from remaining in the synthesized height image data.
(6) When only one height data out of four height data at each pixel position indicates a value other than an illumination invalid value and the other height data is set to an illumination invalid value In addition, it may be possible to further select whether to set one height data as the height data of the pixel position or set the value of the height data of the pixel position to an invalid value.
In this case, appropriate synthetic height image data can be generated according to the shape, material, surface condition, etc. of the object to be measured.

DESCRIPTION OF SYMBOLS 100... Head part 110,110A-110D... Illumination part, 111-113... Light source, 114, 115... Dichroic mirror, 116... Illumination lens, 117... Mirror, 118... Pattern generation part, 119... Projector lens, 120, Reference numerals 120A to 120D: Imaging unit 121: Imaging element 122, 123: Light receiving lens 130: Calculation unit 131: Image pickup processing unit 132: Calculation processing unit 133: Image processing unit 134: Storage unit 135: Output DESCRIPTION OF SYMBOLS Processing part 200... Controller part 210... Head control part 220... Image memory 230... Inspection part 300... Inspection apparatus 301... Belt conveyor 301i... Conveyor image 310... Operation part 320... Display part 400 External device, pg1 to pgn, group, pi, pixel, rr, multiple reflection portion, rri, multiple reflection image, S, object to be measured, s0, base portion, s01, one end surface, s02, other end surface, s0i, base Image, ss... non-illumination portion, ssi... non-illumination image, tpi... target pixel, W... wall portion, wi... wall image

Claims (6)

a first illumination unit, a second illumination unit, a third illumination unit, and a fourth illumination unit that irradiate an object to be measured while changing the structured light from four different directions into a plurality of patterns;
generating a plurality of first pattern image data corresponding to the first illumination unit by receiving structured light reflected from the measurement target; and generating a plurality of second pattern image data corresponding to the second illumination unit , generating a plurality of third pattern image data corresponding to the third lighting unit, and generating a plurality of fourth pattern image data corresponding to the fourth lighting unit Department and
generating first height image data of the measurement object based on the plurality of first pattern image data, and generating second height image data of the measurement object based on the plurality of second pattern image data; , generating third height image data of the measurement object based on the plurality of third pattern image data, and generating a fourth height image data of the measurement object based on the plurality of fourth pattern image data a data generator that generates height image data;
an image processing unit that generates synthesized height image data by synthesizing the first to fourth height image data,
The image processing unit extracts two height data according to a predetermined condition from the height data of the same pixel positions corresponding to each other in the first to fourth height image data, and calculates the difference between them. It is determined whether the calculated difference is within a predetermined allowable range, and if the calculated difference is within the allowable range, the two heights used to calculate the difference are determining the height data of the pixel position of the synthetic height image data based on the height data of at least one of the height data, and determining the synthetic height image if the calculated difference is not within the allowable range; Set the height data of the pixel position in the data to an invalid value ,
The predetermined condition is used to calculate the minimum difference among a plurality of calculated differences when the difference between each two height data among the height data at the same pixel position is calculated. The inspection device is to extract the two height data obtained . If the difference between the two height data according to a predetermined condition among the height data at the same pixel position is within the allowable range, the image processing unit calculates the difference between the two height data used to calculate the difference. 2. The inspection apparatus according to claim 1 , wherein an average value of height data or one of two height data values is determined as the height data value at the pixel position of said combined height image data. a first illumination unit, a second illumination unit, a third illumination unit, and a fourth illumination unit that irradiate an object to be measured while changing the structured light from four different directions into a plurality of patterns;
generating a plurality of first pattern image data corresponding to the first illumination unit by receiving structured light reflected from the measurement target; and generating a plurality of second pattern image data corresponding to the second illumination unit , generating a plurality of third pattern image data corresponding to the third lighting unit, and generating a plurality of fourth pattern image data corresponding to the fourth lighting unit Department and
generating first height image data of the measurement object based on the plurality of first pattern image data, and generating second height image data of the measurement object based on the plurality of second pattern image data; , generating third height image data of the measurement object based on the plurality of third pattern image data, and generating a fourth height image data of the measurement object based on the plurality of fourth pattern image data a data generator that generates height image data;
an image processing unit that generates synthesized height image data by synthesizing the first to fourth height image data,
The image processing unit extracts two height data according to a predetermined condition from the height data of the same pixel positions corresponding to each other in the first to fourth height image data, and calculates the difference between them. It is determined whether the calculated difference is within a predetermined allowable range, and if the calculated difference is within the allowable range, the two heights used to calculate the difference are determining the average value of the height data or one of the two height data values as the value of the height data at the pixel position of the composite height image data , and the calculated difference is not within the allowable range; setting the height data of the pixel position of the composite height image data to an invalid value if the height image data is in the above-mentioned case . a first illumination unit, a second illumination unit, a third illumination unit, and a fourth illumination unit that irradiate an object to be measured while changing the structured light from four different directions into a plurality of patterns;
generating a plurality of first pattern image data corresponding to the first illumination unit by receiving structured light reflected from the measurement target; and generating a plurality of second pattern image data corresponding to the second illumination unit , generating a plurality of third pattern image data corresponding to the third lighting unit, and generating a plurality of fourth pattern image data corresponding to the fourth lighting unit Department and
generating first height image data of the measurement object based on the plurality of first pattern image data, and generating second height image data of the measurement object based on the plurality of second pattern image data; , generating third height image data of the measurement object based on the plurality of third pattern image data, and generating a fourth height image data of the measurement object based on the plurality of fourth pattern image data a data generator that generates height image data;
an image processing unit that generates synthesized height image data by synthesizing the first to fourth height image data,
The image processing unit extracts two height data according to a predetermined condition from the height data of the same pixel positions corresponding to each other in the first to fourth height image data, and calculates the difference between them. It is determined whether the calculated difference is within a predetermined allowable range, and if the calculated difference is within the allowable range, the two heights used to calculate the difference are determining the height data of the pixel position of the synthetic height image data based on the height data of at least one of the height data, and determining the synthetic height image if the calculated difference is not within the allowable range; Set the height data of the pixel position in the data to an invalid value,
The data generation unit, when generating each of the first to fourth height image data, based on a plurality of pattern image data generated corresponding to the illumination unit and a predetermined light reception determination condition. determines whether or not a portion of the object to be measured corresponding to each pixel position of the height image data to be generated is an unilluminatable portion where structured light cannot be reflected to the imaging unit. , setting the height data of the pixel position corresponding to the non-illumination portion to an illumination invalid value;
The inspection apparatus, wherein the image processing unit does not use the illumination invalid value for calculating a difference between two pieces of height data according to the predetermined condition. When only one height data out of four height data at each pixel position indicates a value other than the illumination invalid value and the other height data is set to the illumination invalid value and ( 4 ) further selectable between setting said one height data as the height data of said pixel position and setting the height data value of said pixel position to said invalid value. Inspection equipment as described. the first illumination unit and the second illumination unit are arranged to face each other with the imaging unit interposed in a first direction;
The third illumination unit and the fourth illumination unit are arranged to face each other with the imaging unit interposed therebetween in a second direction that intersects with the first direction. An inspection device according to any one of the preceding paragraphs.

Images