JP2020128933A - Inspection device - Google Patents

Abstract

To provide an inspection device with which it is possible to improve the accuracy of inspection with regard to the height of a measurement object.SOLUTION: Lighting units 110A, 110B face each other across an imaging unit 120, and lighting units 110C, 110D face each other across the imaging unit 120. A measurement object S is sequentially irradiated with structured light from the lighting units 110A-110D. A plurality of height data corresponding to the lighting units 110A-110D are generated on the basis of the structured light reflected from the measurement object S. The first pair of height data corresponding to the lighting units 110A, 110D are synthesized, and first intermediate height data is generated. The second pair of height data corresponding to the lighting units 110B, 110C are synthesized, and second intermediate height data is generated. The first and second reliability of the first and second intermediate height data are calculated for a plurality of portions of a height image. The first and second intermediate data are synthesized on the basis of the first and second reliability.SELECTED DRAWING: Figure 6

Description

The present invention relates to an inspection device that inspects the height of an object to be measured.

In the triangulation type inspection apparatus, for example, the surface of an object to be measured is irradiated with structured light by a light projecting unit, and the reflected light is received by a light receiving unit having pixels arranged one-dimensionally or two-dimensionally. .. Height data indicating a height image of the measurement target is generated based on the data of the received light amount distribution obtained by the light receiving unit.

Regarding the generation of height data, it is impossible to measure on the surface of the measurement object depending on the irradiation direction of the structured light to the measurement object, the receiving direction of the structured light reflected from the measurement object or the shape of the measurement object. There is a possibility that some parts will occur.

Therefore, structured light is sequentially emitted toward a measurement target in a plurality of directions, and structured light reflected in one direction by the measurement target is sequentially received, so that a plurality of high light beams corresponding to a plurality of irradiation directions are respectively received. There is a method to generate the data and combine them. Alternatively, a plurality of height data corresponding to a plurality of light receiving directions can be obtained by irradiating the structured light in one direction toward the measurement target and receiving the structured light reflected from the measurement target in a plurality of directions, respectively. There is a way to generate and synthesize them. By combining a plurality of height data respectively corresponding to a plurality of directions, height data in which unmeasurable portions are reduced is generated.

For example, in the three-dimensional measuring device described in Patent Document 1, structured light is emitted from one projector to a measurement target. The measuring object irradiated with the structured light is imaged by a plurality of imaging means arranged at a plurality of positions. The three-dimensional point cloud (height data) of the measurement target is measured using the image obtained by each image pickup means. The measurement results of a plurality of three-dimensional point groups respectively corresponding to a plurality of image pickup means are integrated.

Patent No. 5633058

When integrating the measurement results of multiple 3D point clouds, the reliability of each pixel is compared among multiple 3D point clouds, and the measurement result with high reliability is the final measurement for that pixel. It remains as a candidate for the result. The final measurement result is obtained from the measurement result left for each pixel.

In the technique described in Patent Document 1, the comparison of the reliability is performed by comparing the amplitudes of the luminance of each pixel obtained during the phase analysis using the structured light. Specifically, for a certain pixel, the measurement result corresponding to the image pickup unit that is determined to have a large luminance amplitude among the plurality of image pickup units is left.

In this case, for example, it is possible to reduce the adoption of a saturated measurement value due to the occurrence of halation and the adoption of a noise level measurement value due to the generation of a shadow. However, even if the amplitude of the luminance for each pixel is used as described above, it may not be possible to perform appropriate reliability comparison.

For example, depending on the shape, surface state, and material of the measurement object, multiple reflection may occur on the surface of the measurement object when the structured light is irradiated. In this case, the amplitude of luminance may not be reduced (not attenuated) in the portion where the multiple reflection occurs. Therefore, if the measurement result including the component of multiple reflection is left as a final measurement result candidate for the portion, the accuracy of the inspection of the measurement target is deteriorated.

It is an object of the present invention to provide an inspection device that makes it possible to improve the accuracy of inspection for the height of a measurement target.

(1) An inspection apparatus according to a first aspect of the present invention includes a plurality of irradiation units that irradiate a measurement target with structured light in a plurality of directions, and a plurality of structured lights respectively irradiated from the plurality of irradiation units. A plurality of irradiation units based on the one image capturing unit that sequentially generates a plurality of pattern image data indicating the image of the measurement target by receiving light and the pattern image data generated corresponding to each of the plurality of irradiation units. The image processing for generating the combined height data by combining the data generation section for generating a plurality of height data respectively indicating the height images of the measurement target and the plurality of height data for a plurality of irradiation sections. A plurality of irradiation units, and a plurality of irradiation units in a second direction intersecting the first direction with a first pair of irradiation units arranged so as to face each other across the one imaging unit in the first direction. The image processing unit includes a second pair of irradiation units arranged so as to face each other with one imaging unit interposed therebetween, and the image processing unit combines the first pair of height data generated for the first pair of irradiation units. The first intermediate height data generation unit that generates the first intermediate height data by this, and the reliability of the generated first intermediate height data for each of the plurality of portions of the height image are A second reliability that generates a second intermediate height data by synthesizing a first reliability calculation unit that calculates the second reliability and a second pair of height data that is generated for each of the second pair of irradiation units. And a second reliability calculation unit that calculates the reliability of the generated second intermediate height data as the second reliability for each of the plurality of portions of the height image. , A selection process for selecting one of the first and second intermediate height data corresponding to each other based on the first and second reliability corresponding to each other for each of the plurality of portions of the height image Or a combined height data generation unit for generating combined height data by performing weighting processing with weighting of the first and second intermediate height data corresponding to each other.

In the inspection apparatus, the structured light is emitted toward the measurement target in a plurality of directions by the plurality of irradiation units, and each structured light reflected by the measurement target is received by one imaging unit. Thereby, a plurality of height data of the measuring object for a plurality of irradiation units are generated. The height data generated for each irradiation unit includes an error component due to the direction of irradiation or reception of structured light and the shape and surface state of the measurement target.

The first pair of height data among the plurality of height data is combined to generate the first intermediate height data. The first pair of irradiation units respectively irradiate the measurement object with structured light in mutually opposite directions. As a result, when the first intermediate height data is generated, the error components included in each of the first pair of height data cancel each other out. Therefore, the first intermediate height data has higher accuracy than the height data generated for each irradiation unit. Further, the reliability of the first intermediate height data for each of the plurality of portions of the height image is calculated as the first reliability. The first reliability can be calculated based on the first pair of height data. As a result, a more effective first reliability can be calculated as compared with the case where the reliability is calculated using only height data generated for either one of the first pair of irradiation units.

A second pair of height data among the plurality of height data is combined to generate second intermediate height data. The second pair of irradiation units respectively irradiate the measurement object with structured light in mutually opposite directions. As a result, when the second intermediate height data is generated, the error components included in each of the second pair of height data are canceled by each other. Therefore, the second intermediate height data has higher accuracy than the height data generated for each irradiation unit. Further, the reliability of the second intermediate height data for each of the plurality of portions of the height image is calculated as the second reliability. The second reliability can be calculated based on the height data of the second pair. As a result, a more effective second reliability can be obtained as compared with the case where the reliability is calculated using only the height data generated for either one of the second pair of irradiation units.

For each of the plurality of portions of the height image, the selection processing or the weighting processing of the first and second intermediate height data corresponding to each other is performed based on the first and second reliability corresponding to each other that are effectively calculated. Is done. Thereby, the first and second intermediate height data having high accuracy for each of the plurality of portions of the height image are appropriately combined to generate the combined height data. As a result, by using the generated combined height data, it is possible to improve the accuracy of the inspection regarding the height of the measuring object.

(2) The first intermediate height data generation unit generates the first intermediate height data by averaging and combining the first pair of height data for each of the plurality of portions of the height image. The second intermediate height data generation unit may generate the second intermediate height data by averaging and synthesizing the second pair of height data for each of the plurality of portions of the height image. Good.

In this case, the error component included in each of the first pair of height data is appropriately canceled when the first intermediate height data is generated. Further, when the second intermediate height data is generated, the error component included in each of the second pair of height data is appropriately canceled. Therefore, the accuracy of the first and second intermediate height data is further improved.

(3) The first reliability calculation unit calculates, for each of the plurality of portions of the height image, a value related to the difference between the height data of the first pair as the first reliability of the portion, and the second reliability is calculated. The reliability calculation unit may calculate a value regarding the difference between the second pair of height data for each of the plurality of portions of the height image as the second reliability of the portion.

For each of the plurality of parts of the height image, the height data of the first pair is more accurate as the difference between the height data is smaller, and the accuracy of at least one of the height data is larger as the difference between the height data is larger. Is considered to be low. In this way, the value regarding the difference between the height data of the first pair can be used as an evaluation value indicating the accuracy of the height data of the first pair. Similarly, for each of the plurality of portions of the height image, the second pair of height data has a higher accuracy as the difference between the height data is smaller, and at least one as the difference between the height data is larger. The accuracy of is considered to be low. Thus, the value related to the difference between the height data of the second pair can be used as an evaluation value indicating the accuracy of the height data of the second pair. Therefore, according to the above configuration, the effective first reliability and the effective second reliability can be appropriately calculated by a simple process.

(4) The inspection device sets the permissible condition in which the height data of a plurality of portions of the height image of the irradiation unit is predetermined based on the pattern image data generated corresponding to each of the irradiation units. The first intermediate height data generation unit further includes a permissible condition determination unit that determines whether or not the conditions are satisfied, and the first pair of height data together satisfies the permissible condition for each of the plurality of portions of the height image. In such a case, the height data are averaged, and when one of the height data of the first pair satisfies the allowance condition and the other does not satisfy the allowance condition, one height data is selected and the height data of the first pair is selected. If none of the height data satisfies the allowance condition, the height data of the portion is invalidated to generate the first intermediate height data, and the second intermediate height data generation unit is configured to generate the height image. For each of the plurality of parts of the height data, if the height data of the second pair both satisfy the acceptance condition, the height data are averaged, and one of the height data of the second pair satisfies the acceptance condition and the other By selecting one height data when the permissible condition is not satisfied, and invalidating the height data of the relevant part when none of the height data of the second pair satisfies the permissible condition, the second intermediate data is obtained. Height data may be generated.

According to the above configuration, when both the height data of the first pair satisfy the allowance condition, the error component included in each of the height data of the first pair is appropriately canceled when the height data of the first pair is generated. Be done. Further, when both the height data of the second pair satisfy the allowance condition, the error component included in each of the height data of the second pair is appropriately canceled when the second intermediate height data is generated. On the other hand, when both the height data of the first pair do not satisfy the allowance condition, it is prevented that the components of the height data of the first pair that do not meet the allowance condition are included in the first intermediate height data. It Further, when both the height data of the second pair do not satisfy the allowance condition, it is prevented that the components of the height data of the second pair that do not meet the allowance condition are included in the second intermediate height data. It These results produce synthetic height data with greater accuracy.

(5) For each of the plurality of portions of the height image, the first reliability calculation unit determines a value relating to the difference between the height data when the height data of the first pair both satisfy the acceptance condition. As the first reliability of the first pair of height data, and if at least one of the height data of the first pair does not satisfy the permissible condition, a value indicating that the reliability of the part is the lowest is determined as the first reliability. , The second reliability calculation unit determines, for each of the plurality of portions of the height image, a value relating to the difference between the height data of the second pair when the height data of the second pair both satisfy the acceptance condition. Even if a value indicating that the reliability of the relevant portion is the lowest is determined as the second reliability when at least one of the height data of the second pair does not satisfy the allowance condition. Good.

In this case, it is possible to reduce inclusion of height data components that do not satisfy the allowance condition in the combined height data. Therefore, it is possible to obtain composite height data with higher accuracy.

(6) The inspection device sets the reliability condition in which the first and second reliability calculated by the first and second reliability calculation units are predetermined for each of the plurality of portions of the height image. A reliability condition determining unit that determines whether or not the condition is satisfied, and the combined height data generating unit determines whether the first and second reliability conditions satisfy the reliability condition for each of a plurality of portions of the height image. The first and second intermediate height data are averaged, and the combined height data is generated by performing the selection processing when the first and second reliability do not satisfy the reliability condition. The degree condition is that the difference between the first and second reliability values corresponding to each other is within a predetermined difference range, or the first and second reliability values corresponding to each other are both determined from a predetermined reference reliability value. May be higher.

Accuracy of the first and second intermediate height data corresponding to the first and second reliability, when the difference between the first and second reliability corresponding to each other is within a predetermined difference range. Are considered to be approximately equal. Further, when the first and second reliability levels corresponding to each other are higher than a predetermined reference reliability level, it is considered that the accuracy of the first and second intermediate height data for the relevant part is high. .. Therefore, when the first and second reliability levels satisfy the reliability level, the first and second intermediate height data are averaged to improve the accuracy of the combined height data for the portion. ..

On the other hand, when the first and second reliability levels corresponding to each other do not satisfy the reliability level condition, the high accuracy of the first and second intermediate height data corresponding to the first and second reliability levels is used. One data having the property is selected as the combined height data. This reduces the inclusion of height data components with low accuracy in the composite height data.

(7) The combined height data generation unit performs a weighted averaging process on the first and second intermediate height data based on the first and second reliability for each of the plurality of portions of the height image. The combined height data may be generated by.

In this case, the first and second intermediate height data are appropriately combined according to the first and second reliability. This improves the accuracy of the composite height data.

(8) The inspection device according to the second aspect of the present invention includes an irradiation unit that irradiates the measurement target with structured light, and the structured light reflected from the measurement target in a plurality of directions to measure the measurement target. Based on the plurality of imaging units that respectively generate a plurality of pattern image data representing the image of the object and the pattern image data generated corresponding to each of the plurality of imaging units, A data generation unit that generates a plurality of height data each indicating a height image, and an image processing unit that generates a combined height data by combining a plurality of height data for a plurality of imaging units, Of the first imaging unit and the imaging unit of the first pair arranged so as to face each other with the one illumination unit sandwiched in the first direction, and the one illumination unit sandwiched in the second direction intersecting the first direction. And a second pair of image pickup units arranged so as to face each other, the image processing unit combines the first pair of height data generated for the first pair of image pickup units to obtain the first intermediate image data. A first intermediate height data generation unit that generates height data and the reliability of the generated first intermediate height data for each of the plurality of portions of the height image are calculated as the first reliability. Second intermediate height data generation for generating second intermediate height data by synthesizing the first reliability calculation unit and the second pair of height data generated for the second pair of imaging units, respectively. A second reliability calculation unit that calculates the reliability of the generated second intermediate height data as the second reliability for each of the plurality of height images, and the plurality of height images. Selection processing of selecting one of the first and second intermediate height data corresponding to each other based on the first and second reliability corresponding to each other, or the first corresponding to each other. And a combined height data generation unit that generates combined height data by performing weighting processing involving weighting of the second intermediate height data.

In the inspection device, the structured light is emitted toward the measurement target by one irradiation unit, and the structured light reflected by the measurement target is received by the plurality of imaging units in the plurality of directions. As a result, a plurality of height data of the measuring object for the plurality of imaging units are generated. The height data generated for each imaging unit includes an error component due to the direction of irradiation or reception of structured light and the shape and surface state of the measurement target.

The first pair of height data among the plurality of height data is combined to generate the first intermediate height data. The first pair of imaging units respectively receive the structured lights reflected in the opposite directions from the target object. As a result, when the first intermediate height data is generated, the error components included in each of the first pair of height data cancel each other out. Therefore, the first intermediate height data has higher accuracy than the height data generated for each imaging unit. Further, the reliability of the first intermediate height data for each of the plurality of portions of the height image is calculated as the first reliability. The first reliability can be calculated based on the first pair of height data. As a result, a more effective first reliability can be calculated as compared with the case where the reliability is calculated using only height data generated for one of the first pair of imaging units.

A second pair of height data among the plurality of height data is combined to generate second intermediate height data. The second pair of imaging units respectively receive the structured lights reflected in opposite directions from the measurement target. As a result, when the second intermediate height data is generated, the error components included in each of the second pair of height data are canceled by each other. Therefore, the second intermediate height data has higher accuracy than the height data generated for each imaging unit. Further, the reliability of the second intermediate height data for each of the plurality of portions of the height image is calculated as the second reliability. The second reliability can be calculated based on the height data of the second pair. As a result, a more effective second reliability can be obtained as compared with the case where the reliability is calculated using only height data generated for one of the second pair of imaging units.

For each of the plurality of portions of the height image, the selection processing or the weighting processing of the first and second intermediate height data corresponding to each other is performed based on the first and second reliability corresponding to each other that are effectively calculated. Is done. Thereby, the first and second intermediate height data having high accuracy for each of the plurality of portions of the height image are appropriately combined to generate the combined height data. As a result, by using the generated combined height data, it is possible to improve the accuracy of the inspection regarding the height of the measuring object.

According to the present invention, the accuracy of the inspection of the height of the measuring object is improved.

It is a block diagram which shows the structure of the inspection apparatus which concerns on 1st Embodiment. It is a block diagram which shows the structure of the inspection apparatus which concerns on 1st Embodiment. It is a figure which shows an example of a structure of each illumination part of FIG. It is a figure for explaining the principle of the triangulation method. 6 is a flowchart showing an example of an inspection process executed by the inspection device of FIG. 1. FIG. 3 is a schematic perspective view showing a positional relationship between a plurality of illumination units and imaging units in FIG. 2 and a measurement target when inspecting the measurement target. FIG. 7A is a side view showing a state in which structured light is irradiated onto the measurement object from the one illumination unit in FIG. 6, and FIG. 6B is a height obtained by the structured light irradiation in FIG. It is a figure which shows an image. FIG. 7A is a side view showing a state in which structured light is irradiated onto the measurement object from the other illuminating unit in FIG. 6, and FIG. 6B is a height obtained by the structured light irradiation in FIG. It is a figure which shows an image. FIG. 3 is a block diagram for explaining a detailed configuration of an arithmetic unit according to the first embodiment. 6 is a flowchart of a combining process according to the first embodiment. It is a block diagram for explaining a detailed configuration of a calculation unit according to the second embodiment. 9 is a flowchart of a combining process according to the second embodiment. It is a block diagram for explaining a detailed configuration of a calculation unit according to the third embodiment. 9 is a flowchart of a combining process according to the third embodiment. It is a flow chart of composition processing concerning a 4th embodiment. It is a block diagram which shows the structure of the inspection apparatus which concerns on other embodiment.

Hereinafter, an inspection device according to an embodiment of the present invention will be described with reference to the drawings.

[1] First Embodiment (1) Configuration of Inspection Apparatus FIGS. 1 and 2 are block diagrams showing the configuration of an inspection apparatus according to the first embodiment. As shown in FIGS. 1 and 2, the inspection device 300 includes a head unit 100, a controller unit 200, an operation unit 310, and a display unit 320. The controller unit 200 is connected to an external device 400 such as a programmable logic controller.

As indicated by thick arrows in FIG. 1, a plurality of measurement objects S are sequentially conveyed by the belt conveyor 301 so as to pass through the space below the head unit 100. When each measurement object S passes through the space below the head unit 100, the belt conveyor 301 is kept for a certain time so that the measurement object S temporarily stops at a predetermined position below the head unit 100. Stop.

The head unit 100 is, for example, an image pickup device integrated with light projection and reception, and includes a plurality of illumination units 110, an image pickup unit 120, and a calculation unit 130. Note that the illustration of the calculation unit 130 is omitted in FIG. 2. In this embodiment, four illumination units 110 are provided so as to surround the imaging unit 120 at intervals of 90 degrees.

Each illuminating section 110 is configured to be capable of irradiating red, blue, green or white light having an arbitrary pattern while obliquely shifting the measurement target S from the upper side. In addition, each illumination unit 110 is configured to be capable of irradiating the measurement object S with uniform light of red, blue, green, or white having no pattern from diagonally above. Hereinafter, light having an arbitrary pattern is referred to as structured light, and uniform light is referred to as uniform light.

Further, when distinguishing the four illumination units 110, the four illumination units 110 are referred to as illumination units 110A, 110B, 110C, and 110D, respectively. The illuminating section 110A and the illuminating section 110B face each other with the image capturing section 120 interposed therebetween. The illumination unit 110C and the illumination unit 110D are opposed to each other with the imaging unit 120 interposed therebetween. The configuration of the illumination unit 110 will be described later.

The imaging unit 120 includes an imaging element 121 and light receiving lenses 122 and 123 as shown in FIG. At least the light receiving lens 122 of the light receiving lenses 122 and 123 is a telecentric lens. The structured light reflected upward by the measuring object S is collected and imaged by the light receiving lenses 122 and 123 of the image pickup unit 120, and then received by the image pickup device 121. The image sensor 121 is, for example, a monochrome CCD (charge coupled device), and generates image data by outputting an analog electric signal corresponding to the amount of light received from each pixel. The image pickup device 121 may be another image pickup device such as a CMOS (complementary metal oxide semiconductor) image sensor.

In the following description, the image data indicating the image of the measurement target S generated by the imaging unit 120 in the state where the structured light is irradiated on the measurement target S is referred to as pattern image data. On the other hand, the image data indicating the image of the measurement target S generated by the imaging unit 120 in the state where the uniform light is irradiated on the measurement target S is referred to as texture image data.

The arithmetic unit 130 is realized by, for example, an FPGA (field programmable gate array), and includes an imaging processing unit 131, an arithmetic processing unit 132, an image processing unit 133, a storage unit 134, and an output processing unit 135. In the present embodiment, arithmetic unit 130 is realized by an FPGA, but the present invention is not limited to this. The arithmetic unit 130 may be realized by a CPU (central processing unit) and RAM (random access memory), or may be realized by a microcomputer.

The imaging processing unit 131 controls the operations of the four illumination units 110 and the imaging unit 120. The arithmetic processing unit 132 generates height data indicating a height image of the measuring object S for each of the four illumination units 110 based on the plurality of pattern image data.

The image processing unit 133 generates combined height data by combining the plurality of height data generated by the arithmetic processing unit 132 so as to correspond to the plurality of illumination units 110, respectively. Further, the image processing unit 133 generates combined texture image data by combining the plurality of texture image data generated by the imaging unit 120 corresponding to each of the plurality of illumination units 110. Details of the image processing unit 133 will be described later.

The storage unit 134 is used as a work area in the calculation unit 130, and also includes pattern image data, height data, combined height data, texture image data generated by the imaging unit 120, the calculation processing unit 132, and the image processing unit 133. The synthetic texture image data is temporarily stored. The output processing unit 135 outputs the combined height data or the combined texture image data stored in the storage unit 134.

The controller unit 200 includes a head control unit 210, an image memory 220, and an inspection unit 230. The head control unit 210 controls the operation of the head unit 100 based on a command given by the external device 400. The image memory 220 stores the combined height data or the combined texture image data output by the calculation unit 130.

The inspection unit 230 executes processing such as edge detection or dimension measurement based on the combined height data or the combined texture image data stored in the image memory 220 based on the inspection content designated by the user. In addition, the inspection unit 230 determines the quality of the measurement target S by comparing the measured value with a predetermined threshold value, and gives the determination result to the external device 400.

An operation unit 310 and a display unit 320 are connected to the controller unit 200. The operation unit 310 includes a keyboard, a pointing device, or a dedicated console. A mouse, a joystick, or the like is used as the pointing device. By operating the operation unit 310, the user can specify desired inspection contents in the controller unit 200.

The display unit 320 is composed of, for example, an LCD (liquid crystal display) panel or an organic EL (electroluminescence) panel. The display unit 320 displays a height image based on the combined height data stored in the image memory 220. The display unit 320 also displays the determination result of the measurement target S by the inspection unit 230.

FIG. 3 is a diagram showing an example of the configuration of each illumination unit 110 in FIG. As shown in FIG. 3, each illumination unit 110 includes light sources 111, 112 and 113, dichroic mirrors 114 and 115, an illumination lens 116, a mirror 117, a pattern generation unit 118 and a light projecting lens 119. The light sources 111, 112, and 113 are, for example, LEDs (light emitting diodes), and emit green light, blue light, and red light, respectively. Each of the light sources 111 to 113 may be a light source other than the LED.

The dichroic mirror 114 is arranged so that the green light emitted by the light source 111 and the blue light emitted by the light source 112 can be superimposed on each other. The dichroic mirror 115 is arranged so that the light superposed by the dichroic mirror 114 and the red light emitted by the light source 113 can be superposed on each other. Thereby, the lights emitted from the light sources 111 to 113 are superposed on the common optical path, and white light can be generated.

The illumination lens 116 collects the light that has passed through or reflected the dichroic mirror 115. The mirror 117 reflects the light condensed by the illumination lens 116 to the pattern generation unit 118. The pattern generation unit 118 is, for example, a DMD (digital micromirror device), and imparts an arbitrary pattern to incident light. The pattern generation unit 118 may be an LCD or an LCOS (reflection type liquid crystal element). The light projecting lens 119 expands the light from the light generation surface of the pattern generation unit 118 and irradiates the measurement target S of FIG. 1 with the light. The light projecting lens 119 may be configured to collimate the light from the pattern generation unit 118 and irradiate the measurement target S.

The imaging processing unit 131 in FIG. 1 controls the emission of light by the light sources 111 to 113, and controls the pattern given to the light by the pattern generation unit 118. Accordingly, the structured light of red, blue, green or white and the uniform light of red, blue, green or white can be selectively emitted from the illumination unit 110. In addition, in this Embodiment, the structured light or uniform light of white is selectively irradiated to the measuring object S from each illuminating part 110.

(2) Generation of Height Data In the inspection device 300, a three-dimensional coordinate system unique to the head unit 100 (hereinafter, referred to as a device coordinate system) is defined. The apparatus coordinate system of this example includes an X axis, a Y axis, and a Z axis that are orthogonal to the origin. In the following description, the direction parallel to the X axis of the device coordinate system is called the X direction, the direction parallel to the Y axis is called the Y direction, and the direction parallel to the Z axis is called the Z direction. The X direction and the Y direction are orthogonal to each other in a plane parallel to the upper surface of the belt conveyor 301 (hereinafter referred to as a reference plane). The Z direction is orthogonal to the reference plane.

In the head unit 100, height data indicating a height image of the measuring object S is generated by the triangulation method. FIG. 4 is a diagram for explaining the principle of the triangulation method. In FIG. 4, the X direction, the Y direction, and the Z direction are indicated by arrows. As shown in FIG. 4, an angle α between the optical axis of light emitted from the illumination unit 110 and the optical axis of light incident on the imaging unit 120 is set in advance. The angle α is larger than 0 degrees and smaller than 90 degrees.

When the measurement target S does not exist below the head unit 100, the light emitted from the illumination unit 110 is reflected by the point O on the reference plane R and enters the imaging unit 120. On the other hand, when the measurement target S is present below the head unit 100, the light emitted from the illumination unit 110 is reflected by the point A on the surface of the measurement target S and enters the imaging unit 120. As a result, the measurement target S is imaged, and image data indicating an image of the measurement target S is generated.

When the distance between the point O and the point A in the X direction is d, the height h of the point A of the measuring object S with respect to the reference plane R is given by h=d÷tan(α). The calculation unit 130 calculates the distance d based on the image data generated by the imaging unit 120. In addition, the calculation unit 130 calculates the height h of the point A on the surface of the measuring object S based on the calculated distance d. By calculating the heights of all the points on the surface of the measuring object S, the coordinates represented by the device coordinate system can be specified for all the points irradiated with light. Thereby, height data of the measuring object S is generated.

In order to irradiate all points on the surface of the measuring object S with light, various structured lights are emitted from the respective illumination units 110. In the present embodiment, the striped structured light (hereinafter referred to as striped light) having a linear cross section that is parallel to the Y direction and aligned in the X direction has its spatial phase changed. The light is emitted from each illumination unit 110 multiple times. Further, the coded structured light (hereinafter, referred to as coded light) having a linear cross section parallel to the Y direction and having the bright portion and the dark portion aligned in the X direction is the bright portion and the dark portion. While being changed into a gray code, the light is emitted from each illuminating unit 110 a plurality of times.

(3) Basic Flow of Inspection Process FIG. 5 is a flowchart showing an example of the inspection process executed by the inspection device 300 of FIG. In the inspection process of this example, the combined height data and the combined texture image data are generated in this order. As illustrated in FIG. 5, the imaging processing unit 131 first selects any one of the illumination units 110A to 110D (step S11). Next, the imaging processing unit 131 controls the illumination unit 110 selected in step S11 or step S19 described below so as to emit white structured light having a predetermined pattern (step S12).

In addition, the imaging processing unit 131 controls the imaging unit 120 so as to image the measurement target S in synchronization with the emission of the structured light in step S12 (step S13). As a result, the pattern image data of the measuring object S is generated by the image capturing unit 120. After that, the imaging processing unit 131 stores the pattern image data generated in step S13 in the storage unit 134 (step S14). Subsequently, the imaging processing unit 131 determines whether or not imaging has been performed a predetermined number of times (step S15).

In step S15, when the imaging has not been executed a predetermined number of times, the imaging processing unit 131 controls the pattern generation unit 118 of FIG. 3 to change the pattern of structured light (step S16), and returns to step S12. .. Steps S12 to S16 are repeated until imaging is performed a predetermined number of times. Thereby, a plurality of pattern image data when the measurement object S is sequentially irradiated with the striped light and the coded light while the pattern is changed is stored in the storage unit 134. Either the striped light or the coded light may be emitted first.

When the imaging is performed a predetermined number of times in step S15, the arithmetic processing unit 132 generates height data by performing an arithmetic operation on the plurality of pattern image data stored in the storage unit 134 (step S17). Next, the arithmetic processing unit 132 determines whether or not a predetermined number (four in this example) of height data corresponding to each of the plurality of illumination units 110 has been generated (step S18). When the predetermined number of height data has not been generated, the imaging processing unit 131 selects another illumination unit 110 (step S19) and returns to step S12. Steps S12 to S19 are repeated until a predetermined number of height data are generated.

When a predetermined number of height data are generated in step S18, the image processing unit 133 performs a combining process of combining the generated predetermined number of height data (step S20). As a result, combined height data is generated. Details of the combining process will be described later. Further, the output processing unit 135 outputs the combined height data generated in step S20 to the controller unit 200 (step S21). As a result, the combined height data is stored in the image memory 220 of the controller unit 200.

Subsequently, texture generation processing is executed in the head unit 100 (step S22). In the texture generation process, the imaging processing unit 131 controls the plurality of illumination units 110 so that uniform white light is sequentially emitted. Further, the image capturing processing unit 131 controls the image capturing unit 120 to capture an image of the measurement target S in synchronization with the emission of each uniform light. As a result, a plurality of texture image data corresponding to the plurality of illumination units 110 are generated. Further, the image processing unit 133 synthesizes the plurality of generated texture image data. Thereby, synthetic texture image data is generated. Further, the output processing unit 135 outputs the generated synthetic texture image data to the controller unit 200. As a result, the composite texture image data is stored in the image memory 220 of the controller unit 200.

After that, the determination process is executed in the controller unit 200 (step S23). In the determination process, the inspection unit 230 determines the quality of the measuring object S based on the combined height data or the combined texture image data accumulated in the image memory 220 and various threshold values set in advance in the image memory 220. Make a judgment (inspection). In addition, the inspection unit 230 may display the determination result in step S23 on the display unit 320 or may provide the determination result to the external device 400.

Any one of the series of processes of steps S11 to S21 and the texture generation process of step S22 may be executed first, or may be partially executed in parallel.

(4) Accuracy of Height Data In the present embodiment, for each of the plurality of portions of the measurement object S, the value of height data corresponding to the true height of the portion with respect to a reference plane, corresponding to the portion. The degree of coincidence of (height) is called the accuracy of height data. The accuracy of height data is considered to be high when the height data values match or almost match the true height. On the other hand, when the value of the height data deviates greatly from the true height, the accuracy of the height data is considered to be low.

FIG. 6 is a schematic perspective view showing the positional relationship between the measurement target S and the plurality of illumination units 110 and imaging units 120 of FIG. 2 when the measurement target S is inspected. In FIG. 6, the four illumination units 110A, 110B, 110C, 110D and the imaging unit 120 of FIG. 2 are indicated by circles in order to facilitate understanding of the positional relationship between the plurality of illumination units 110 and the imaging unit 120. Be done. Further, two straight lines extending in parallel with each other in the X direction and the Y direction through the image capturing unit 120 are indicated by alternate long and short dash lines.

In the example of FIG. 6, the image pickup unit 120 is a belt conveyor such that the image pickup field faces downward and the optical axes of the optical systems (the light receiving lenses 122 and 123 of FIG. 1) of the image pickup unit 120 are parallel to the Z direction. It is arranged above 301. On the other hand, the illumination units 110A and 110B are arranged at the same or substantially the same height position as the image capturing unit 120 in the Z direction and symmetrically so as to sandwich the image capturing unit 120 in the X direction. Further, the illumination units 110C and 110D are arranged at the same or almost the same height position as the image capturing unit 120 in the Z direction and symmetrically so as to sandwich the image capturing unit 120 in the Y direction.

When inspecting the measuring object S, the belt conveyor 301 is temporarily stopped, for example, in a state where the center of the measuring object S is located on the optical axis of the optical system of the imaging unit 120. Further, as a result of the processing of steps S11 to S19 in the above-described inspection processing, a series of structured light is sequentially provided from the illumination units 110A, 110B, 110C, 110D to the measurement target S as shown by the thick solid arrows in FIG. Is irradiated. At this time, as indicated by the thick double-dashed line arrow in FIG. 6, each structured light reflected from the measuring object S and traveling upward is received by the image pickup device 121 of the image pickup unit 120 in FIG. 1. Thereby, a series of pattern image data corresponding to a series of structured light is generated for each illumination unit.

Here, the traveling direction of the structured light with which the measurement target S is irradiated from each illumination unit 110 is inclined with respect to the optical axis of the optical system of the imaging unit 120. Therefore, depending on the shape of the measurement object S, there is a possibility that a portion of the surface of the measurement object S where the structured light is not irradiated (hereinafter, referred to as a shadow portion) may occur in each irradiation direction of the structured light. .. Alternatively, depending on the surface state of the measurement object S, a portion (hereinafter referred to as a multiple reflection portion) where multiple reflection of structured light occurs on the surface of the measurement object S for each irradiation direction of the structured light occurs. there is a possibility. Height data with high accuracy cannot be obtained for these parts.

The measuring object S of FIG. 6 has a plate-shaped portion S0, one wall portion S1 and the other wall portion S2. The plate-shaped portion S0 is composed of a square flat plate-shaped member. The one wall portion S1 and the other wall portion S2 are formed so as to extend upward by a certain height from a pair of side edges of the plate portion S0 facing each other. The one wall portion S1 and the other wall portion S2 have flat upper end surfaces having a common height and have inner side surfaces facing each other.

FIG. 7A is a side view showing a state in which structured light is irradiated to the measurement object S from the illumination unit 110A of FIG. 6, and FIG. 7B is a structured light of FIG. 7A. It is a figure which shows the height image acquired by irradiation of. In FIG. 7A, the irradiation range of the structured light with which the illumination target 110A irradiates the measurement target S is indicated by a dot pattern. In the height image shown in FIG. 7B and FIG. 8B described later, the height of each portion on the surface of the measurement object S in plan view is represented by the density of hatching. .. The higher the hatching density is, the lower the height of the corresponding portion is. The lower the hatching density is, the higher the height of the corresponding portion is. Further, in the height image shown in FIG. 7B and FIG. 8B described later, the missing portion of the height data for which the height data could not be obtained is represented as an invalid data portion by a blank. Shall be done.

As shown in FIG. 7A, a part of the structured light that is directed toward the surface (upper surface) of the plate-shaped portion S0 is blocked by the one wall portion S1. As a result, a portion of the surface of the plate-shaped portion S0 near the one wall portion S1 becomes a shaded portion as. In addition, as shown by the two-dot chain line arrow in FIG. 7A, a part of the structured light with which the other wall portion S2 is irradiated has an inner surface (one wall of the other wall portion S2). It is reflected by the surface facing the portion S1). The reflected light is incident on the surface of the plate-shaped portion S0 in the vicinity of the other wall portion S2. As a result, the portion of the surface of the plate-shaped portion S0 near the other wall portion S2 becomes the multiple reflection portion ar. As a result, in the height image of the illuminating section 110A, as shown in FIG. 7B, the image portion S0i of the plate-like portion S0, which should originally be hatched with a single density, is hatched with a plurality of densities. And blank. The blank portion indicated by the white arrow aa in the image portion S0i corresponds to the shadow portion as. Further, the portion indicated by the white arrow ab in the image portion S0i corresponds to the multiple reflection portion ar. As such, the height data acquired for the illumination unit 110A has low accuracy as a whole. In FIG. 7B, the image portions S1i and S2i located on both sides of the image portion S0i correspond to the upper end surfaces of the one wall portion S1 and the other wall portion S2 of the measuring object S, respectively.

The illumination unit 110B is arranged at a position symmetrical to the illumination unit 110A with reference to the YZ plane passing through the center of the measuring object S. The measurement object S has a symmetrical structure with respect to the YZ plane passing through the center of the measurement object S. Therefore, the height data acquired for the illumination unit 110B has low accuracy as a whole, as in the example of the illumination unit 110A.

FIG. 8A is a side view showing a state in which the structured light is irradiated onto the measuring object S from the other illumination unit 110C of FIG. 6, and FIG. 8B is the structured light of FIG. 8A. It is a figure which shows the height image acquired by irradiation of. In FIG. 8A, similarly to the example of FIG. 7A, the irradiation range of the structured light with which the illumination target 110D irradiates the measurement target S is indicated by a dot pattern.

In the example of FIG. 8A, there is no element on the measurement target S that blocks structured light traveling from the illumination unit 110C to the surface (upper surface) of the plate-shaped portion S0. Further, there is no element on the measurement target S that reflects the structured light toward the surface (upper surface) of the plate-shaped portion S0. As a result, the shadow portion as and the multiple reflection portion ar do not occur on the surface of the plate-shaped portion S0. As a result, in the height image of the illumination unit 110D, as shown in FIG. 8B, the image portion S0i of the plate-shaped portion S0 is entirely hatched with a single density. Therefore, the height data acquired for the illumination unit 110C has high accuracy as a whole. In FIG. 8B, the image portions S1i and S2i located on both sides of the image portion S0i represent the heights of the upper end surfaces of the one wall portion S1 and the other wall portion S2 of the measuring object S.

The illumination unit 110D is arranged at a position symmetrical to the illumination unit 110C with the XZ plane passing through the center of the measuring object S as a reference. The measurement object S has a symmetrical structure with respect to the XZ plane passing through the center of the measurement object S. Therefore, the height data acquired for the illumination unit 110D has high accuracy as a whole, as in the example of the illumination unit 110C.

As described above, in the measurement object S, the shadow portion as and the multiple reflection portion ar corresponding to the irradiation direction of the structured light are generated. Therefore, the accuracy of the height data generated for each lighting unit 110 differs from each other in a plurality of portions of the measuring object S. When a plurality of height data generated for the plurality of illumination units 110 are combined, if the combined height data contains a large amount of inaccurate height data components, the overall accuracy of the combined height data may be increased. Sex decreases. Therefore, in the present embodiment, a combining process is performed to obtain combined height data with higher accuracy. The details of the combining process executed in step S20 of FIG. 5 will be described below.

(5) Height Data Combining Process FIG. 9 is a block diagram for explaining the detailed configuration of the arithmetic unit 130 according to the first embodiment. FIG. 9 mainly shows a detailed configuration of the image processing unit 133 of the calculation unit 130 of FIG.

As shown in FIG. 9, the image processing unit 133 according to the present embodiment includes a first intermediate height data generation unit 11, a first reliability calculation unit 12, and a second intermediate height data generation unit 13. The second reliability calculation unit 14 and the combined height data generation unit 15 are included.

The first intermediate height data generation unit 11 combines the first pair of height data generated for the illuminating units 110A and 110B facing each other with the image capturing unit 120 in between in the X direction to synthesize the first intermediate height data. To generate. The first reliability calculation unit 12 calculates the reliability of the first intermediate height data for each of the plurality of portions (pixels in this example) of the height image as the first reliability. The second intermediate height data generation unit 13 synthesizes the second pair of height data generated for the illumination units 110C and 110D facing each other with the imaging unit 120 in between in the Y direction to synthesize the second intermediate height data. To generate. The second reliability calculation unit 14 calculates the reliability of the second intermediate height data as the second reliability for each of the plurality of portions (pixels in this example) of the height image. The first and second reliabilities will be described later. The combined height data generation unit 15 combines the combined height data by combining the first and second intermediate height data for each of the plurality of pixels of the height image based on the first and second reliability. To generate.

FIG. 10 is a flowchart of the combining process according to the first embodiment. At the start of the combining process, it is assumed that, by the process of step S15 of FIG. 3, the arithmetic processing unit 132 has generated four height data for each of the four illumination units 110A to 110D in advance.

Basically, the four height data generated for the four illumination units 110A to 110D include an error component due to the irradiation direction of the structured light. A pair of height data generated for a pair of illumination units that irradiate structured light in mutually symmetrical directions with respect to the measurement object S as a reference are averaged and combined. In this case, in the height data obtained by the synthesis, the error components included in each of the pair of height data before the synthesis cancel each other.

Therefore, when the synthesizing process is started, the first intermediate height data generation unit 11 first generates the first pair of illumination units 110A and 110B that face each other across the image capturing unit 120 in the X direction. Get the height data of the. Further, the first intermediate height data generation unit 11 generates the first intermediate height data by averaging and synthesizing the first pair of height data for each of the plurality of pixels of the height image. (Step S31). More specifically, the first intermediate height data generation unit 11 calculates, for each pixel of height images corresponding to each other, an average value of the values of the first pair of height data corresponding to the pixel. Note that in step S31, for pixels in which only one of the height data of the first pair is missing, a valid value of the other height data is selected. Further, for a pixel in which both the height data of the first pair are missing, the height data of the portion is invalid.

For each of the plurality of pixels of the height image, the first pair of height image data is more accurate as the difference between the height data is smaller, and at least one of the height image data is more accurate as the difference between the height data is larger. It is considered to be low in sex. Therefore, the first reliability calculation unit 12 calculates a value regarding the difference between the height data of the first pair for each of the plurality of pixels of the height image as the first reliability of the portion (step S32). .. More specifically, the first reliability calculation unit 12 calculates, as the first reliability, the absolute value of the difference between the height data of the first pair for each of the plurality of pixels of the height image. In this case, the value of the first reliability is smaller as the reliability is higher (the accuracy of the first intermediate height data is higher). Further, the lower the reliability is (the lower the accuracy of the first intermediate height data is), the higher the reliability is.

Next, the second intermediate height data generation unit 13 acquires the second pair of height data generated for the second pair of illumination units 110C and 110D that face each other across the image capturing unit 120 in the Y direction. In addition, the second intermediate height data generation unit 13 generates the second intermediate height data by averaging and synthesizing the second pair of height data for each of the plurality of pixels of the height image. (Step S33). More specifically, the second intermediate height data generation unit 13 calculates, for each pixel of the height images corresponding to each other, the average value of the values of the second pair of height data corresponding to the pixel. Note that in step S33, for pixels in which only one of the height data of the second pair is missing, a valid value of the other height data is selected. Further, for a pixel in which both the height data of the second pair are missing, the height data of the portion is invalid.

For each of the plurality of pixels of the height image, the second pair of height image data is more accurate as the difference between the height data is smaller, and at least one is more accurate as the difference between the height data is larger. It is considered to be low in sex. Therefore, the second reliability calculation unit 14 calculates, for each of the plurality of pixels of the height image, a value relating to the difference between the height data of the second pair as the second reliability of the portion (step S34). .. More specifically, the second intermediate height data generation unit 13 calculates, as the second reliability, the absolute value of the difference between the height data of the second pair for each of the plurality of pixels of the height image. .. In this case, the value of the second reliability is smaller as the reliability is higher (the accuracy of the second intermediate height data is higher). Further, the lower the reliability is (the lower the accuracy of the second intermediate height data is), the higher the reliability is.

Finally, the combined height data generation unit 15 selects, for each of the plurality of pixels of the height image, the intermediate height data having higher reliability from the first and second intermediate height data. Combined height data is generated (step S35). For example, when the first reliability value is smaller than the second reliability value for a pixel having height images corresponding to each other, the combined height data generation unit 15 generates the first intermediate height data. Select. In addition, the combined height data generation unit 15 determines the second intermediate height data when the value of the second reliability is smaller than the value of the first reliability for a pixel having height images corresponding to each other. Select.

In the above combining process, the processes of steps S31 to S34 are not limited to the above order, and may be performed in the reverse order of the above order, or may be performed in another order. Furthermore, at least a part of the processes of steps S31 to S34 may be performed at the same time.

(6) Effects of First Embodiment In the inspection device 300 described above, the first pair of height data corresponding to the illumination units 110A and 110B are combined to generate the first intermediate height data. The illumination units 110A and 110B irradiate structured light in directions symmetrical to each other with respect to the measurement target S. Thereby, when the first intermediate height data is generated, the error components included in each of the first pair of height data are canceled by averaging. Therefore, the first intermediate height data has high accuracy. Further, the reliability of the first intermediate height data for each of the plurality of portions of the height image is calculated as the first reliability. The first reliability is calculated based on the first pair of height data. As a result, a more effective first reliability can be calculated as compared with the case where the reliability is calculated using only height data generated for either one of the illumination units 110A and 110B.

The second pair of height data corresponding to the illumination units 110C and 110D are combined to generate second intermediate height data. The illumination units 110C and 110D irradiate structured light in directions symmetrical to each other with respect to the measuring object S as a reference. Thereby, when the second intermediate height data is generated, the error components included in each of the second pair of height data are canceled by averaging. Therefore, the second intermediate height data has high accuracy. Further, the reliability of the second intermediate height data for each of the plurality of portions of the height image is calculated as the second reliability. The second reliability is calculated based on the height data of the second pair. As a result, a more effective second reliability can be calculated as compared with the case where the reliability is calculated using only height data generated for either one of the illumination units 110C and 110D.

For each of the plurality of portions of the height image, the first and second intermediate height data corresponding to each other are appropriately combined based on the first and second reliabilities to generate combined height data. .. As a result, by using the generated combined height data, it becomes possible to improve the accuracy of the inspection regarding the height of the measuring object S.

[2] Second Embodiment The inspection apparatus according to the second embodiment is the inspection apparatus according to the first embodiment except for the configuration of the arithmetic unit 130 and the synthesis processing executed by the arithmetic unit 130. It has the same configuration and operation as 300.

FIG. 11 is a block diagram for explaining a detailed configuration of the arithmetic unit 130 according to the second embodiment. As shown in FIG. 11, the calculation unit 130 according to the present embodiment includes an allowable condition determination unit 21 in addition to the configuration of the calculation unit 130 according to the first embodiment. The permissible condition determining unit 21 predetermines height data of a plurality of pixels of a height image of the illumination unit 110 based on a plurality of pattern image data generated corresponding to each of the four illumination units 110. It is determined whether or not the specified acceptance condition is satisfied.

The permissible condition is a condition for determining whether or not the height data is generated based on the permissible data for each of the plurality of pixels of the height image, and is stored in the storage unit 134 in advance. It The inspection apparatus 300 may be configured to be able to set the above-mentioned allowable conditions based on the operation of the operation unit 310 of FIG. 1 by the user or the manufacturer.

The fact that the magnitude of the variation in the brightness value of each pixel corresponding to each other in the plurality of pattern image data generated for one illumination unit 110 is equal to or smaller than a predetermined threshold means that the measurement target S corresponding to the pixel is It means that the light does not reach the part of (the generation of shadow). Alternatively, if the magnitude of the variation in the brightness value of each pixel corresponding to each other in the plurality of pattern image data generated for one illumination unit 110 is less than or equal to a predetermined threshold value, for example, the measurement target corresponding to the pixel This means that the reflected light from the part of the object S does not enter the imaging unit 120. Alternatively, when the magnitude of the variation in the brightness value of each pixel corresponding to each other in the plurality of pattern image data generated for one illumination unit 110 is equal to or less than a predetermined threshold value, for example, the measurement target corresponding to the pixel. This means that the amount of structured light that enters the imaging unit 120 from the part of the object S is extremely high (occurrence of halation). Therefore, in the present embodiment, the permissible condition is that for each of the plurality of pixels of the height image, the magnitude (amplitude) of the variation in the brightness value of the plurality of pattern image data of the corresponding pixels is a predetermined threshold value. It is supposed to be larger than.

The first and second intermediate height data generation units 11 and 13 according to the present embodiment generate the first and second intermediate height data based on the determination result of the allowable condition by the allowable condition determination unit 21. .. Specifically, the first and second intermediate height data generation units 11 and 13 treat height data satisfying the allowance condition as valid height data, and height data not satisfying the allowance condition as invalid height data. Treated as data.

FIG. 12 is a flowchart of the combining process according to the second embodiment. At the start of the combining process, it is assumed that four height data corresponding to the four illumination units 110A to 110D have been generated in advance in the arithmetic processing unit 132 by the process of step S15 of FIG. Further, in the present example, it is assumed that the plurality of pattern image data used to generate each of the four height data are stored in the storage unit 134 in a state of being associated with each height data. To do.

When the synthesizing process is started, the permissible condition determining unit 21 first determines, for each of the plurality of pixels of the height image, whether or not the first pair of height data satisfies the permissible condition (step S41).

Next, the 1st intermediate|middle height data production|generation part 11 produces|generates 1st intermediate|middle height data based on the determination result of a permissive condition (step S42). More specifically, the first intermediate height data generation unit 11 determines, for each of the plurality of pixels of the height image, the first pair of height data effective when both the height data of the first pair satisfy the allowance condition. Calculate the average value of the height data. On the other hand, the first intermediate height data generation unit 11 determines, for each of the plurality of pixels of the height image, the other height data that is effective when one of the height data of the first pair does not satisfy the allowable condition. Select a value. On the other hand, the first intermediate height data generation unit 11 determines, for each of the plurality of pixels of the height image, the height data corresponding to the pixel when both the first pair of height data do not satisfy the allowance condition. Invalidate.

Next, the first reliability calculation unit 12 calculates the first reliability as in the process of step S32 of FIG. 10 according to the first embodiment (step S43).

Next, the allowance condition determination unit 21 determines whether or not the second pair of height data satisfies the allowance condition for each of the plurality of pixels of the height image (step S44).

Next, the second intermediate height data generation unit 13 generates the second intermediate height data based on the determination result of the allowance condition (step S45). More specifically, the second intermediate height data generation unit 13 determines, for each of the plurality of pixels of the height image, the second pair of height data effective when both of the height data of the second pair satisfy the allowable condition. Calculate the average value of the height data. On the other hand, the second intermediate height data generation unit 13 determines, for each of the plurality of pixels of the height image, the other height data that is effective when one of the height data of the second pair does not satisfy the allowable condition. Select a value. On the other hand, the second intermediate height data generation unit 13 determines, for each of the plurality of pixels of the height image, the height data corresponding to the pixel when both of the height data of the second pair do not satisfy the allowance condition. Invalidate.

Next, the second reliability calculation unit 14 calculates the second reliability as in the process of step S34 of FIG. 10 according to the first embodiment (step S46).

Finally, the combined height data generation unit 15 generates combined height data (step S47), similar to the process of step S35 of FIG. 10 according to the first embodiment.

According to the synthesizing process according to the second embodiment, it is possible to reduce the inclusion of height data components that do not satisfy the allowance condition in the first and second intermediate height data. As a result, synthetic height data with higher accuracy is generated.

In the present embodiment, the first and second intermediate height data generation units 11 and 13 treat height data satisfying the allowable condition as effective height data, and height data not satisfying the allowable condition. Although it is treated as invalid height data, the present invention is not limited to this.

Instead of the height data that the first and second intermediate height data generation units 11 and 13 do not satisfy the permissible condition as invalid height data, the first and second reliability calculation units 12 and 14 The first and second reliability may be calculated as follows.

For example, the first reliability calculation unit 12 makes the first reliability of the first intermediate height data the lowest for the pixel in which at least one of the height data of the first pair is determined to be invalid (first Make the reliability value of 1 the maximum value.). That is, the first reliability calculation unit 12 determines the value of the first reliability so that it is shown that the accuracy of the first intermediate height data of the pixel is the lowest. Further, the second reliability calculation unit 14 sets the second reliability of the second intermediate height data to the lowest value for the pixel for which at least one of the height data of the second pair is determined to be invalid (the second reliability data). Maximize the confidence value of 2.). That is, the second reliability calculation unit 14 determines the second reliability value so as to show that the accuracy of the second intermediate height data of the pixel is the lowest. In this case, in the process of step S47, selection of height data that does not satisfy the allowance condition is suppressed, and the components of the first and second pairs of height data that do not meet the allowance condition become the combined height data. Inclusion is reduced.

In the above combining process, the processes of steps S42, S43, S45, and S46 are not limited to the above order, and may be performed in the reverse order of the above order, or may be performed in another order. .. Furthermore, at least a part of the processes of steps S42, S43, S45, and S46 may be performed at the same time.

[3] Third Embodiment An inspection apparatus according to a third embodiment is the inspection apparatus according to the first embodiment except for the configuration of the arithmetic unit 130 and the composition processing executed by the arithmetic unit 130. It has the same configuration and operation as 300.

FIG. 13 is a block diagram for explaining the detailed configuration of the arithmetic unit 130 according to the third embodiment. As shown in FIG. 13, the arithmetic unit 130 according to the present embodiment includes a reliability condition determination unit 22 in addition to the configuration of the arithmetic unit 130 according to the first embodiment. The reliability condition determination unit 22 determines whether or not the first and second reliability satisfy a predetermined reliability condition for each of the plurality of pixels of the height image.

The reliability condition is that at least one of the following first condition and second condition is satisfied. The first condition is that the difference between the first and second reliability levels corresponding to each other is within a predetermined difference range. The second condition is that the first and second reliability levels corresponding to each other are both higher than a predetermined reference reliability level. Here, the difference range of the first condition is set so that, for example, it can be considered that the first and second reliabilities are the same or substantially the same. In addition, the reference reliability of the second condition is set to be the maximum value of the reliability of the height data that can be allowed for the inspection of the measuring object S, for example.

The combined height data generation unit 15 according to the present embodiment generates combined height data based on the determination result of the reliability condition by the reliability condition determination unit 22. Specifically, the combined height data generation unit 15 determines, for each of the plurality of pixels of the height image, that the first and second intermediate heights satisfy the reliability conditions. The average height data is averaged to generate combined height data. On the other hand, when the first and second reliability levels do not satisfy the reliability level condition for each of the plurality of pixels of the height image, the combined height data generation section 15 performs the same operation as in the first embodiment. The combined height data is generated by selecting the intermediate height data having higher reliability from the first and second intermediate height data.

FIG. 14 is a flowchart of the combining process according to the third embodiment. At the start of the combining process, it is assumed that, by the process of step S15 of FIG. 3, the arithmetic processing unit 132 has generated four height data for each of the four illumination units 110A to 110D in advance.

In the present embodiment, the first intermediate height data generation unit 11, the first reliability calculation unit 12, the second intermediate height data generation unit 13 and the second reliability calculation unit 14 are the first The same processing as the processing of steps S31 to S34 of the combining processing according to the embodiment is performed. That is, the first intermediate height data generation unit 11 generates the first intermediate height data (step S51), and the first reliability calculation unit 12 calculates the first reliability (step S52). In addition, the second intermediate height data generation unit 13 generates the second intermediate height data (step S53), and the second reliability calculation unit 14 calculates the second reliability (step S54).

Next, the reliability condition determination unit 22 determines whether or not the first and second reliability satisfy a predetermined reliability condition for each of the plurality of pixels of the height image (step S55). ..

After that, the combined height data generation unit 15 performs averaging or selection on each of the plurality of pixels of the height image based on the determination result of the first reliability, the second reliability, and the reliability condition. To generate combined height data (step S56). Specifically, the combined height data generation unit 15 performs averaging on each of the plurality of pixels of the height image when the first and second reliability conditions satisfy the reliability, and then performs the averaging. When the reliability of 2 does not satisfy the reliability condition, the selection based on the first and second reliability is performed. Thereby, composite height data is generated.

When the difference between the first and second reliability values corresponding to each other for a pixel in the height image is within a predetermined difference range, the accuracy of the first and second intermediate height data is almost equal. Considered equal. Further, when both the first and second reliabilities are higher than a predetermined reference reliance for a pixel in the height image, the accuracy of the first and second intermediate height data is both high. It is considered expensive.

Therefore, the accuracy of the combined height data for the pixel is improved by averaging the first and second intermediate height data when the first and second reliability satisfy the reliability condition. ..

If the first and second reliability do not satisfy the reliability condition, one of the first and second intermediate height data having high accuracy is selected as the combined height data. .. This reduces the inclusion of height data components with low accuracy in the composite height data.

In the above combining process, the processes of steps S51, S52, S53, and S54 are not limited to the above order, and may be performed in the reverse order of the above order, or may be performed in another order. .. Furthermore, at least a part of the processing of steps S51, S52, S53, and S54 may be performed at the same time.

[4] Fourth Embodiment The inspection apparatus according to the third embodiment has the same configuration and operation as the inspection apparatus 300 according to the first embodiment, except for the combining process executed by the arithmetic unit 130. Have.

In the inspection device 300 according to the present embodiment, the method of generating the combined height data by the combined height data generation unit 15 of the calculation unit 130 is different from that of the first embodiment. The composite height data generation unit 15 according to the present embodiment weights and averages the first and second intermediate height data based on the first and second reliability for each of the plurality of portions of the height image. The combined height data is generated by performing the processing. A specific example of the weighted averaging process by the combined height data generation unit 15 will be described.

For each of the plurality of pixels of the height image, the value of the first intermediate height data is H ₁ and the value of the second intermediate height data is H ₂ . In addition, the first reliability value (in this example, the absolute value of the difference between the first pair of height data) is set to P _1, and the second reliability value (in this example, the second pair of height data The absolute value of the difference) is P ₂ .

In this case, for each of the plurality of pixels of the height image, the combined height data generation unit 15 calculates the value H ₀ of the combined height data for the pixel according to the following equation (1).
H ₀ =H ₁ ×{P ₂ /(P ₁ +P ₂ )}+H ₂ ×{P ₁ /(P ₁ +P ₂ )}...(1)
According to the above calculation method, the weighted averaging process is performed with emphasis placed on the height data having high reliability among the first and second intermediate height data.

FIG. 15 is a flowchart of the combining process according to the fourth embodiment. In the present embodiment, the first intermediate height data generation unit 11, the first reliability calculation unit 12, the second intermediate height data generation unit 13 and the second reliability calculation unit 14 are the first The same processing as the processing of steps S31 to S34 of the combining processing according to the embodiment is performed. That is, the first intermediate height data generation unit 11 generates the first intermediate height data (step S61), and the first reliability calculation unit 12 calculates the first reliability (step S62). In addition, the second intermediate height data generation unit 13 generates the second intermediate height data (step S63), and the second reliability calculation unit 14 calculates the second reliability (step S64).

Then, the combined height data generation unit 15 generates combined height data for each of the plurality of pixels of the height image by weighted averaging based on the first and second reliability (step S65).

In this case, the first and second intermediate height data are appropriately combined according to the first and second reliability. This improves the accuracy of the composite height data.

In the above combining process, the processes of steps S61, S62, S63, and S64 are not limited to the above order, but may be performed in the reverse order of the above order, or may be performed in another order. .. Furthermore, at least a part of the processes of steps S61, S62, S63, and S64 may be performed at the same time.

In the present embodiment, the processes of steps S41 to S46 according to the second embodiment may be performed instead of the processes of steps S61 to S64.

[5] Other Embodiments (a) In the above embodiment, the head unit 100 includes two pairs of illumination units 110 and one imaging unit 120, but the present invention is not limited to this. The head unit 100 may have a configuration in which a plurality of pairs of image capturing units 120 are provided for one illumination unit 110.

FIG. 16 is a block diagram showing the configuration of an inspection device 300 according to another embodiment. As shown in FIG. 16, the head unit 100 in this example includes four imaging units 120. Note that the illustration of the calculation unit 130 is omitted in FIG. 16. In addition, FIG. 16 shows an apparatus coordinate system defined in the inspection apparatus 300 inside the head unit 100.

In the following description, when distinguishing the four image capturing units 120, the four image capturing units 120 are referred to as image capturing units 120A to 120D, respectively. The imaging units 120A to 120D have the same structure and are provided so as to surround the illumination unit 110 at 90-degree intervals. Specifically, the image capturing unit 120A and the image capturing unit 120B are arranged so as to face each other with the illumination unit 110 interposed therebetween in the X direction. Further, the image capturing unit 120C and the image capturing unit 120D are arranged so as to face each other with the illumination unit 110 interposed therebetween in the Y direction.

With this configuration, the four image capturing units 120A to 120D can image the measurement target S from four different directions with respect to the measurement target S. Thereby, the height data of the first pair can be acquired for the first pair of imaging units 120A and 120B that face each other across the illumination unit 110 in the X direction. Further, the height data of the second pair can be acquired for the second pair of imaging units 120C and 120D that face each other across the illumination unit 110 in the Y direction.

Accordingly, the calculation unit 130 (not shown) can perform the combining process according to any of the first to fourth embodiments. As a result, similarly to the above-described embodiment, the first and second intermediate height data having high accuracy are appropriately combined, and the combined height data is generated. As a result, by using the generated combined height data, it becomes possible to improve the accuracy of the inspection regarding the height of the measuring object S.

(B) The inspection device 300 according to the above-described embodiment has the first pair of illumination units 110A and 110B and the second pair of illumination units 110C and 110D for one image capturing unit 120. Not limited to. The inspection apparatus 300 may include the third pair of illumination units 110 arranged to face each other with the imaging unit 120 in between in one direction different from the X direction and the Y direction. Furthermore, the inspection apparatus 300 may include a fourth pair of illumination units 110 arranged so as to face each other with the imaging unit 120 sandwiched in another direction different from the X direction and the Y direction.

In this case, three or more intermediate height data are generated for each of the three or more pairs of illumination units 110, and three or more reliability levels are calculated. Thereby, it is possible to appropriately combine the plurality of intermediate height data of three or more based on the plurality of reliability of three or more. This further improves the accuracy of the combined height data.

(C) When the first pair of height data and the second pair of height data are combined, the combining process according to any one of the first to fourth embodiments may be repeated for each pixel of the height image. For example, when the height image is composed of the 1st to nth pixels, the series of the processing of FIG. 10 is repeated n times for each pixel, so that the entire height image (the imaging field of view of the imaging unit 120 Synthetic height data corresponding to (whole) may be generated.

(D) In the above embodiment, the first reliability calculation unit 12 sets the absolute value of the difference between the height data of the first pair as the first reliability for each of the plurality of pixels of the height image. However, the present invention is not limited to this. The first reliability calculation unit 12 may calculate, as the first reliability, a value obtained by performing a predetermined calculation process (such as addition of a constant value) on the absolute value of the difference. The reciprocal of the absolute value of the difference may be calculated as the first reliability.

The second reliability calculation unit 14 calculates the absolute value of the difference between the height data of the second pair as the second reliability for each of the plurality of pixels of the height image. Not limited to. The second reliability calculation unit 14 may calculate, as the second reliability, a value obtained by performing a predetermined calculation process (such as addition of a constant value) on the absolute value of the difference. The reciprocal of the absolute value of the difference may be calculated as the second reliability.

[6] Correspondence between each constituent of the claims and each part of the embodiment An example of correspondence between each constituent of the claims and each part of the embodiment will be described. In the above embodiment, the measurement target S is an example of the measurement target, and the four illumination units 110A to 110D according to the first to fourth embodiments are examples of the plurality of illumination units, and other examples. The four image capturing units 120A to 120D according to the embodiment are examples of a plurality of image capturing units, the one image capturing unit 120 according to the first to fourth embodiments is an example of one image capturing unit, and One illumination unit 110 according to the embodiment is an example of one illumination unit.

The imaging processing unit 131 and the arithmetic processing unit 132 are examples of the data generation unit, the image processing unit 133 is an example of the image processing unit, the X direction is an example of the first direction, and the first to fourth The illumination units 110A and 110B according to the embodiment are examples of the first pair of illumination units, and the imaging units 120A and 120B according to other embodiments are examples of the first pair of imaging units. The illumination units 110C and 110D according to the fourth embodiment are examples of the second pair of illumination units, and the imaging units 120C and 120D according to other embodiments are examples of the second pair of imaging units.

The first intermediate height data generation unit 11 is an example of the first intermediate height data generation unit, the first reliability calculation unit 12 is an example of the first reliability calculation unit, and the second The intermediate height data generation unit 13 is an example of a second intermediate height data generation unit, and the second reliability calculation unit 14 is an example of a second reliability calculation unit.

Further, the combined height data generation unit 15 is an example of a combined height data generation unit, the permissible condition determination unit 21 is an example of an allowable condition determination unit, and the reliability condition determination unit 22 is an example of a reliability condition determination unit. Is.

As each constituent element of the claims, various other elements having the configurations or functions described in the claims can be used.

11... 1st middle height data generation part, 12... 1st reliability calculation part, 13... 2nd middle height data generation part, 14... 2nd reliability calculation part, 15... Combined height data Generating unit, 21... Allowable condition determining unit, 22... Reliability condition determining unit, 100... Head unit, 110, 110A to 110D... Illuminating unit, 111 to 113... Light source, 114, 115... Dichroic mirror, 116... Illuminating lens, Reference numeral 117... Mirror, 118... Pattern generation unit, 119... Projection lens, 120, 120A to 120D... Imaging unit, 121... Imaging device, 122, 123... Light receiving lens, 130... Calculation unit, 131... Imaging processing unit, 132... Arithmetic processing section, 133... Image processing section, 134... Storage section, 135... Output processing section, 200... Controller section, 210... Head control section, 220... Image memory, 230... Inspection section, 300... Inspection apparatus, 301... Belt Conveyor, 310... Operation part, 320... Display part, 400... External device, ar... Multiple reflection part, as... Shadow part, S... Measurement object, S0... Plate part, S0i, S1i, S2i... Image part, S1 …One wall, S2…other wall

Claims (8)

A plurality of irradiation units for irradiating the measurement object with structured light in a plurality of directions,
An imaging unit that sequentially generates a plurality of pattern image data indicating an image of a measurement target by receiving a plurality of structured lights emitted from the plurality of irradiation units, respectively,
A data generation unit that generates a plurality of height data respectively showing height images of the measurement object for the plurality of irradiation units, based on the pattern image data generated corresponding to each of the plurality of irradiation units. ,
An image processing unit that generates combined height data by combining the plurality of height data for the plurality of irradiation units,
The plurality of irradiation units,
A first pair of irradiation units arranged to face each other across the one imaging unit in a first direction;
A second pair of irradiation units arranged so as to face each other across the one imaging unit in a second direction intersecting the first direction,
The image processing unit,
A first intermediate height data generation unit for generating first intermediate height data by synthesizing a first pair of height data generated for the first pair of irradiation units;
A first reliability calculation unit that calculates the reliability of the generated first intermediate height data as the first reliability for each of the plurality of portions of the height image;
A second intermediate height data generation unit that generates second intermediate height data by combining the second pair of height data generated for each of the second pair of irradiation units;
A second reliability calculation unit that calculates the reliability of the generated second intermediate height data as the second reliability for each of the plurality of portions of the height image;
Selection for selecting one of the first and second intermediate height data corresponding to each other based on the first and second reliability corresponding to each other for each of the plurality of portions of the height image And a combined height data generator that generates the combined height data by performing a weighting process involving weighting of the first and second intermediate height data corresponding to each other. The first intermediate height data generation unit generates the first intermediate height data by averaging and synthesizing the first pair of height data for each of the plurality of portions of the height image. ,
The second intermediate height data generation unit generates the second intermediate height data by averaging and synthesizing the second pair of height data for each of the plurality of portions of the height image. The inspection apparatus according to claim 1. The first reliability calculation unit calculates, for each of a plurality of portions of the height image, a value relating to a difference between the height data of the first pair as a first reliability of the portion,
The second reliability calculation unit calculates, for each of a plurality of portions of a height image, a value related to a difference between the height data of the second pair as a second reliability of the portion. Or the inspection device according to 2. Based on pattern image data generated corresponding to each of the plurality of irradiation units, it is determined whether the height data of a plurality of portions of the height image of the irradiation unit satisfies a predetermined tolerance condition. Further comprising a permissible condition determination unit that
The first intermediate height data generation unit averages the height data of each of the plurality of portions of the height image when the height data of the first pair both satisfy the acceptance condition, If one of the height data of the first pair satisfies the allowance condition and the other does not satisfy the allowance condition, the height data of the one pair is selected, and any one of the height data of the first pair is set to the above. When the permissible condition is not satisfied, the height data of the portion is invalidated to generate the first intermediate height data,
The second intermediate height data generation unit averages the height data of each of the plurality of portions of the height image when the height data of the second pair both satisfy the acceptance condition, and If one of the height data of the second pair satisfies the allowance condition and the other does not satisfy the allowance condition, the height data of the one pair is selected, and any one of the height data of the second pair is set to the above. The inspection apparatus according to claim 1, wherein the second intermediate height data is generated by invalidating the height data of the portion when the acceptance condition is not satisfied. For each of the plurality of portions of the height image, the first reliability calculation unit obtains a value related to a difference between the height data when the height data of the first pair both satisfy the acceptance condition. Is calculated as the first reliability, and a value indicating that the reliability of the part is lowest when at least one of the height data of the first pair does not satisfy the allowable condition is set as the first reliability. Decide,
The second reliability calculation unit determines, for each of the plurality of portions of the height image, a value related to a difference between the height data when the height data of the second pair both satisfy the acceptance condition. Is calculated as the second reliability, and a value indicating that the reliability of the part is the lowest when at least one of the height data of the second pair does not satisfy the allowable condition is set as the second reliability. The inspection apparatus according to claim 4, wherein the determination is performed. For each of the plurality of portions of the height image, it is determined whether the first and second reliability calculated by the first and second reliability calculation units satisfy a predetermined reliability. A reliability condition determination unit is further provided,
The composite height data generation unit averages the first and second intermediate height data for each of the plurality of portions of the height image when the first and second reliability levels satisfy the reliability level. Conversion processing is performed, and when the first and second reliability degrees do not satisfy the reliability condition, the selection processing is performed to generate combined height data,
The reliability condition is that the difference between the first and second reliability values corresponding to each other is within a predetermined difference range, or the first and second reliability values corresponding to each other are both predetermined reference reliability. The inspection apparatus according to any one of claims 1 to 5, which is higher than the degree. The combined height data generation unit performs a weighted averaging process on the first and second intermediate height data based on the first and second reliability for each of the plurality of portions of the height image. The inspection apparatus according to any one of claims 1 to 5, which generates height data. One irradiation unit for irradiating the measurement object with structured light,
A plurality of imaging units that respectively generate a plurality of pattern image data indicating an image of the measurement target by receiving structured light reflected from the measurement target in a plurality of directions,
A data generation unit that generates a plurality of height data respectively showing height images of the measuring object for the plurality of image pickup units, based on pattern image data generated corresponding to each of the plurality of image pickup units. ,
And an image processing unit that generates combined height data by combining the plurality of height data for the plurality of imaging units,
The plurality of imaging units,
A first pair of imaging units arranged so as to face each other across the one illumination unit in a first direction;
A second pair of imaging units arranged so as to face each other across the one illumination unit in a second direction intersecting the first direction,
The image processing unit,
A first intermediate height data generation unit for generating first intermediate height data by synthesizing a first pair of height data generated for the first pair of imaging units;
A first reliability calculation unit that calculates the reliability of the generated first intermediate height data as the first reliability for each of the plurality of portions of the height image;
A second intermediate height data generation unit that generates second intermediate height data by synthesizing a second pair of height data generated for each of the second pair of imaging units;
A second reliability calculation unit that calculates the reliability of the generated second intermediate height data as the second reliability for each of the plurality of portions of the height image;
Selection for selecting one of the first and second intermediate height data corresponding to each other based on the first and second reliability corresponding to each other for each of the plurality of portions of the height image And a combined height data generator that generates the combined height data by performing a weighting process involving weighting of the first and second intermediate height data corresponding to each other.

Images