WO2023170814A1 - Operation device for three-dimensional measurement, three-dimensional measurement program, recording medium, three-dimensional measurement device, and operation method for three-dimensional measurement - Google Patents

Abstract

In the present invention, a component model 83 (reference model) is acquired which indicates a component mount range Ref on a board B in which a component E is mounted and the appearance of the component E mounted in the component mount range Ref. In addition, a shadow area As and a secondary reflection area Ar (irradiation failure area) are calculated on the basis of the component model 83 and projection direction information 84 (irradiation direction information) that indicates a projection direction D in which pattern light L (S) is projected onto the component E in a three-dimensional measurement device 1. In this way, in measuring a three-dimensional shape of an object J to be measured on the basis of a pattern image I(S) acquired by capturing an image of the pattern light L (S) irradiated to the component E, it is possible to acquire the shadow area As and the secondary reflection area A in which the irradiation failure of the pattern light L occurs.

Description

3D measurement calculation device, 3D measurement program, recording medium, 3D measurement device, and 3D measurement calculation method

The present invention relates to a technique for measuring the three-dimensional shape of a measurement object based on a pattern image obtained by imaging a predetermined pattern of light irradiated onto the measurement object.

Patent Documents 1 and 2 describe three-dimensional measurement techniques that measure the three-dimensional shape of a measurement target using a so-called phase shift method. In such three-dimensional measurement technology, the three-dimensional shape of the measurement object is measured based on an image obtained by capturing a predetermined pattern of light irradiated onto the measurement object from a projector using a camera.

Additionally, Patent Document 2 proposes a technique for measuring a three-dimensional shape by suppressing the influence of secondary reflection that occurs when a tall object is present among the objects that constitute the measurement target. In other words, when secondary reflection occurs, such as light emitted from a projector and reflected on the side of a tall object and further reflected by another object, it becomes difficult to accurately measure a three-dimensional shape. Therefore, in Patent Document 2, the range of light irradiation from the projector is limited so that the light from the projector does not enter a tall object (causing object) that causes secondary reflection.

Japanese Patent Application Publication No. 2012-112952 Japanese Patent Application Publication No. 2016-130663

However, in order to limit the range of light irradiated from the projector as in Patent Document 2, it is necessary to control hardware such as a projector included in the three-dimensional measurement device, so the control tends to be complicated. Furthermore, in addition to the above-mentioned secondary reflection, so-called occlusion can be cited as a defective light irradiation that affects the measurement of a three-dimensional shape. This occlusion is a phenomenon in which a shadow occurs where the light from the projector cannot reach due to being blocked by objects that make up the measurement target. Patent Document 2 cannot deal with such occlusion.

On the other hand, if it is possible to know the poor irradiation area where poor light irradiation occurs in the measurement target, the poor irradiation area will be can be referred to. As a result, it is possible to perform calculations for calculating a three-dimensional shape while suppressing the effects of poor light irradiation.

This invention was made in view of the above-mentioned problems, and it is possible to measure the three-dimensional shape of a measurement object based on a pattern image obtained by imaging a predetermined pattern of light irradiated onto the measurement object. The object of the present invention is to enable acquisition of irradiation defect areas where irradiation defects occur.

A computing device for three-dimensional measurement according to the present invention includes an object support section that supports a measurement object having a substrate and a component mounted on the substrate, and a predetermined pattern on the measurement object supported by the object support section. A pattern obtained by a three-dimensional measuring device that includes a pattern irradiation section that irradiates light from the pattern irradiation section and an imaging section that obtains a two-dimensional pattern image by capturing the light irradiated onto the measurement target from the pattern irradiation section. A three-dimensional measurement calculation device that executes calculations to measure the three-dimensional shape of a measurement target based on an image, and includes a component mounting range where components are mounted on a board and a component mounted in the component mounting range. There is a reference model acquisition unit that obtains a reference model that shows the external shape, a shadow area where a shadow of the component is created on the board because the light irradiated from the imaging unit to the component is blocked by the component, and a shadow area where the light irradiated from the imaging unit is reflected by the component. Area calculation that calculates the irradiation failure area in at least one of the secondary reflection areas where the light enters the board based on irradiation direction information indicating the direction in which the component is irradiated with light using a three-dimensional measuring device and a reference model. It is equipped with a section.

The three-dimensional measurement program according to the present invention causes a computer to function as the above-mentioned three-dimensional measurement calculation device.

A recording medium according to the present invention records the three-dimensional measurement program described above so as to be readable by a computer.

A three-dimensional measuring device according to the present invention includes an object support section that supports a measurement object having a substrate and components mounted on the substrate, and a predetermined pattern of light on the measurement object supported by the object support section. a pattern irradiation unit that irradiates the measurement target, an imaging unit that acquires a two-dimensional pattern image by capturing the light irradiated onto the measurement target from the pattern irradiation unit, and a three-dimensional shape of the measurement target based on the pattern image. and the above-mentioned three-dimensional measurement arithmetic device that executes arithmetic operations for the three-dimensional measurement.

A calculation method for three-dimensional measurement according to the present invention includes an object support section that supports a measurement object having a substrate and a component mounted on the substrate, and a predetermined pattern on the measurement object supported by the object support section. A pattern obtained by a three-dimensional measuring device that includes a pattern irradiation section that irradiates light from the pattern irradiation section and an imaging section that obtains a two-dimensional pattern image by capturing the light irradiated onto the measurement target from the pattern irradiation section. A calculation method for three-dimensional measurement that executes calculations to measure the three-dimensional shape of a measurement target based on an image, and includes a component mounting range where components are mounted on a board and a component mounted in the component mounting range. The process of acquiring a reference model that shows the external shape, the shadow area where the part's shadow is created on the board due to the part blocking the light irradiated from the imaging unit to the part, and the shadow area where the part's shadow is created by the part blocking the light irradiated from the imaging unit and reflected by the part. The method includes a step of calculating an irradiation failure region of at least one of the secondary reflection regions incident on the substrate based on irradiation direction information indicating a direction in which light is irradiated onto the component by a three-dimensional measuring device and a reference model.

In the present invention (three-dimensional measurement calculation device, three-dimensional measurement program, recording medium, three-dimensional measurement device, and three-dimensional measurement calculation method) configured as described above, the component mounting range where components are mounted on the board A reference model indicating the external shape of the component to be mounted in the component mounting range is obtained. Then, a poor irradiation area is calculated based on the irradiation direction information indicating the direction in which the component is irradiated with light by the three-dimensional measuring device and the reference model. In this way, when measuring the three-dimensional shape of the measurement target based on the pattern image obtained by imaging the predetermined pattern of light irradiated onto the measurement target, the illumination defect area where light irradiation failure occurs is obtained. It is now possible to do so.

Here, the poor illumination area is defined as a shadow area where the component's shadow occurs on the board due to the part blocking the light irradiated from the imaging unit to the component, and a shadow area where the light irradiated from the imaging unit and reflected by the component enters the board. at least one of the secondary reflection areas.

In addition, the reference model acquisition unit acquires inspection data indicating a standard for inspecting the suitability of the positional relationship between the components mounted in the component mounting range and the component mounting range, and CAD (Computer-Aided Design) data indicating the configuration of the board. The three-dimensional measurement arithmetic device may be configured to create a reference model from at least one of the data. With such a configuration, a reference model is created using existing data such as board inspection data or CAD data, and a defective irradiation area can be calculated based on this reference model. Note that the information indicated by the inspection data is not limited to the above criteria, but may also include criteria for inspecting the suitability of the component itself and criteria for inspecting the suitability of the bond between the component and the board.

Further, the area calculation unit configures the three-dimensional measurement calculation device to further calculate the poor irradiation area based on the result of recognizing the position of the substrate carried into the three-dimensional measurement device and supported by the object support unit. You may. With this configuration, it is possible to accurately calculate the irradiation failure area according to the actual position of the substrate loaded into the three-dimensional measurement device.

Furthermore, the three-dimensional measurement computing device may be configured such that the position of the substrate supported by the object support section is recognized based on the result of imaging a fiducial mark attached to the substrate by the imaging section. good. With this configuration, the irradiation defect area can be accurately calculated according to the position of the substrate loaded into the three-dimensional measurement device.

Furthermore, the area calculation unit may configure the three-dimensional measurement calculation device to further calculate the irradiation failure area based on the result of detecting the position of the component mounted in the component mounting range. With this configuration, it is possible to accurately calculate the irradiation failure area according to the actual position of the component mounted in the component mounting range.

Furthermore, the three-dimensional measurement arithmetic device may be configured so that the position of the component mounted in the component mounting range is detected based on the result of imaging the component. With this configuration, it is possible to accurately calculate the irradiation failure area according to the actual position of the component mounted in the component mounting range.

Further, the pattern irradiation unit has a plurality of projectors that irradiate light onto the component from mutually different directions, and the irradiation direction information indicates, for each of the plurality of projectors, the direction in which the component is irradiated with light from the projector. The calculation unit may configure a three-dimensional measurement calculation device so as to calculate a poor irradiation area for each of the plurality of projectors. With such a configuration, it is possible to obtain a poor irradiation area that occurs when a component is irradiated with light from each of a plurality of projectors.

In addition, based on the pattern image obtained by capturing the light irradiated onto the board from the projector using the imaging unit, calculations are performed to calculate unidirectional shape data that indicates shape-related values, which are values related to the three-dimensional shape, for each pixel. , further comprising a shape calculation section that calculates a plurality of unidirectional shape data by executing for each of the plurality of projectors, the shape calculation section calculating an average of shape-related values indicated by each of the plurality of unidirectional shape data for each pixel. A first integrated calculation process is performed that integrates multiple unidirectional shape data by calculating the three-dimensional shape of the measurement target, and in the first integrated calculation process, the shape relationship indicated by the unidirectional shape data is calculated. Three-dimensional measurement is performed so that the three-dimensional shape of the measurement object is calculated by a weighted average in which the shape-related values of pixels included in the poor irradiation area are multiplied by a weighting coefficient of less than 1 and greater than or equal to 0. A computing device may also be configured.

In such a configuration, unidirectional shape data indicating a value related to a three-dimensional shape (shape-related value) for each pixel is calculated based on a pattern image obtained by capturing an image of light irradiated onto the board from a projector using an imaging unit. Ru. In other words, the unidirectional shape data is data regarding a three-dimensional shape calculated based on a pattern image obtained by capturing light irradiated onto a component from one projector. The calculation for calculating this unidirectional shape data is executed for each of the plurality of projectors. As a result, unidirectional shape data obtained when the component is irradiated with light from each of the plurality of projectors is obtained. The plurality of unidirectional shape data obtained in this way are based on pattern images obtained while irradiating the component with light from different directions. Therefore, a pixel that corresponds to a poorly irradiated region in one unidirectional shape data may correspond to a region well irradiated with light in another unidirectional shape data. Therefore, by calculating a weighted average of the shape-related values indicated by each of the plurality of unidirectional shape data for each pixel, it is possible to integrate the plurality of unidirectional shape data and calculate the three-dimensional shape of the measurement target object. Moreover, in the first integrated calculation process, among the shape-related values indicated by the unidirectional shape data, the shape-related values of pixels included in the poorly irradiated area are weighted averaged by multiplying them by a weighting coefficient of less than 1 and greater than or equal to 0. A three-dimensional shape of the object to be measured is calculated. This makes it possible to appropriately calculate the three-dimensional shape of the measurement target while suppressing the influence of the poorly irradiated area.

In addition, in the first integrated calculation process, the three-dimensional shape of the measurement target is calculated by a weighted average in which shape-related values of pixels included in the poor irradiation area are multiplied by a weighting coefficient of 0. A measurement arithmetic device may also be configured. In this way, the operation of multiplying by a weighting coefficient of 0 is an operation of excluding a poorly irradiated region. This makes it possible to appropriately calculate the three-dimensional shape of the measurement target while eliminating the influence of poor irradiation areas.

Note that various specific contents of the shape-related value are assumed. For example, the shape-related value may be given by the product of the calculated value of the three-dimensional shape calculated based on the pattern image and the reliability of the calculated value, and the shape-related value is calculated based on the pattern image. It may be a calculated value of a three-dimensional shape.

In addition, the shape calculation unit integrates the plurality of unidirectional shape data by calculating, for each pixel, a representative value of shape-related values indicated by the plurality of unidirectional shape data without weighting according to the poor irradiation area. , by performing both the second integrated calculation process for calculating the three-dimensional shape of the measurement object and the first integrated calculation process on the same measurement object, the first integrated calculation process and the second integrated calculation process are performed. The first integrated calculation is performed by performing an evaluation process for evaluating the difference between the three-dimensional shapes of the measurement objects calculated in each case on a predetermined number of measurement objects whose pattern images are to be acquired by the three-dimensional measuring device. After the necessity determination is executed to determine whether or not the process is necessary, and it is determined in the necessity determination that the first integrated calculation process is unnecessary, the shape calculation unit performs the second integration process without executing the first integrated calculation process. The three-dimensional measurement arithmetic device may be configured to calculate the three-dimensional shape of the object to be measured through arithmetic processing. In such a configuration, when the poor irradiation area is not a problem, the three-dimensional shape of the measurement object can be accurately calculated by the second integrated calculation process, which is simpler than the first integrated calculation process. Here, the representative value of the shape-related values indicated by the plurality of unidirectional shape data, which is not weighted according to the poor irradiation area, includes an average value or a median value based on a simple average.

Note that the three-dimensional measurement arithmetic device may be configured to further include a setting operation section that accepts an operation to set a predetermined number. With this configuration, the user can appropriately adjust the period during which both the first integrated calculation process and the second integrated calculation process are executed.

According to the present invention, when measuring the three-dimensional shape of a measurement target based on a pattern image obtained by imaging a predetermined pattern of light irradiated onto the measurement target, irradiation that causes light irradiation failure It becomes possible to acquire defective areas.

FIG. 1 is a block diagram schematically illustrating a three-dimensional measuring device according to the present invention. FIG. 2 is a plan view schematically showing the relationship between a projector included in the three-dimensional measurement device of FIG. 1 and an imaging field of view. FIG. 2 is a block diagram showing details of a control device included in the three-dimensional measuring device of FIG. 1. FIG. FIG. 3 is a diagram schematically showing the contents of a part model created by a part model acquisition unit. FIG. 3 is a diagram showing an example of board configuration data used for creating a component model in a table format. The figure which shows an example of part data used for creating a part model in a table format. A diagram schematically showing an example of a created part model. FIG. 3 is a diagram schematically showing the contents of a poorly irradiated area calculated by an area calculation unit. 5 is a flowchart illustrating an example of a calculation for calculating a defective irradiation area. A flowchart showing a first example of three-dimensional measurement. 9 is a flowchart showing omnidirectional shape data acquisition executed in the three-dimensional measurement of FIG. 8; 9 is a flowchart showing the first data integration process executed in the three-dimensional measurement of FIG. 8. 11 is a diagram schematically showing calculations executed in the first data integration process of FIG. 10. FIG. 8 is a flowchart showing an example of calculation for correcting the irradiation defect area calculated in FIG. 7. FIG. A flowchart showing a second example of three-dimensional measurement. 14 is a flowchart showing the second data integration process executed in the three-dimensional measurement of FIG. 13. 14 is a diagram schematically showing the contents of calculations executed in the three-dimensional measurement of FIG. 13. FIG.

FIG. 1 is a block diagram schematically illustrating a three-dimensional measuring device according to the present invention. In this figure and the following figures, the X direction, which is a horizontal direction, the Y direction, which is a horizontal direction orthogonal to the X direction, and the Z direction, which is a vertical direction, are shown as appropriate. The three-dimensional measuring device 1 of FIG. 1 measures the three-dimensional shape (external shape) of the measurement target J by controlling the conveyor 2, the measuring head 3, and the drive mechanism 4 by the control device 100. The measurement object J is composed of a board B (printed board) and a component E mounted on the board B. This measurement object J is produced by a surface mounter that mounts the component E on the board B, and is carried into the three-dimensional measuring device 1.

The conveyor 2 conveys the measurement target object J along a predetermined conveyance path. Specifically, the conveyor 2 carries the measurement target J before measurement to the measurement position in the three-dimensional measurement device 1, and holds the measurement target J at the measurement position so that the substrate B is horizontal. Further, when the measurement of the three-dimensional shape of the measurement object J at the measurement position is completed, the conveyor 2 carries the measured measurement object J out of the three-dimensional measurement device 1.

The measurement head 3 has an imaging camera 31 that images the inside of the imaging field of view V31 from above, and the imaging camera 31 captures an image of the measurement object J carried into the measurement position within the imaging field of view V31. The imaging camera 31 includes a solid-state image sensor 311 that detects reflected light from the measurement target object J, and captures an image of the measurement target object J using the solid-state image sensor 311.

Further, the measurement head 3 includes a projector 32 that projects striped pattern light L(S) in which the light intensity distribution changes sinusoidally onto the imaging field of view V31. The projector 32 includes a light source such as an LED (Light Emitting Diode), and a digital micromirror device that reflects light from the light source toward the imaging field of view V31. By adjusting the angle of each micromirror of the digital micromirror device, the projector 32 can project a plurality of types of patterned light L(S) having mutually different phases onto the imaging field of view V31. In other words, the measurement head 3 captures an image using the imaging camera 31 while changing the phase of the patterned light L(S) projected from the projector 32, so that the measurement target J within the imaging field of view V31 can be three-dimensionally captured using the phase shift method. Shape can be measured.

FIG. 2 is a plan view schematically showing the relationship between a projector included in the three-dimensional measuring device of FIG. 1 and an imaging field of view. As shown in FIG. 2, the measurement head 3 has a plurality of (in this example, four) projectors 32 (in FIG. 1, two projectors 32 are representative for simplicity of illustration). ). Each projector 32 projects pattern light L(S) from diagonally above onto the imaging field of view V31 of the imaging camera 31. In plan view, the plurality of projectors 32 are arranged so as to surround the imaging camera 31, and are arranged circumferentially at equal pitches with the vertical direction Z as the center. Therefore, the plurality of projectors 32 project pattern light L(S) onto the imaging field of view V31 from mutually different projection directions D. Note that the number of projectors 32 included in the measurement head 3 is not limited to four as in the example of FIG. 2 .

Furthermore, the measurement head 3 has an illumination 33 (FIG. 1) that irradiates light onto the imaging field of view V31. The above projector 32 projects the pattern light L(S) onto the imaging field of view V31 when measuring a three-dimensional shape, whereas the illumination 33 projects the pattern light L(S) onto the imaging field of view V31 when measuring a three-dimensional shape. The field of view V31 is irradiated with illumination light.

The drive mechanism 4 supports the measurement head 3 and drives the measurement head 3 in the X direction, Y direction, and Z direction using a motor. By driving this drive mechanism 4, the measurement head 3 can move above the measurement target part of the measurement target object J and capture the measurement target part within the imaging field of view V31. The three-dimensional shape of a location can be measured. In particular, the three-dimensional measuring device 1 places the component E within the imaging field of view V31 so as to help inspect the floating of the component E from the substrate B, for example, the floating of the terminal of a package such as a QFP (Quad Flat Package) from the substrate B. At the same time, the three-dimensional shape of the part E and its surroundings (measurement target location) can be measured.

The control device 100 has a main control section 110, which is a processor composed of a CPU (Central Processing Unit) and memory, and the main control section 110 controls the various parts of the device, so that the three-dimensional shape can be measured. be done. Also. The control device 100 has a UI (User Interface) 200 configured with input/output devices such as a display, a keyboard, and a mouse. A user can input commands to the control device 100 via the UI 200, and 100 measurement results can be confirmed. Further, the control device 100 includes a projection control section 120, an imaging control section 130, and a drive control section 140. The projection control unit 120 controls the projection of the pattern light L(S) by the projector 32. The imaging control unit 130 controls the imaging of the imaging field of view V31 by the imaging camera 31 and the irradiation of light from the illumination 33 to the imaging field of view V31. The drive control unit 140 controls the drive of the measurement head 3 by the drive mechanism 4 .

When the conveyor 2 carries the measurement object J to the measurement position, the main control section 110 controls the drive mechanism 4 by the drive control section 140 to move the measurement head 3 above the measurement point of the measurement object J. let As a result, the measurement target location falls within the imaging field of view V31 of the imaging camera 31. Next, the main control unit 110 projects the pattern light L(S) from the projector 32 onto the imaging field of view V31 and images the patterned light L(S) projected onto the imaging field of view V31 with the imaging camera 31 (pattern imaging operation). ). Specifically, the control device 100 has a storage unit 150, and reads out the projection pattern T(S) stored in the storage unit 150. The main control unit 110 then controls the projection control unit 120 based on the projection pattern T(S) read from the storage unit 150 to adjust the angle of each micromirror of the digital micromirror device of the projector 32. Adjust according to pattern T(S). In this way, the patterned light L(S) having the projection pattern T(S) is projected onto the imaging field of view V31. Further, the main control unit 110 controls the imaging control unit 130 to capture the pattern light L(S) projected onto the imaging field of view V31 using the imaging camera 31 to obtain a pattern image I(S). This pattern image I is stored in the storage section 150. Note that the storage unit 150 stores four types of projection patterns T(S) whose phases differ by 90 degrees from each other, and the pattern imaging operation is executed four times while changing the projection pattern T(S). S=1, 2, 3, 4). As a result, four types of pattern images I(S) are obtained by capturing patterned light L(S) each having a phase different by 90 degrees. The main control unit 110 determines the height of the imaging field of view V31 for each pixel of the imaging camera 31 from the four types of pattern images I(S) acquired in this way using the phase shift method. Note that variations in the projection pattern T(S) are not limited to this example; for example, eight types of projection patterns T(S) whose phases differ by 45 degrees may be used, or three types of projection patterns T(S) whose phases differ by 120 degrees may be used. A projection pattern T(S) may also be used.

FIG. 3 is a block diagram showing details of a control device included in the three-dimensional measuring device of FIG. 1. The control device 100 stores the three-dimensional measurement program 92 downloaded from the external server computer 91 in the storage unit 150. Note that the manner in which the three-dimensional measurement program 92 is provided is not limited to external downloading, and the three-dimensional measurement program 92 may be provided in a state recorded on a DVD (Digital Versatile Disc) or USB (Universal Serial Bus). . When the main control section 110 executes the three-dimensional measurement program 92, the main control section 110 includes a component model acquisition section 111, an area calculation section 112, a pattern image acquisition section 113, and a shape calculation section 114. These functional units 111, 112, 113, and 114 perform a function to correspond to a poor irradiation area that occurs due to the component E on the board B when the pattern light L(S) is projected onto the measurement target J. . Details of these are as follows.

FIG. 4 is a diagram schematically showing the contents of a component model created by the component model acquisition unit, and FIG. 5A is a diagram showing an example of board configuration data used for creating the component model in a table format. , FIG. 5B is a diagram showing an example of component data used for creating a component model in a table format, and FIG. 5C is a diagram schematically showing an example of the created component model. The component model acquisition unit 111 uses board configuration data 81 indicating the configuration of the board B and component data 82 indicating the external shape of the component E to be mounted on the board B to obtain a three-dimensional model of the component E mounted on the board B. A part model 83 is created that shows the existing range in the original. Note that the board configuration data 81 and the component data 82 are stored in the storage unit 150 in advance.

The board configuration data 81 (FIG. 5A) indicates the component mounting range Ref(R) provided on the board B and the type of component E to be mounted in the component mounting range Ref(R), for each component mounting range Ref( 1), Ref(2), Ref(3),... (R=1, 2, 3,...). In the board configuration data 81, the component mounting range Ref(R) is defined by the position of the component mounting range Ref(R) in the X direction (X coordinate) and the position of the component mounting range Ref(R) in the Y direction (Y coordinate). Identified by Note that the component mounting range Ref(R) is defined by, for example, lands provided on the board B, and is a wide range. On the other hand, the position (x, y) of the component mounting range Ref(R) corresponds to the center of the component mounting range Ref(R) in plan view. The component mounting range Ref(R) indicated by the board configuration data 81 is the ideal range in which the component E should be mounted, and is not the range in which the component E is actually mounted by the surface mounter. The board configuration data 81 includes CAD data indicating the configuration of the board B, and checking the suitability of the positional relationship between the component E mounted in the component mounting range Ref(R) and the component mounting range Ref(R) in the surface mounter. Inspection data or the like can be used to indicate standards for

As shown in FIG. 5B, the component data 82 shows the configuration of the component E for each type of component E. Specifically, the component data 82 includes the external shape of the rectangular component E such as the length El, width Ew, and height Eh, and the presence or absence of the property of reflecting light (patterned light L(S)) (reflection). ) are shown for various parts Ea, Eb, Ec, .

The component model acquisition unit 111 confirms the type of component E to be mounted in the component mounting range Ref(R) of the board B using the board configuration data 81, and confirms the configuration of the corresponding type of component E using the component data 82. A part model 83 is created based on. This component model 83 includes the position (x, y) of the component mounting range Ref(R) and the external shape of the component E mounted in the component mounting range Ref(R) (X size E_x, Y size E_y, and Z size E_z). In other words, the component model 83 indicates the range in which the component E mounted in the component mounting range Ref(R) protrudes from the board B in the Z direction (height direction). Furthermore, the component model 83 indicates whether the component E mounted in the component mounting range Ref(R) has the property of reflecting light (pattern light L(S)). This component model 83 is created for each of the plurality of component mounting ranges Ref(R) provided on the board B.

Incidentally, the specific data used to create the part model 83 and the specific method of creating the part model 83 using each data are not limited to this example. For example, when the component model 83 is input by the user's operation on the UI 200, the component model acquisition unit 111 does not need to create the component model 83 based on the board configuration data 81 and the component data 82; What is necessary is to acquire the part model 83 that has been created.

FIG. 6 is a diagram schematically showing the contents of the poor irradiation area calculated by the area calculation unit. Component E shown in FIG. 6 has a property of reflecting pattern light L(S). In FIG. 6, the component E mounted in the component mounting range Ref(R) has a width E_x in the X direction, a length E_y in the Y direction, and a height E_z in the Z direction. Then, the pattern light L(S) emitted from the projector 32 is projected onto the component E from the projection direction D.

As a result, in plan view, a shadow area As (poor irradiation area) occurs downstream of the component E in the projection direction D. This shadow area As is an area where a shadow of the component E is generated on the board B because the pattern light L(S) projected onto the component E from the projector 32 is blocked by the component E. Furthermore, in plan view, a secondary reflection area Ar (defective irradiation area) occurs upstream of the component E in the projection direction D. This secondary reflection area Ar is an area where light projected onto the component E from the projector 32 and reflected by the side surface of the component E enters the substrate B. In this example, the length of the shadow area As and the secondary reflection area Ar in the Y direction corresponds to the length E_y of the component E in the Y direction. Further, the width of the shadow area As and the secondary reflection area Ar in the X direction corresponds to the width obtained by multiplying the height E_z of the component E by the tangent (tanθs) of the projection angle θs. Note that for the component E that does not reflect the pattern light L(S), no secondary reflection area Ar is generated.

Such a poor irradiation area (shadow area As/secondary reflection area Ar) is calculated for each of the plurality of projectors 32. Next, this point will be explained using FIG. 7. FIG. 7 is a flowchart showing an example of calculation for calculating the irradiation defect area. The flowchart in FIG. 7 is executed by the calculations of the main control unit 110.

In step S101, the count value P (=1, 2, 3, 4) that identifies the projector 32 is reset to zero, and in step S102, the count value P is incremented. Furthermore, in step S103, the count value R (=1, 2, 3, . . . ) identifying the component mounting range Ref is reset to zero, and in step S104, the count value R is incremented.

In step S105, the component model acquisition unit 111 creates a component model 83 of the component E to be mounted in the R-th component mounting range Ref(R). In step S106, the area calculation unit 112 confirms the projection direction D in which the P-th projector 32 projects the patterned light L(S) with respect to the R-th component mounting range Ref. Specifically, projection direction information 84 indicating the projection direction D of the patterned light L(S) by the projector 32 for each of the plurality of projectors 32 is stored in advance in the storage unit 150, and the area calculation unit 112 The projection direction D is confirmed by referring to the direction information 84.

In step S107, a shadow area occurs when pattern light L(S) is projected from the projection direction D by the P-th projector 32 onto the component E indicated by the component model 83 created for the R-th component mounting range Ref. As (poor reflection area) is calculated by the area calculation unit 112. Furthermore, when the component E indicated by the component model 83 created for the R-th component mounting range Ref reflects the pattern light L(S), the P-th projector 32 projects the component E in the projection direction D. The area calculation unit 112 calculates a secondary reflection area Ar (defective reflection area) that occurs when patterned light L(S) is projected from the area.

In step S108, it is determined whether the count value R of the component mounting range Ref has reached the number Rmax of component mounting ranges Ref provided on the board B, that is, for all the component mounting ranges Ref provided on the board B in steps S105 to S107. It is checked whether the has been executed. If there is a component mounting range Ref for which steps S105 to S107 have not been performed (“NO” in step S108), the process returns to step S104. In this way, steps S104 to S107 are repeated until steps S105 to S107 are executed for all component mounting ranges Ref ("YES" in step S108). As a result, the illumination failure area (shadow area As/secondary reflection area Ar) that occurs when the pattern light L(S) is projected from the P-th projector 32 is calculated for the component E in all component mounting ranges Ref. .

In step S109, it is checked whether the count value P of the projectors 32 has reached the number Pmax (=4) of the projectors 32, that is, whether steps S103 to S108 have been executed for all projectors 32. If there is a projector 32 that has not executed steps S103 to S108 (“NO” in step S109), the process returns to step S102. In this way, steps S102 to S108 are repeated until steps S103 to S108 are executed for all projectors 32 (“YES” in step S109). As a result, when the pattern light L(S) is projected from the projector 32, the poor irradiation area (shadow area As/secondary reflection area Ar) that occurs due to the component E in each component mounting range Ref is removed from all the projectors. Poor irradiation area information 85 shown for 32 is calculated and stored in the storage unit 150.

Three-dimensional measurement of the measurement target J is performed using the irradiation defect area information 85 acquired in this way (FIG. 8). FIG. 8 is a flowchart showing a first example of three-dimensional measurement, FIG. 9 is a flowchart showing omnidirectional shape data acquisition executed in three-dimensional measurement in FIG. 8, and FIG. 11 is a flowchart showing the first data integration process to be executed, and FIG. 11 is a diagram schematically showing the calculations to be executed in the first data integration process in FIG. 10. The flowcharts in FIGS. 8, 9, and 10 are executed under the control of the main control unit 110.

In the three-dimensional measurement in FIG. 8, omnidirectional shape data acquisition is executed in step S201. As shown in FIG. 9, in omnidirectional shape data acquisition, recognition of the fiducial mark attached to the substrate B carried into the three-dimensional measuring device 1 is executed (step S301). Specifically, with the imaging camera 31 facing the fiducial mark in the imaging field of view V31 from above, the imaging camera 31 illuminates the fiducial mark while irradiating illumination light from the illumination 33 to the fiducial mark. Take an image and obtain a mark image. The mark image is sent from the imaging control section 130 to the main control section 110, and the main control section 110 detects the position of the substrate B held on the transport conveyor 2 from the position of the fiducial mark appearing in the mark image.

In step S302, the count value P of the projector 32 is reset to zero, and in step S303, the count value P is incremented. Further, in step S304, the count value R of the component mounting range Ref is reset to zero, and in step S305, the count value R is incremented.

To start step S306, the main control unit 110 adjusts the position of the board B using the drive mechanism 4 so that the Rth component mounting range Ref falls within the imaging field of view V31. In particular, the main control unit 110 executes position adjustment of the substrate B to the imaging field of view V31 based on the position of the substrate B detected in step S301. Then, in step S306, the pattern image acquisition unit 113 controls the projection control unit 120 and the imaging control unit 130 to transmit pattern light L(S) from the P-th projector 32 to the component E in the R-th component mounting range Ref. While projecting the pattern light L(S), the image pickup camera 31 images the pattern light L(S). As a result, as described above, four types of pattern images I(S) corresponding to mutually different phases are acquired (S=1, 2, 3, 4) and stored in the storage unit 150.

In step S307, the shape calculation unit 114 calculates unidirectional shape data 86 based on the four types of pattern images I(S) using the phase shift method. This unidirectional shape data 86 is data that indicates, for each pixel PX (FIG. 11), a shape-related value Q regarding the three-dimensional shape of the component E mounted in the component mounting range Ref. Here, the pixel PX corresponds to, for example, a pixel of the solid-state image sensor 311, and since the imaging camera 31 takes an image while facing the substrate B from the Z direction, the plurality of pixels of the solid-state image sensor 311 are different from each other on the substrate B. Corresponds to the position (X coordinate, Y coordinate). This shape-related value Q is composed of a measured value Qm calculated as a height (Z coordinate) from four types of pattern images I(S) by a phase shift method, and a reliability Qr of the measured value Qm. In particular, in this example, the shape-related value Q is given by the product of the measured value Qm and the reliability Qr.

Incidentally, in the phase shift method, the reliability Qr at each pixel PX is calculated based on the difference in luminance shown by the pixels PX of four types of pattern images I(S) (S = 1, 2, 3, 4). Can be done. Specifically, when four types of pattern images I(S) are captured while shifting the phase by π/2 and each luminance value is d0, d1, d2, and d3, the phase shift angle α is calculated by the following formula. α=atan[(d2-d0)/(d3-d1)]
The reliability Qr is calculated by the following formula: Qr=[(d2-d0) ² +(d3-d1) ² ] ^1/2
Calculated by

Note that the pixel PX used when determining the shape-related value Q does not need to correspond to the pixel of the solid-state image sensor 311. For example, when image processing is performed to convert the resolution of an image captured by the solid-state image sensor 311, the shape-related value Q may be calculated using the pixel PX corresponding to the converted resolution.

In step S308, it is determined whether the count value R of the component mounting range Ref has reached the number Rmax of component mounting ranges Ref provided on the board B, that is, the pattern image I( It is confirmed whether acquisition of S) (step S306) and calculation of unidirectional shape data 86 (step S307) have been performed. If there is a component mounting range Ref for which steps S306 to S307 have not been performed (“NO” in step S308), the process returns to step S305. In this way, steps S306 to S307 is repeated. As a result, unidirectional shape data 86 when patterned light L(S) is projected from the P-th projector 32 is calculated for the components E in all component mounting ranges Ref.

In step S309, it is checked whether the count value P of the projectors 32 has reached the number Pmax (=4) of the projectors 32, that is, whether steps S304 to S308 have been executed for all projectors 32. If there is a projector 32 that has not executed steps S304 to S308 (“NO” in step S309), the process returns to step S303. In this way, steps S304 to S308 are repeated until steps S304 to S308 are executed for all projectors 32 (“YES” in step S309). As a result, the unidirectional shape data 86 calculated for the component E in each component mounting range Ref when the pattern light L(S) is projected from the projector 32 is acquired for all the projectors 32 and saved in the storage unit 150. be done.

When the omnidirectional shape data acquisition of the three-dimensional measurement in FIG. 9 (step S201 in FIG. 8) is thus completed, the first data integration process (step S202) is executed. In the first data integration process shown in FIG. 10, the count value R (=1, 2, 3, ...) that identifies the component mounting range Ref is reset to zero (step S401), and the count value R is incremented ( Step S402). As a result, the four unidirectional shape data 86 obtained by projecting the projection pattern T(S) from each of the four projectors 32 on the component E mounted in the R-th component mounting range Ref. It is specified. In other words, for the component E in the R-th component mounting range Ref, four pieces of unidirectional shape data 86 corresponding to the four projectors 32 are specified.

Subsequently, the count value N (N=1, 2, 3, ...) that identifies the pixel PX is reset to zero (step S403), and the count value N is incremented (step S404). Furthermore, in step S405, the count value P (=1, 2, 3, 4) for identifying the projector 32 is reset to zero, and in step S406, the count value P is incremented. As a result, the Nth pixel PX of the unidirectional shape data 86 corresponding to the Pth projector 32 is specified.

Then, the shape calculation unit 114 determines whether the Nth pixel PX designated in this way belongs to the shadow area As or the secondary reflection area Ar based on the illumination defect area information 85 in the storage unit 150 (step S407 ), a weighting coefficient W(P) to be multiplied by the shape-related value Q (=measured value Qm×reliability Qr) of the Nth pixel PX is determined (steps S408, S409). In other words, when it is determined based on the illumination defect area information 85 that the Nth pixel PX belongs to the shadow area As or the secondary reflection area Ar ("YES" in step S407), the shape-related value Q of the Nth pixel PX is The weighting coefficient W(P) is determined to be "0", and the shape-related value Q is excluded (step S408). On the other hand, if it is determined based on the illumination defect area information 85 that the N-th pixel PX does not belong to either the shadow area As or the secondary reflection area Ar ("NO" in step S407), the shape of the N-th pixel PX The weighting coefficient W(P) for the related value Q is determined to be 1 (step S410).

The calculations in steps S407 to S409 for the shape-related value Q of the Nth pixel PX are performed while incrementing the count value P of the projector 32 (step S406) until the count value P reaches Pmax (=4) (step S410). (“YES”), is repeated. As a result, the weighting coefficient W(P) for the shape-related value Q of the N-th pixel PX corresponding to P=1, 2, 3, and 4 is determined. In the example of FIG. 11, it is determined that the Nth pixel PX corresponding to the projector 32 with P=2 belongs to the shadow area As or the secondary reflection area Ar, and the weighting coefficient W( 2) is determined to be "0". On the other hand, the N-th pixel PX corresponding to each of the projectors 32 of P=1, 3, and 4 is determined not to belong to either the shadow area As or the secondary reflection area Ar, and the shape-related value Q of these pixels PX The weighting coefficients W(1), W(3), and W(4) for the above are determined to be "1".

In step S411, the shape calculation unit 114 calculates the weighting coefficient determined in steps S407 to S409 for the shape-related value Q of the Nth pixel PX corresponding to each projector 32 of P=1, 2, 3, and 4. By executing a calculation to obtain a weighted average using W(P), an integrated value H that integrates these shape-related values Q is obtained. The weighted average formula for determining the integrated value H is as shown in the "conversion formula" column of FIG. 11. As a result, in the example of FIG. 11, the integrated value H of the Nth pixel PX is 159.

The calculations in steps S405 to S411 are performed while incrementing the count value N (step S404) until the count value N reaches the number Nmax of pixels PX constituting the unidirectional shape data 86 ("YES" in step S412). Repeated. In this way, integrated shape data 87 indicating, for each pixel PX, an integrated value H obtained by integrating four shape-related values Q corresponding to mutually different projectors 32 is calculated and stored in the storage unit 150. This integrated shape data 87 corresponds to data that is an integration of four pieces of unidirectional shape data 86 corresponding to the four projectors 32 acquired for the component E mounted in the R-th component mounting range Ref. The three-dimensional shape of E is shown.

The calculations in steps S403 to S412 are repeated while incrementing the count value R (step S402) until the count value R reaches the number Rmax of the component mounting range Ref (“YES” in step S413). In this way, the integrated shape data 87 is calculated for the components E in all component mounting ranges Ref.

In the embodiment described above, a component model 83 (reference model) indicating the component mounting range Ref in which the component E is mounted on the board B and the external shape of the component E to be mounted in the component mounting range Ref is acquired ( Step S105). Based on the projection direction information 84 (irradiation direction information) indicating the projection direction D in which the pattern light L(S) is projected onto the component E by the three-dimensional measuring device 1 and the component model 83, the shadow area As and the secondary A reflection area Ar (defective irradiation area) is calculated (step S107). In this way, when measuring the three-dimensional shape of the measurement object J based on the pattern image I(S) acquired by imaging the pattern light L(S) irradiated onto the measurement object J, the irradiation of the pattern light L It is possible to obtain the shadow area As and the secondary reflection area Ar where defects occur.

The component model acquisition unit 111 (reference model acquisition unit) also acquires inspection data indicating a standard for inspecting the suitability of the positional relationship between the component E mounted in the component mounting range Ref and the component mounting range Ref, and the configuration of the board. A part model 83 is created from at least one of the CAD data indicating (step S105). With this configuration, the component model 83 is created by utilizing existing data such as inspection data or CAD data of the board B, and the shadow area As and the secondary reflection area Ar can be calculated based on this component model 83.

Further, a plurality of projectors 32 (pattern irradiation units) are provided that irradiate pattern light L(S) onto the component E from mutually different projection directions D, and the projection direction information 84 indicates that the light irradiates from the projector 32 onto the component E. A projection direction D is shown for each of the plurality of projectors 32. In this configuration, if the projector 32 that projects the pattern light L(S) changes among the plurality of projectors 32, the shadow area As and the secondary reflection area Ar also change. In contrast, the area calculation unit 112 calculates the shadow area As and the secondary reflection area Ar for each of the plurality of projectors 32 (steps S102, S107). Thereby, it is possible to obtain the shadow area As and the secondary reflection area Ar that occur when light is projected onto the component E from each of the plurality of projectors 32.

Also, based on the pattern image I(S) obtained by capturing the pattern light L(S) irradiated onto the substrate B from the projector 32 with the imaging camera 31 (imaging unit), the shape is a value related to the three-dimensional shape. An operation (step S307) for calculating unidirectional shape data 86 indicating the related value Q for each pixel PX is executed by the shape calculation unit 114. This calculation (step S307) is executed for each of the plurality of projectors 32 (step S303), and a plurality of unidirectional shape data 86 corresponding to mutually different projectors 32 is calculated. Furthermore, the shape calculation unit 114 integrates the plurality of unidirectional shape data 86 by calculating a weighted average of the shape-related values Q indicated by each of the plurality of unidirectional shape data 86 for each pixel PX, and The first integrated calculation process (FIG. 10) for calculating the three-dimensional shape of is executed. In this first integrated calculation process, among the shape-related values Q indicated by the unidirectional shape data 86, the shape-related value Q of the pixel PX included in the shadow area As or the secondary reflection area Ar is less than 1 and greater than or equal to 0. The three-dimensional shape of the measurement object J is calculated by the weighted average multiplied by the weighting coefficient W(P) (step S411).

In this configuration, a shape-related value Q regarding a three-dimensional shape is calculated from a pixel based on a pattern image I(S) obtained by capturing pattern light L(S) irradiated onto the substrate B from the projector 32 by the imaging camera 31. Unidirectional shape data 86 shown for each PX is calculated (step S307). In other words, the unidirectional shape data 86 is data regarding a three-dimensional shape calculated based on a pattern image I(S) obtained by capturing the patterned light L(S) irradiated onto the component E from one projector 32. The calculation for calculating this unidirectional shape data 86 (step S307) is executed for each of the plurality of projectors 32 (step S303). As a result, unidirectional shape data 86 obtained when the pattern light L(S) is irradiated onto the component E from each of the plurality of projectors 32 is obtained. The plurality of unidirectional shape data 86 thus obtained are based on pattern images I(S) obtained while irradiating pattern light L(S) onto the component E from different projection directions D, respectively. Therefore, a pixel PX that corresponds to a shadow area As or a secondary reflection area Ar in one unidirectional shape data 86 may correspond to an area well irradiated with light in another unidirectional shape data 86. Therefore, the three-dimensional shape of the measurement object J is calculated by integrating the plurality of unidirectional shape data 86 by calculating the average of the shape-related values Q indicated by each of the plurality of unidirectional shape data 86 for each pixel PX. (Figure 10). Moreover, in the first integrated calculation process in FIG. 10, among the shape-related values Q indicated by the unidirectional shape data 86, 1 The three-dimensional shape of the measurement object J is calculated by weighted averaging, which is multiplied by a weighting coefficient W(P) that is less than 0 and is greater than or equal to 0 (steps S407 to S410). This makes it possible to appropriately calculate the three-dimensional shape of the measurement object J while suppressing the influence of the shadow area As or the secondary reflection area Ar.

In particular, in the first integrated calculation process shown in FIG. The three-dimensional shape of the object J is calculated (steps S407 to S410). In this way, the operation of multiplying by the weighting coefficient W(P) of 0 is an operation of excluding the shadow area As and the secondary reflection area Ar. This makes it possible to appropriately calculate the three-dimensional shape of the measurement object J while eliminating the effects of the shadow area As and the secondary reflection area Ar.

FIG. 12 is a flowchart showing an example of calculation for correcting the irradiation defect area calculated in FIG. 7. The flowchart in FIG. 12 is executed by the calculations of the main control unit 110 in parallel with the acquisition of the omnidirectional shape data in FIG. In step S501, the count value P of the projector 32 is reset to zero, and in step S502, the count value P is incremented. Further, in step S503, the count value R of the component mounting range Ref is reset to zero, and in step S504, the count value R is incremented. As a result, the shadow area As and the secondary reflection area Ar calculated for the component E in the R-th component mounting range Ref are specified in correspondence with the P-th projector 32.

In step S505, the shadow area As and secondary reflection area Ar specified in steps S502 and S504 are corrected based on the position of the substrate B detected in step S301 of omnidirectional shape data acquisition in FIG. In other words, it is assumed that the position of the substrate B detected in step S301 deviates from the ideal position of the substrate B based on the assumption in calculating the shadow area As and the secondary reflection area Ar by a positional deviation amount Δa. be done. In such a case, by correcting the shadow area As and the secondary reflection area Ar according to the positional deviation amount Δa of the substrate B, the shadow area As and the secondary reflection area Ar can be corrected according to the actual position of the substrate B carried into the three-dimensional measuring device 1. , the shadow area As and the secondary reflection area Ar can be accurately calculated.

In step S506, the shadow area As and secondary reflection area Ar specified in steps S502 and S504 are corrected based on the position of the component E actually mounted in the R-th component mounting range Ref. The details of this are as follows.

In the control using illumination defect area correction in FIG. 12, in step S306 of omnidirectional shape data acquisition in FIG. 9, a two-dimensional image of the component E is acquired in addition to the pattern image I(S). Specifically, a two-dimensional image of the component E is acquired by the imaging camera 31 capturing an image of the component E from above while the illumination 33 irradiates the component E within the imaging field of view V31 with illumination light. Furthermore, in step S306, the component position indicating the positional relationship between the component E actually mounted in the component mounting range Ref of the board B carried into the three-dimensional measuring device 1 and the component mounting range Ref is changed to It is calculated based on the dimensional image and stored in the storage unit 150.

On the other hand, the positional relationship between the component mounting range Ref indicated by the component position calculated in step S306 and the component E is different from the ideal positional relationship assumed in calculating the shadow area As and the secondary reflection area Ar. It is assumed that there is a deviation by a positional deviation amount Δb. In such a case, by correcting the shadow area As and the secondary reflection area Ar according to the positional deviation amount Δb of the component E with respect to the component mounting range Ref, the component of the board B carried into the three-dimensional measuring device 1 can be adjusted. The shadow area As and the secondary reflection area Ar can be accurately calculated according to the actual position of the component E mounted in the mounting range Ref.

In step S507, it is determined whether the count value R of the component mounting ranges Ref has reached the number Rmax of component mounting ranges Ref provided on the board B, that is, whether or not the count value R of the component mounting ranges Ref provided on the board B is determined in steps S505 to S506. It is confirmed whether the correction has been performed. If there is a component mounting range Ref for which the corrections in steps S505 to S506 have not been performed (“NO” in step S507), the process returns to step S504. In this way, steps S505 and S506 are repeated until the corrections of steps S505 and S506 are executed for all component mounting ranges Ref (“YES” in step S507). As a result, the illumination failure area (shadow area As/secondary reflection area Ar) for each component mounting range Ref that occurs when the pattern light L(S) is projected from the P-th projector 32 is corrected.

In step S508, it is confirmed whether the count value P of the projectors 32 has reached the number Pmax (=4) of the projectors 32, that is, whether steps S503 to S507 have been executed for all projectors 32. If there is a projector 32 that has not executed steps S503 to S507 (“NO” in step S508), the process returns to step S502. In this way, steps S503 to S507 are repeated until steps S503 to S507 are executed for all projectors 32 (“YES” in step S508). As a result, when the pattern light L(S) is projected from each projector 32, the illumination defect area (shadow area As/secondary reflection area Ar) that occurs due to the component E in each component mounting range Ref is corrected. Ru.

In the first data integration process in FIG. 10, steps S407 to S411 are executed based on the thus corrected shadow area As and secondary reflection area Ar. As a result, the integrated shape data 87 is generated based on the shadow area As and the secondary reflection area Ar that are accurately set for the measurement object J (board B and component E) actually carried into the three-dimensional measuring device 1. It can be calculated.

In the example of FIG. 12, the area calculation unit 112 further calculates the shadow area based on the result of recognizing the position of the substrate B carried into the three-dimensional measuring device 1 and supported by the conveyor 2 (object support unit) in step S301. As and the secondary reflection area Ar are calculated (step S505). With this configuration, the shadow area As and the secondary reflection area Ar can be accurately calculated according to the actual position of the substrate B carried into the three-dimensional measuring device 1.

In particular, the position of the substrate B supported by the transport conveyor 2 is recognized based on the result of imaging the fiducial mark attached to the substrate B by the imaging camera 31 (step S301). With this configuration, the shadow area As and the secondary reflection area Ar can be accurately calculated according to the position of the substrate B carried into the three-dimensional measurement device 1.

Furthermore, the area calculation unit 112 calculates a shadow area As and a secondary reflection area Ar based on the result of detecting the position of the component E mounted in the component mounting range Ref in step S306 (step S506). With this configuration, the shadow area As and the secondary reflection area Ar can be accurately calculated according to the actual position of the component E mounted in the component mounting range Ref.

Furthermore, the position of the component E mounted in the component mounting range Ref is detected based on a two-dimensional image of the component E (step S306). With this configuration, the shadow area As and the secondary reflection area Ar can be accurately calculated according to the actual position of the component E mounted in the component mounting range Ref.

13 is a flowchart showing a second example of three-dimensional measurement, FIG. 14 is a flowchart showing a second data integration process executed in three-dimensional measurement in FIG. 13, and FIG. 15 is a flowchart showing a second data integration process executed in three-dimensional measurement in FIG. FIG. 3 is a diagram schematically showing the contents of an operation to be executed. Here, it is assumed that a plurality of substrates B are sequentially loaded into the three-dimensional measuring device 1 and three-dimensional measurement is performed on each substrate B.

As shown in FIG. 13, in the second example of three-dimensional measurement, omnidirectional shape data acquisition (step S601) and first data integration processing (step S602) are performed similarly to the first example. Furthermore, in the second example, a second data integration process (FIG. 14) is executed in step S603. Note that the execution order of the first data integration process and the second data integration process is not limited to the example in FIG. 13, and may be reversed to the example in FIG. 13.

As shown in FIG. 14, the second data integration process is different from the first data integration process in that instead of steps S405 to S411 of the first data integration process, step S414 is a simple average (all weighting coefficients are 1). This is the point at which the calculation of the average) is performed. That is, in step S414 of the second data integration process, a simple average of the shape-related values Q of the N-th pixel PX obtained for each of the four projectors 32 is calculated. In this way, the integrated value H is calculated not by a weighted average but by a simple average, and the integrated shape data 87 is obtained.

When the three-dimensional measurement is completed for one board B (steps S601 to S603), it is determined whether steps S601 to S603 have been performed for a predetermined number (one or more) of boards B (step S604). . If the measurement has not been completed (“NO” in step S604), it is determined in step S605 whether to end the measurement. If the measurement is to be terminated ("YES" in step S605), the three-dimensional measurement in FIG. return.

In step S604, if it is determined that steps S601 to S603 have been executed for a predetermined number of substrates B (YES), the integrated shape data 87 (three-dimensional shape) acquired in the first data integration process in step S602 , the difference from the integrated shape data 87 (three-dimensional shape) acquired in the second data integration process in step S603 is calculated. For example, regarding the same component mounting range Ref of the same board B, the integrated shape data 87 (three-dimensional shape) obtained in the first data integration process and the integrated shape data 87 (three-dimensional shape) obtained in the second data integration process The average value of the square of the difference from the original shape (that is, the mean square error) is calculated. In addition, when one board B has a plurality of component mounting ranges Ref, the mean square error may be calculated for each component mounting range Ref, or the mean square error may be calculated for one representative component mounting range Ref. may be calculated. Further, the mean square error is calculated for each of the predetermined number of substrates B. The average value, median value, or maximum value of each mean square error calculated in this way is calculated as a difference. Note that the specific method for calculating the difference is not limited to this example, and any calculation method that can evaluate the difference between two pieces of data can be adopted.

In step S607, it is determined whether the first data integration process is necessary based on the difference. Specifically, if the difference is greater than or equal to a predetermined threshold, it is determined that the first data integration process is necessary (YES), and if the difference is less than the threshold, the first data integration process is determined to be necessary. It is determined that it is unnecessary. For example, as shown in each waveform (three-dimensional shape) in the column "Poor irradiation effect" in Figure 15, when there is an influence of poor irradiation, a peak occurs in the three-dimensional shape obtained in the second data integration process. Noise does not appear in the three-dimensional shape obtained in the first data integration process. This indicates that the first data integration process is necessary to eliminate the effects of poor irradiation. on the other hand,
As shown in each waveform (three-dimensional shape) in the column "No effect of irradiation failure" in Figure 15, if there is no influence of irradiation failure, the three-dimensional shape obtained in the first data integration process and the second data integration process There is no significant difference between the three-dimensional shape obtained through processing. This indicates that the first data integration process is unnecessary.

If it is determined that the first data integration process is necessary ("YES" in step S607), every time one substrate B is carried into the three-dimensional measuring device 1, omnidirectional shape data is acquired (step S608). 1 data integration processing (step S609) and determination of end of measurement (step S610) are executed. In this way, the integrated shape data 87 is calculated not by the second data integration process but by the first data integration process (step S609). On the other hand, if it is determined that the first data integration process is unnecessary ("NO" in step S607), omnidirectional shape data is acquired every time one substrate B is carried into the three-dimensional measuring device 1 (step S611). , a second data integration process (step S612), and a determination of end of measurement (step S613) are executed. In this way, the integrated shape data 87 is calculated not by the first data integration process but by the second data integration process (step S612).

In the example of FIG. 13, the shape calculation unit 114 executes both the first data integration process (step S602) and the second data integration process (step S603) on the same measurement object J. In this second data integration process, the shape calculation unit 114 calculates the average (simple average) of the shape-related values Q indicated by the plurality of unidirectional shape data 86 without weighting according to the shadow area As and the secondary reflection area Ar. By calculating for each pixel PX, the plurality of unidirectional shape data 86 are integrated, and the three-dimensional shape (integrated shape data 87) of the measurement target object J is calculated. Furthermore, the shape calculation unit 114 evaluates the difference between the integrated shape data 87 (three-dimensional shape) of the measurement object J calculated in each of the first data integration process (step S602) and the second data integration process (step S603). The evaluation process (step S606) is performed on a predetermined number of measurement objects J from which pattern images I(S) are to be obtained by the three-dimensional measuring device 1. Based on this evaluation process, a necessity determination is performed to determine whether the first data integration process is necessary (step S607). Then, in the necessity determination in step S607, after it is determined that the first data integration process is unnecessary, the shape calculation unit 114 performs the measurement target J by the second data integration process without executing the first data integration process. The integrated shape data 87 (three-dimensional shape) is calculated (steps S611 to S613). In such a configuration, if the shadow area As and the secondary reflection area Ar are not a problem, the integrated shape data 87 of the measurement object J can be accurately calculated by the second data integration process, which is simpler than the first data integration process. I can do it.

In this example, the "predetermined number of sheets" determined in step S604 may be configured to be set by the user. In this case, the user performs an operation to set a predetermined number of sheets on the UI 200, and the UI 200 receives the predetermined number of sheets input by the user and sets it to be used in the three-dimensional measurement shown in FIG. With this configuration, the user can appropriately adjust the period (steps S601 to S604) during which both the first data integration process (step S602) and the second data integration process (step S603) are executed.

In the above embodiment, the three-dimensional measuring device 1 corresponds to an example of the "three-dimensional measuring device" of the present invention, and the control device 100 corresponds to an example of the "three-dimensional measurement computing device" of the present invention. , the part model acquisition section 111 corresponds to an example of the "reference model acquisition section" of the present invention, the area calculation section 112 corresponds to an example of the "area calculation section" of the present invention, and the shape calculation section 114 corresponds to an example of the "reference model acquisition section" of the present invention. The conveyor 2 corresponds to an example of the "object support section" of the present invention, the imaging camera 31 corresponds to an example of the "imaging section" of the present invention, and the projector 32 corresponds to an example of the "object support section" of the present invention. This corresponds to an example of the "projector" of the invention, the plurality of projectors 32 corresponds to an example of the "pattern irradiation unit" of the invention, the component model 83 corresponds to an example of the "reference model" of the invention, and the projection direction information 84 corresponds to an example of "irradiation direction information" of the present invention, unidirectional shape data 86 corresponds to an example of "unidirectional shape data" of the present invention, and server computer 91 corresponds to an example of "recording medium" of the present invention. The three-dimensional measurement program 92 corresponds to an example of the "three-dimensional measurement program" of the present invention, the UI 200 corresponds to an example of the "setting operation section" of the present invention, and the secondary reflection area Ar corresponds to an example of the "three-dimensional measurement program" of the present invention. The shadow area As corresponds to an example of the "secondary reflection area" and the "poor irradiation area" of the present invention, the shadow area As corresponds to an example of the "shadow area" and the "poor irradiation area" of the present invention, and the substrate B corresponds to an example of the "substrate B" of the present invention. ”, the part E corresponds to an example of the “component” of the present invention, the pattern image I(S) corresponds to an example of the “pattern image” of the present invention, and the measurement object J corresponds to an example of the “pattern image” of the present invention. This corresponds to an example of a "measurement object," the pattern light L(S) corresponds to an example of "light" of the present invention, the pixel PX corresponds to an example of a "pixel" of the present invention, and the shape-related value Q corresponds to an example of a "pixel" of the present invention. This corresponds to an example of the "shape-related value" of the present invention, the measured value Qm corresponds to an example of the "calculated value" of the present invention, the reliability Qr corresponds to an example of the "reliability" of the present invention, and the component mounting The range Ref corresponds to an example of the "component mounting range" of the present invention, step S606 corresponds to an example of the "evaluation process" of the present invention, step S607 corresponds to an example of the "necessity determination" of the present invention, The first data integration process in FIG. 10 corresponds to an example of the "first integration calculation process" of the present invention, and the second data integration process in FIG. 14 corresponds to an example of the "second integration calculation process" of the invention.

Note that the present invention is not limited to the embodiments described above, and various changes can be made to what has been described above without departing from the spirit thereof. For example, it is not necessary to calculate both the secondary reflection area Ar and the shadow area As as the illumination defect area. For example, if the influence of one of the secondary reflection area Ar and the shadow area As is large and the influence of the other is slight, only one of the secondary reflection areas Ar and the shadow area As may be calculated as the poor irradiation area in step S107 of FIG. .

Furthermore, in the illumination defect area correction shown in FIG. 12, it is possible to perform only one of the correction based on the position of the substrate B (step S505) and the correction based on the position of the component E (step S506) without performing the other. good.

Furthermore, the specific contents of the shape-related value Q may be changed as appropriate. For example, the measured value Qm may be directly calculated as the shape-related value Q without calculating the reliability Qr.

Furthermore, in step S408, the weighting coefficient W(P) to be multiplied corresponding to the defective irradiation area does not need to be 0, but may be a value of 0 or more and less than 1.

Furthermore, the functional units 111, 112, 113, and 114 configured in the main control unit 110 by executing the three-dimensional measurement program 92 do not necessarily need to be configured in the control device 100 included in the three-dimensional measurement device 1. Therefore, each of the functional units 111, 112, 113, and 114 may be configured in a processor included in a computer provided separately from the three-dimensional measuring device 1.

Furthermore, in the second data integration process in FIG. 1, in step S414, a median value (representative value) may be calculated instead of calculating the average value (representative value) by simple average.

1...Three-dimensional measuring device 100...Control device (computing device for three-dimensional measurement)
111...Parts model acquisition unit (reference model acquisition unit)
112...Area calculation unit (area calculation unit)
114... Shape calculation section 2... Conveyor (object support section)
31...Imaging camera (imaging section)
32...Projector (pattern irradiation section)
83...Parts model (standard model)
84...Projection direction information (irradiation direction information)
86...Unidirectional shape data 91...Server computer (recording medium)
92...Three-dimensional measurement program 200...UI (setting operation section)
Ar…Secondary reflection area (poor irradiation area)
As…Shadow area (poor irradiation area)
B... Board E... Component I(S)... Pattern image J... Measurement object L(S)... Pattern light (light)
PX...pixel Q...shape related value Qm...measured value (calculated value)
Qr…Reliability Ref…Component mounting range

Claims (17)

1. an object support section that supports a measurement object having a substrate and components mounted on the substrate; a pattern irradiation section that irradiates the measurement object supported by the object support section with a predetermined pattern of light; the measurement target based on the pattern image acquired by a three-dimensional measuring device including an imaging unit that acquires a two-dimensional pattern image by imaging the light irradiated onto the measurement target from the pattern irradiation unit; A calculation device for three-dimensional measurement that executes calculations for measuring the three-dimensional shape of
   a reference model acquisition unit that obtains a reference model indicating a component mounting range in which the component is mounted on the board and an outer shape of the component mounted in the component mounting range;
   A shadow area in which a shadow of the component is created on the board because light irradiated from the imaging unit to the component is blocked by the component, and light irradiated from the imaging unit and reflected by the component enters the board. an area calculating unit that calculates a poor irradiation area in at least one of the secondary reflection areas to be irradiated based on the reference model and irradiation direction information indicating a direction in which light is irradiated to the component by the three-dimensional measuring device; A calculation device for three-dimensional measurement.
2. The reference model acquisition unit includes inspection data indicating a standard for inspecting the suitability of the positional relationship between the component mounted in the component mounting range and the component mounting range, and a CAD (Computer-Aided Model Acquisition Unit) indicating the configuration of the board. 2. The three-dimensional measurement arithmetic device according to claim 1, wherein the reference model is created from at least one of (Design) data.
3. The area calculation unit calculates the ill-irradiation area further based on a result of recognizing the position of the substrate carried into the three-dimensional measurement device and supported by the object support unit. Computing device for three-dimensional measurement.
4. The three-dimensional measurement computing device according to claim 3, wherein the position of the substrate supported by the object support section is recognized based on a result of imaging a fiducial mark attached to the substrate by the imaging section. .
5. The three-dimensional measurement calculation according to any one of claims 1 to 4, wherein the area calculation unit calculates the ill-irradiation area further based on a result of detecting the position of the component mounted in the component mounting range. Device.
6. The three-dimensional measurement arithmetic device according to claim 5, wherein the position of the component mounted in the component mounting range is detected based on a result of imaging the component.
7. The pattern irradiation unit includes a plurality of projectors that irradiate the component with light from mutually different directions,
   The irradiation direction information indicates, for each of the plurality of projectors, a direction in which light is irradiated from the projector to the component;
   The three-dimensional measurement arithmetic device according to any one of claims 1 to 6, wherein the area calculation unit calculates the ill-irradiation area for each of the plurality of projectors.
8. Unidirectional shape data indicating a shape-related value, which is a value related to a three-dimensional shape, for each pixel is calculated based on the pattern image acquired by capturing the light irradiated onto the substrate from the projector by the imaging unit. further comprising a shape calculation unit that calculates a plurality of unidirectional shape data by performing calculations for each of the plurality of projectors,
   The shape calculation unit integrates the plurality of unidirectional shape data by calculating the average of the shape-related values indicated by each of the plurality of unidirectional shape data for each pixel, and calculates the three-dimensional shape of the measurement target. Execute a first integrated calculation process to calculate the shape,
   In the first integrated calculation process, among the shape-related values indicated by the unidirectional shape data, the shape-related values of the pixels included in the poor irradiation area are multiplied by a weighting coefficient of less than 1 and greater than or equal to 0. The three-dimensional measurement computing device according to claim 7, wherein the three-dimensional shape of the measurement object is calculated by weighted averaging.
9. 2. A three-dimensional shape of the object to be measured is calculated by a weighted average in which the shape-related values of the pixels included in the poor irradiation area are multiplied by a weighting coefficient of 0 in the first integrated calculation process. 8. The three-dimensional measurement arithmetic device according to 8.
10. The three-dimensional measurement arithmetic device according to claim 8 or 9, wherein the shape-related value is given as a product of a calculated value of the three-dimensional shape calculated based on the pattern image and a reliability of the calculated value.
11. The arithmetic device for three-dimensional measurement according to claim 8 or 9, wherein the shape-related value is a calculated value of a three-dimensional shape calculated based on the pattern image.
12. The shape calculation unit calculates, for each pixel, a representative value of the shape-related values indicated by the plurality of unidirectional shape data without performing weighting according to the poor irradiation area. A second integrated calculation process for calculating the three-dimensional shape of the measurement target object and the first integrated calculation process are performed on the same measurement target object, and the first integration process is performed on the same measurement target object. An evaluation process for evaluating the difference in the three-dimensional shapes of the measurement objects calculated by each of the arithmetic processing and the second integrated arithmetic processing is performed on a predetermined number of measurement objects from which the pattern images are acquired by the three-dimensional measuring device. executing a necessity determination for determining whether or not the first integrated calculation process is necessary by executing it on the object;
    After it is determined that the first integrated calculation process is unnecessary in the necessity determination, the shape calculation unit calculates the three-dimensional shape of the measurement target by the second integrated calculation process without executing the first integrated calculation process. The three-dimensional measurement arithmetic device according to any one of claims 8 to 11, which calculates an original shape.
13. The three-dimensional measurement arithmetic device according to claim 12, further comprising a setting operation unit that accepts an operation to set the predetermined number.
14. A three-dimensional measurement program that causes a computer to function as the three-dimensional measurement arithmetic device according to any one of claims 1 to 13.
15. A recording medium on which the three-dimensional measurement program according to claim 14 is recorded so as to be readable by a computer.
16. an object support section that supports a measurement object having a board and components mounted on the board;
    a pattern irradiation unit that irradiates the measurement target object supported by the target object support unit with a predetermined pattern of light;
    an imaging unit that acquires a two-dimensional pattern image by imaging the light irradiated onto the measurement target from the pattern irradiation unit;
    A three-dimensional measurement device comprising: a three-dimensional measurement calculation device according to any one of claims 1 to 13, which executes calculation for measuring a three-dimensional shape of the object to be measured based on the pattern image.
17. an object support section that supports a measurement object having a substrate and components mounted on the substrate; a pattern irradiation section that irradiates the measurement object supported by the object support section with a predetermined pattern of light; the measurement target based on the pattern image acquired by a three-dimensional measuring device including an imaging unit that acquires a two-dimensional pattern image by imaging the light irradiated onto the measurement target from the pattern irradiation unit; A three-dimensional measurement calculation method for performing calculations for measuring a three-dimensional shape of
    obtaining a reference model indicating a component mounting range in which the component is mounted on the board and an external shape of the component mounted in the component mounting range;
    A shadow area in which a shadow of the component is created on the board because light irradiated from the imaging unit to the component is blocked by the component, and light irradiated from the imaging unit and reflected by the component enters the board. calculating a poor irradiation area in at least one of the secondary reflection areas to be irradiated based on irradiation direction information indicating a direction in which light is irradiated onto the component by the three-dimensional measuring device and the reference model. Computation method for three-dimensional measurement.
    
    

Images