KR20240060825A - Three-dimensional measurement calculation device, three-dimensional measurement program, recording medium, three-dimensional measurement device, and three-dimensional measurement calculation method - Google Patents

Abstract

A component mounting range (Ref) in which the component (E) is mounted on the board (B) and a component model (83) (reference model) representing the external appearance of the component (E) mounted in the component mounting range (Ref) are acquired. In addition, projection direction information 84 (irradiation direction information) indicating the projection direction D in which the pattern light L(S) is projected onto the part E in the three-dimensional measurement device 1, and the part model 83 Based on this, the shadow area As and the secondary reflection area Ar (poorly irradiated area) are calculated. In this way, in measuring the three-dimensional shape of the measurement object J based on the pattern image I(S) acquired by imaging the pattern light L(S) irradiated on the measurement object J, the pattern light It is possible to acquire the shadow area (As) and the secondary reflection area (Ar) where irradiation defects (L) occur.

Description

Three-dimensional measurement calculation device, three-dimensional measurement program, recording medium, three-dimensional measurement device, and three-dimensional measurement calculation method

This invention relates to a technique for measuring the three-dimensional shape of a measurement object based on a pattern image obtained by imaging a predetermined pattern of light irradiated on the measurement object.

Patent Document 1 and Patent Document 2 describe a three-dimensional measurement technology that measures the three-dimensional shape of a measurement object using the so-called phase shift method. In this three-dimensional measurement technology, the three-dimensional shape of the measurement object is measured based on an image acquired by using a camera to capture a predetermined pattern of light irradiated from a projector to the measurement object.

Additionally, Patent Document 2 proposes a technique for measuring a three-dimensional shape by suppressing the influence of secondary reflection that occurs when a tall object is present among the objects constituting the measurement object. In other words, if secondary reflection occurs, such as when light emitted from a projector and reflected from the side of a tall object is further reflected by another object, it becomes difficult to accurately measure the three-dimensional shape. Therefore, in Patent Document 2, the range in which light is irradiated from the projector is limited so that the light from the projector does not enter a tall object (causing object) that causes secondary reflection.

Japanese Patent Publication No. 2012-112952 Japanese Patent Publication No. 2016-130663

However, in order to limit the range of light irradiated from the projector as shown in Patent Document 2, it is necessary to control hardware such as the projector provided in the three-dimensional measurement device, so control is likely to become complicated. In addition to the above-mentioned secondary reflection, so-called occlusion can be cited as a light irradiation defect that affects the measurement of a three-dimensional shape. This occlusion is a phenomenon in which a shadow is generated where light from the projector does not reach by being obscured by an object constituting the measurement object. Patent Document 2 cannot respond to such occlusion.

In contrast, if the poor irradiation area in the measurement object where poor irradiation of light occurs can be known, the poor irradiation area can be referred to in calculating the three-dimensional shape based on the pattern image acquired by the three-dimensional measurement device. there is. As a result, calculations for calculating a three-dimensional shape can be performed while suppressing the influence of poor irradiation of light.

This invention was made in consideration of the above problem, and in measuring the three-dimensional shape of a measurement object based on a pattern image acquired by imaging a predetermined pattern of light irradiated to the measurement object, an area of poor irradiation where light irradiation defects occurs is determined. The purpose is to make acquisition possible.

The computing device for three-dimensional measurement according to the present invention includes an object support unit that supports a measurement object having a substrate and components mounted on the substrate, a pattern irradiation unit that irradiates light of a predetermined pattern to the measurement object supported on the object support, and a pattern irradiation unit. A three-dimensional measurement calculation that performs calculations for measuring the three-dimensional shape of a measurement object based on a pattern image acquired by a three-dimensional measurement device provided with an imaging unit that acquires a two-dimensional pattern image by imaging light irradiated to the measurement object from The device includes a reference model acquisition unit that acquires a reference model representing the component mounting range in which the component is mounted on a board and the external appearance of the component being mounted in the component mounting range, and the light emitted from the imaging unit to the component is obscured by the component. At least one of the shadow areas on the substrate where the shadow of the component occurs and the secondary reflection area where the light irradiated from the imaging unit and reflected from the component enters the substrate is an illumination area where light is irradiated to the component from a three-dimensional measuring device. It is provided with irradiation direction information indicating the direction and an area calculation unit that calculates the area based on a reference model.

The program for three-dimensional measurement according to the present invention causes the computer to function as the arithmetic device for three-dimensional measurement.

The recording medium according to the present invention records the three-dimensional measurement program in a manner that is readable by a computer.

The three-dimensional measurement device according to the present invention includes an object support unit that supports a measurement object having a substrate and components mounted on the substrate, a pattern irradiation unit that irradiates light of a predetermined pattern to the measurement object supported on the object support unit, and measurement from the pattern irradiation unit. It is provided with an imaging unit that acquires a two-dimensional pattern image by imaging light irradiated to the object, and the three-dimensional measurement arithmetic device that performs calculations for measuring the three-dimensional shape of the measurement object based on the pattern image.

The calculation method for three-dimensional measurement according to the present invention includes an object support unit that supports a measurement object having a substrate and components mounted on the substrate, a pattern irradiation unit that irradiates light of a predetermined pattern to the measurement object supported on the object support, and a pattern irradiation unit. A three-dimensional measurement calculation that performs calculations for measuring the three-dimensional shape of a measurement object based on a pattern image acquired by a three-dimensional measurement device provided with an imaging unit that acquires a two-dimensional pattern image by imaging light irradiated to the measurement object from As a method, the process includes obtaining a reference model representing the component mounting range in which the component is mounted on the board and the external shape of the component mounted in the component mounting range, and the light emitted from the imaging unit to the component is obscured by the component, thereby At least one poorly irradiated area among the shadow area where the shadow of the part occurs and the secondary reflection area where the light irradiated from the imaging unit and reflected from the part enters the substrate is a three-dimensional measurement device that indicates the direction in which light is irradiated to the part. Survey direction information and a calculation process based on a standard model are provided.

In the present invention (a calculation device for three-dimensional measurement, a program for three-dimensional measurement, a recording medium, a three-dimensional measurement device, and a calculation method for three-dimensional measurement) configured as described above, the component mounting range in which the component is mounted on the board and the component mounting range in the component mounting range are provided. A reference model representing the appearance is obtained. Then, the illumination defect area is calculated based on the reference model and irradiation direction information indicating the direction in which light is irradiated to the part in the three-dimensional measuring device. In this way, when measuring the three-dimensional shape of the measurement object based on a pattern image acquired by imaging a predetermined pattern of light irradiated to the measurement object, it is possible to acquire an irradiation defective area where light irradiation defects occur.

Here, the poorly irradiated area is a shadow area where the light irradiated from the imaging unit to the component is obscured by the component, causing a shadow of the component on the substrate, and the secondary area where the light irradiated from the imaging unit and reflected from the component is incident on the substrate. It is at least one side of the reflection area.

In addition, the reference model acquisition unit includes at least one of inspection data representing a standard for inspecting the adequacy of the positional relationship between the components mounted in the component mounting range and CAD (Computer-Aided Design) data representing the configuration of the board. A calculation device for three-dimensional measurement may be configured to produce a reference model from one piece of data. In this configuration, a reference model can be created using existing data such as board inspection data or CAD data, and the irradiation defective area can be calculated based on this reference model. Additionally, the information represented by the inspection data is not limited to the above standards, and may also include standards for inspecting the adequacy of the component itself and standards for inspecting the adequacy of the joint between the component and the board.

Additionally, the three-dimensional measurement calculation device may be configured so that the area calculation unit calculates the irradiation defective area further based on the results of recognizing the position of the substrate carried in the three-dimensional measurement device and supported on the object support portion. In this configuration, the irradiation defective area can be accurately calculated according to the actual position of the substrate loaded into the three-dimensional measuring device.

Additionally, the arithmetic device for three-dimensional measurement may be configured so that the position of the substrate supported on the object support portion is recognized based on the results of imaging the fiducial mark attached to the substrate by the imaging unit. In this configuration, the irradiation defective area can be accurately calculated according to the position of the substrate loaded into the three-dimensional measuring device.

Additionally, the area calculation unit may configure a three-dimensional measurement calculation device so that the area calculation unit calculates the irradiation defective area further based on the results of detecting the positions of the components mounted in the component mounting range. In this configuration, the irradiation defective area can be accurately calculated according to the actual position of the component mounted in the component mounting range.

Additionally, the three-dimensional measurement calculation device may be configured so that the positions of components mounted in the component mounting range are detected based on the results of imaging the components. In this configuration, the irradiation defective area can be accurately calculated according to the actual position of the component mounted in the component mounting range.

In addition, the pattern irradiation unit has a plurality of projectors that irradiate light to the part from different directions, the irradiation direction information indicates the direction in which light is irradiated from the projector to the part for each of the plurality of projectors, and the area calculation unit has a plurality of projectors. A calculation device for three-dimensional measurement may be configured to calculate the irradiation defective area for each of . In this configuration, it is possible to acquire illumination defect areas that occur when light is irradiated to a component from each of a plurality of projectors.

In addition, based on the pattern image acquired by imaging the light irradiated from the projector to the substrate by the imaging unit, a calculation for calculating unidirectional shape data representing a shape-related value, which is a value related to a three-dimensional shape, for each pixel is performed on each of the plurality of projectors. The shape calculation unit further includes a shape calculation unit that calculates a plurality of unidirectional shape data by executing for , and the shape calculation unit integrates the plurality of unidirectional shape data by calculating the average of the shape-related values represented by each of the plurality of unidirectional shape data for each pixel, A first integrated calculation process for calculating the three-dimensional shape of the measurement object is performed, and in the first integrated calculation process, among the shape-related values indicated by the unidirectional shape data, the shape-related value of the pixel included in the poor irradiation area is less than 1. The arithmetic device for three-dimensional measurement may be configured so that the three-dimensional shape of the measurement object is calculated by a weighted average multiplied by a weight coefficient of 0 or more.

In this configuration, unidirectional shape data indicating a value related to the three-dimensional shape (shape-related value) for each pixel is calculated based on a pattern image acquired by imaging the light irradiated from the projector to the substrate by an imaging unit. In other words, unidirectional shape data is data about a three-dimensional shape calculated based on a pattern image captured by light irradiated to a part from a single projector. The calculation for calculating this unidirectional shape data is performed for each of the plurality of projectors. In this way, unidirectional shape data when light is irradiated to the part from each of the plurality of projectors is acquired. The plurality of unidirectional shape data acquired in this way are based on pattern images acquired while irradiating light to the part from different directions. Therefore, a pixel corresponding to a poorly illuminated area in one unidirectional shape data may correspond to a well irradiated area in other unidirectional shape data. Therefore, by calculating the weighted average of the shape-related values represented by each of the plurality of unidirectional shape data for each pixel, the plurality of unidirectional shape data can be integrated to calculate the three-dimensional shape of the measurement object. Furthermore, in the first integrated calculation process, among the shape-related values indicated by the unidirectional shape data, the shape-related values of the pixels included in the poor irradiation area are calculated by a weighted average that is less than 1 and multiplied by a weight coefficient of 0 or more to obtain a three-dimensional image of the measurement object. The shape is calculated. This makes it possible to appropriately calculate the three-dimensional shape of the measurement object while suppressing the influence of the poorly irradiated area.

In addition, in the first integrated calculation process, the three-dimensional measurement calculation device may be configured so that the three-dimensional shape of the measurement object is calculated by a weighted average that multiplies the shape-related values of the pixels included in the poor irradiation area by a weight coefficient of 0. good night. In this way, the operation of multiplying the weight coefficient of 0 is an operation of excluding the irradiation defective area. This makes it possible to appropriately calculate the three-dimensional shape of the measurement object while excluding the influence of poor irradiation areas.

Additionally, the specific contents of the shape-related values are assumed in various ways. For example, the shape-related value may be given as the product of the calculated value of the three-dimensional shape calculated based on the pattern image and the reliability of the calculated value, and the shape-related value may be the calculated value of the three-dimensional shape calculated based on the pattern image. It's okay to continue.

In addition, the shape calculation unit integrates the plurality of unidirectional shape data by calculating a representative value of the shape-related value represented by the plurality of unidirectional shape data for each pixel without performing weighting according to the irradiation defect area, and calculates the three-dimensional shape of the measurement object. 2. An evaluation process that executes both the integrated calculation process and the first integrated calculation process on the same measurement object, and evaluates the difference in the three-dimensional shape of the measurement object calculated in each of the first integrated calculation process and the second integrated calculation process. , by executing on a predetermined number of measurement objects that are the target of pattern image acquisition in a three-dimensional measurement device, a necessity judgment is performed to determine the necessity of the first integrated calculation process, and the first integrated calculation process is performed in the necessity judgment. After it is determined that is unnecessary, the shape calculation unit may configure the three-dimensional measurement calculation device so that the three-dimensional shape of the measurement object is calculated by the second integration calculation process without performing the first integration calculation process. In this configuration, when poor irradiation areas are not a problem, the three-dimensional shape of the measurement object can be accurately calculated by the second integrated calculation process, which is simpler than the first integrated calculation process. Here, the representative value of the shape-related value represented by the plurality of unidirectional shape data, which is not weighted according to the poor irradiation area, includes the average value by simple average or the median value, etc.

Additionally, the arithmetic device for three-dimensional measurement may be configured to further include a setting operation unit that accepts an operation for setting a predetermined number. In this configuration, the user can appropriately adjust the period during which both the first integrated calculation process and the second integrated calculation process are executed.

According to the present invention, when measuring the three-dimensional shape of a measurement object based on a pattern image acquired by imaging a predetermined pattern of light irradiated to the measurement object, it becomes possible to acquire an irradiation defective area where light irradiation defects occur. .

1 is a block diagram schematically illustrating a three-dimensional measuring device according to the present invention.
FIG. 2 is a plan view schematically showing the relationship between the projector included in the three-dimensional measurement device of FIG. 1 and the imaging field of view.
FIG. 3 is a block diagram showing details of a control device included in the three-dimensional measuring device of FIG. 1.
Figure 4 is a diagram schematically showing the contents of a part model produced by the part model acquisition unit.
FIG. 5A is a diagram showing an example of board configuration data used for manufacturing a component model in table format.
Figure 5b is a diagram showing an example of part data used to produce a part model in table format.
Figure 5c is a diagram schematically showing an example of a manufactured part model.
Figure 6 is a diagram schematically showing the contents of the illumination defect area calculated by the area calculation unit.
Fig. 7 is a flowchart showing an example of calculation for calculating the irradiation defective area.
Figure 8 is a flow chart showing a first example of three-dimensional measurement.
FIG. 9 is a flow chart showing omnidirectional shape data acquisition performed in the three-dimensional measurement of FIG. 8.
FIG. 10 is a flowchart showing a first data integration process performed in the three-dimensional measurement of FIG. 8.
FIG. 11 is a diagram schematically showing operations performed in the first data integration process of FIG. 10.
FIG. 12 is a flowchart showing an example of calculation for correcting the poorly irradiated area calculated in FIG. 7.
Fig. 13 is a flow chart showing a second example of three-dimensional measurement.
FIG. 14 is a flowchart showing a second data integration process performed in the three-dimensional measurement of FIG. 13.
FIG. 15 is a diagram schematically showing the contents of calculations performed in the three-dimensional measurement of FIG. 13.

1 is a block diagram schematically illustrating a three-dimensional measuring device according to the present invention. In the same drawing and the following drawings, the horizontal X direction, the horizontal Y direction perpendicular to the X direction, and the vertical Z direction are shown as appropriate. The three-dimensional measurement device 1 in FIG. 1 controls the conveyor 2, the measurement head 3, and the drive mechanism 4 by the control device 100 to determine the three-dimensional shape (external shape) of the measurement object J. Measure. The measurement object J is composed of a substrate B (printed substrate) and components E mounted on the substrate B. This measurement object J is produced by a surface mounting machine that mounts the component E on the board B, and is loaded into the three-dimensional measurement device 1.

The transport conveyor 2 transports the measurement object J along a predetermined transport path. Specifically, the conveyor 2 transports the measurement object J before measurement to the measurement position in the three-dimensional measurement device 1 and holds the measurement object J at the measurement position so that the substrate B is horizontal. . Additionally, when the measurement of the three-dimensional shape of the measurement object J at the measurement position is completed, the conveyor 2 carries the measured measurement object J out of the three-dimensional measurement device 1.

The measurement head 3 has an imaging camera 31 that captures images within the imaging field of view V31 from above, and causes the measurement object J brought into the measurement position to enter the imaging field of view V31. Taken by. The imaging camera 31 has an individual imaging element 311 that detects reflected light from the measurement object J, and captures an image of the measurement object J using the individual imaging element 311.

Additionally, the measurement head 3 has a projector 32 that projects striped pattern light L(S) whose light intensity distribution changes in a sinusoidal shape onto the imaging field of view V31. The projector 32 has a light source such as an LED (Light Emitting Diode) and a digital micromirror device that reflects light from the light source toward the imaging field of view V31. This projector 32 can project a plurality of types of patterned light L(S) with different phases onto the imaging field of view V31 by adjusting the angle of each micromirror of the digital micromirror device. That is, the measurement head 3 captures images with the imaging camera 31 while changing the phase of the pattern light L(S) projected from the projector 32, thereby changing the imaging field of view V31 by the phase shift method. The three-dimensional shape of the measurement object J inside can be measured.

FIG. 2 is a plan view schematically showing the relationship between the projector included in the three-dimensional measurement device of FIG. 1 and the imaging field of view. As shown in FIG. 2, the measurement head 3 has a plurality (four in this example) of projectors 32 (in FIG. 1, two projectors 32 are typically used to simplify the illustration). appears). Each projector 32 projects pattern light L(S) diagonally upward with respect to the imaging field of view V31 of the imaging camera 31. In plan view, the plurality of projectors 32 are arranged to surround the imaging camera 31 and are arranged at equal pitches in a columnar shape with the vertical direction Z as the center. Accordingly, the plurality of projectors 32 project the pattern light L(S) onto the imaging field of view V31 from different projection directions D. Additionally, the number of projectors 32 included in the measurement head 3 is not limited to four in the example of FIG. 2 .

Additionally, the measurement head 3 has an illumination 33 (FIG. 1) that irradiates light into the imaging field of view V31. The projector 32 projects the pattern light L(S) onto the imaging field of view V31 when measuring a three-dimensional shape, while the illumination 33 projects a two-dimensional image with the imaging camera 31. , illumination light is irradiated into the imaging field of view (V31).

The drive mechanism 4 supports the measurement head 3 and drives the measurement head 3 in the X, Y, and Z directions by a motor. By driving this drive mechanism 4, the measurement head 3 can move above the measurement target portion of the measurement object J, capturing the measurement target portion within the imaging field of view V31, and capturing the measurement target portion within the imaging field of view (V31). The three-dimensional shape of the measurement target location within V31) can be measured. In particular, the three-dimensional measuring device 1 contributes to the inspection of lifting of the component E from the substrate B, for example, lifting of the terminal of a package such as QFP (Quad Flat Package), etc. from the substrate B, While bringing the part E into the imaging field of view V31, the three-dimensional shape of the part E and its surroundings (measurement target location) can be measured.

The control device 100 has a main control unit 110, which is a processor composed of a CPU (Central Processing Unit) and memory, and the main control unit 110 integrates control of each part of the device to measure a three-dimensional shape. In addition, the control device 100 has a UI (User Interface) 200 consisting of input and output devices such as a display, keyboard, and mouse, and the user inputs commands to the control device 100 through the UI 200, or The measurement results by the control device 100 can be confirmed. Additionally, the control device 100 has a projection control unit 120, an imaging control unit 130, and a drive control unit 140. The projection control unit 120 controls projection of the pattern light L(S) by the projector 32. The imaging control unit 130 controls imaging of the imaging field of view V31 by the imaging camera 31 and irradiation of light from the illumination 33 to the imaging field of view V31. The drive control unit 140 controls the driving of the measurement head 3 by the drive mechanism 4.

When the conveyor 2 brings the measurement object J to the measurement position, the main control unit 110 controls the drive mechanism 4 by the drive control unit 140 to change the measurement target location of the measurement object J. Move the measurement head (3) upward. As a result, the measurement target point enters the imaging field of view V31 of the imaging camera 31. Subsequently, the main control unit 110 projects the pattern light L(S) from the projector 32 to the imaging field of view V31 and sends the pattern light L(S) projected to the imaging field of view V31 to the imaging camera. Image is captured by (31) (pattern imaging operation). Specifically, the control device 100 has a storage unit 150 and reads the projection pattern T(S) stored in the storage unit 150. Then, the main control unit 110 controls the projection control unit 120 based on the projection pattern T(S) read from the storage unit 150, thereby controlling each micro mirror of the digital micro mirror device of the projector 32. The angle is adjusted according to the projection pattern (T(S)). In this way, the pattern light L(S) having the projection pattern T(S) is projected onto the imaging field of view V31. In addition, the main control unit 110 controls the imaging control unit 130 to capture the pattern light L(S) projected on the imaging field of view V31 by the imaging camera 31 to produce a pattern image I(S). ) to acquire. This pattern image (I) is stored in the storage unit 150. Additionally, four types of projection patterns (T(S)) each having a phase difference of 90 degrees are stored in the storage unit 150, and the pattern imaging operation is performed four times while changing the projection pattern (T(S)) ( S=1, 2, 3, 4). As a result, four types of pattern images I(S) are acquired, each of which is obtained by capturing pattern light L(S) with a different phase of 90 degrees. The main control unit 110 determines the height of the imaging field of view V31 for each pixel of the imaging camera 31 using the phase shift method from the four types of pattern images I(S) acquired in this way. In addition, the variation of the projection pattern T(S) is not limited to the example here, and for example, eight types of projection patterns T(S) each having a phase difference of 45 degrees may be used, and eight types of projection patterns T(S) each having a phase difference of 120 degrees may be used. Three types of projection patterns (T(S)) may be used.

FIG. 3 is a block diagram showing details of a control device included in the three-dimensional measuring device of FIG. 1. The control device 100 stores the three-dimensional measurement program 92 downloaded from the external server computer 91 in the memory unit 150. Additionally, the mode of providing the three-dimensional measurement program 92 is not limited to downloading from the outside, and the three-dimensional measurement program 92 may be provided as recorded on a DVD (Digital Versatile Disc) or USB (Universal Serial Bus). When the main control unit 110 executes the three-dimensional measurement program 92, the part model acquisition unit 111, area calculation unit 112, pattern image acquisition unit 113, and shape calculation unit 114 are connected to the main control unit 110. ) is composed of These functional portions 111, 112, 113, and 114 are used to illuminate defective areas caused by components E on the substrate B when the pattern light L(S) is projected onto the measurement object J. It has a corresponding function. These details are as follows.

Figure 4 is a diagram schematically showing the contents of a part model produced by the part model acquisition unit, Figure 5a is a diagram showing an example of board configuration data used for manufacturing the part model in a table format, and Figure 5b is a diagram showing an example of board configuration data used for manufacturing the part model. This is a diagram showing an example of part data used to manufacture a part model in table format, and FIG. 5C is a diagram schematically showing an example of the manufactured part model. The component model acquisition unit 111 uses the substrate configuration data 81 representing the configuration of the substrate B and the component data 82 representing the external shape of the component E scheduled to be mounted on the substrate B, A component model 83 representing the three-dimensional existence range of the component E mounted in (B) is produced. Additionally, the board configuration data 81 and the parts data 82 are stored in the storage unit 150 in advance.

The board configuration data 81 (FIG. 5A) represents the component mounting range (Ref(R)) formed on the board (B) and the type of component (E) to be mounted in the component mounting range (Ref(R)). Indicates the component mounting range (Ref(1), Ref(2), Ref(3), …) (R=1, 2, 3, …). In the board configuration data 81, the component mounting range Ref(R) is the position (X coordinate) of the component mounting range Ref(R) in the X direction, and the component mounting range in the Y direction ( It is specified by the position (Y coordinate) of Ref(R)). Additionally, the component mounting range Ref(R) is defined by, for example, lands formed on the substrate B, and is an extended range. In contrast, the position (x,y) of the component mounting range Ref(R) corresponds to the center of the component mounting range Ref(R) in plan view. The component mounting range (Ref(R)) indicated by the board configuration data 81 is an ideal range in which the component E should be mounted, and is not the range in which the component E is actually mounted by the surface mounting machine. The board configuration data 81 includes CAD data showing the configuration of the board B, the component E mounted in the component mounting range Ref(R) in the surface mount machine, and the component mounting range Ref(R). Inspection data representing standards for inspecting the propriety of the positional relationship can be used.

As shown in FIG. 5B, part data 82 represents the configuration of the part E by type of the part E. Specifically, the part data 82 includes the external shape of the part E, such as the length El, width Ew, and height Eh of the part E having a rectangular shape, or the light (pattern light L(S) )))) indicates the presence or absence of reflecting properties (presence or absence of reflection) for various parts (Ea, Eb, Ec, …).

The component model acquisition unit 111 confirms the type of component (E) to be mounted in the component mounting range (Ref(R)) of the substrate (B) using the substrate configuration data 81, and determines the type of component (E) of the corresponding type (E). ) The part model 83 is produced based on the results of confirming the configuration of the part data 82. This component model 83 includes the position (x, y) of the component mounting range (Ref(R)) and the external shape (X size of the component ( E_x), Y size (E_y) and Z size (E_z)). In other words, the component model 83 represents the range where the component E mounted in the component mounting range Ref(R) protrudes from the board B in the Z direction (height direction). Additionally, the component model 83 indicates whether the component E mounted in the component mounting range Ref(R) has the property of reflecting light (patterned light L(S)). This component model 83 is produced for each of the plurality of component mounting ranges Ref(R) formed on the substrate B.

For reference, the specific data used to produce the part model 83 and the specific method of producing the part model 83 using each data are not limited to this example. For example, when the part model 83 is input by the user's operation on the UI 200, the part model acquisition unit 111 creates the part model based on the board configuration data 81 and the part data 82. There is no need to produce (83), just acquire the part model (83) input by the user.

Figure 6 is a diagram schematically showing the contents of the illumination defect area calculated by the area calculation unit. The component E shown in FIG. 6 has the property of reflecting patterned light L(S). In Fig. 6, the component E mounted in the component mounting range Ref(R) has a width E_x in the X direction, a length E_y in the Y direction, and a height E_z in the Z direction. Then, the pattern light L(S) emitted from the projector 32 is projected onto the component E from the projection direction D.

As a result, in plan view, a shadow area As (poorly irradiated area) is generated on the downstream side of the component E in the projection direction D. This shadow area As is such that the pattern light L(S) projected onto the component E from the projector 32 is obscured by the component E, thereby forming a shadow of the component E on the substrate B. This is the area where it occurs. Additionally, in plan view, a secondary reflection area Ar (poorly irradiated area) occurs on the upstream side of the component E in the projection direction D. This secondary reflection area Ar is an area where light projected from the projector 32 onto the component E and reflected from the side of the component E is incident on the substrate B. In this example, the length of the shadow area As and the secondary reflection area Ar in the Y direction corresponds to the length E_y of the part E in the Y direction. Additionally, the width of the shadow area As and the secondary reflection area Ar in the Additionally, for the component E that does not reflect the pattern light L(S), the secondary reflection area Ar does not occur.

Such illumination defective areas (shadow area As and secondary reflection area Ar) are calculated for each of the plurality of projectors 32. Next, this point will be explained using FIG. 7. Figure 7 is a flowchart showing an example of calculation for calculating the irradiation defective area. The flow chart in FIG. 7 is executed by calculations of the main control unit 110.

In step S101, the count value P (=1, 2, 3, 4) identifying the projector 32 is reset to zero, and in step S102, the count value P is incremented. Additionally, in step S103, the count value R (=1, 2, 3,...) that identifies the component mounting range Ref is reset to zero, and in step S104, the count value R is incremented.

In step S105, the component model acquisition unit 111 produces a component model 83 of the component E to be mounted in the R-th component mounting range Ref(R). In step S106, the projection direction D in which the P-th projector 32 projects the pattern light L(S) with respect to the R-th component mounting range Ref is confirmed by the area calculation unit 112. do. Specifically, projection direction information 84 indicating the projection direction D of the pattern light L(S) by the projector 32 for each of the plurality of projectors 32 is stored in the storage unit 150. It is stored in advance, and the area calculation unit 112 checks the projection direction D by referring to the projection direction information 84.

In step S107, the patterned light (L The shadow area As (poor reflection area) generated when (S)) is projected is calculated by the area calculation unit 112. In addition, when the component E represented by the component model 83 manufactured for the R-th component mounting range Ref reflects the pattern light L(S), for the component E, P The secondary reflection area Ar (poor reflection area) generated when the pattern light L(S) is projected from the projection direction D by the th projector 32 is calculated by the area calculation unit 112. It is calculated.

In step S108, it is determined whether the count value R of the component mounting range Ref has reached the number Rmax of the component mounting range Ref formed on the board B, that is, for all component mounting ranges Ref formed on the board B. It is confirmed whether steps S105 to S107 have been completed. If there is a component mounting range Ref in which steps S105 to S107 have not been performed (in the case of “NO” in step S108), the process returns to step S104. In this way, steps S104 to S107 are repeated until steps S105 to S107 are executed for all component mounting ranges Ref (“YES” in step S108). As a result, the poor irradiation area (shadow area (As) and secondary reflection area (Ar)) that occurs when the pattern light (L(S)) is projected from the P-th projector 32 is within the entire component mounting range. Calculated for part (E) of (Ref).

In step S109, it is checked whether the count value P of the projectors 32 has reached the number Pmax (=4) of the projectors 32, that is, whether steps S103 to S108 have been completed for all projectors 32. If there is a projector 32 that has not performed steps S103 to S108 (in the case of “NO” in step S109), the process returns to step S102. In this way, steps S102 to S108 are repeated until steps S103 to S108 are executed for all projectors 32 (“YES” in step S109). As a result, when the pattern light L(S) is projected from the projector 32, the illumination defect area (shadow area As, Poor irradiation area information 85 indicating the secondary reflection area Ar for all projectors 32 is calculated and stored in the storage unit 150.

Using the irradiation defective area information 85 acquired in this way, three-dimensional measurement of the measurement object J is performed (FIG. 8). FIG. 8 is a flowchart showing a first example of three-dimensional measurement, FIG. 9 is a flowchart showing omnidirectional shape data acquisition performed in the three-dimensional measurement of FIG. 8, and FIG. 10 is a first example performed in the three-dimensional measurement of FIG. 8. It is a flowchart showing the data integration process, and FIG. 11 is a diagram schematically showing the calculation performed in the first data integration process of FIG. 10. The flow charts of FIGS. 8, 9, and 10 are executed under the control of the main control unit 110.

In the three-dimensional measurement of Fig. 8, omnidirectional shape data is acquired in step S201. As shown in Fig. 9, in omnidirectional shape data acquisition, recognition of the fiducial mark attached to the substrate B carried into the three-dimensional measurement device 1 is performed (step S301). Specifically, the imaging camera 31 is facing the fiducial mark within the imaging field of view V31 from above while irradiating illumination light from the illumination 33 to the fiducial mark. A mark image is acquired by imaging. The mark image is sent from the imaging control unit 130 to the main control unit 110, and the main control unit 110 determines the position of the substrate B held on the conveyor 2 from the position of the fiducial mark appearing in the mark image. detect.

In step S302, the count value P of the projector 32 is reset to zero, and in step S303, the count value P is incremented. Additionally, in step S304, the count value R of the component mounting range Ref is reset to zero, and in step S305, the count value R is incremented.

At the start of step S306, the main control unit 110 adjusts the position of the substrate B by the drive mechanism 4 so that the R-th component mounting range Ref enters the imaging field of view V31. In particular, the main control unit 110 adjusts the position of the substrate B in the imaging field of view V31 based on the position of the substrate B detected in step S301. Then, in step S306, the pattern image acquisition unit 113 controls the projection control unit 120 and the imaging control unit 130 to select a component E in the R-th component mounting range Ref from the P-th projector 32. ), the pattern light L(S) is imaged by the imaging camera 31 while projecting the pattern light L(S) onto the surface. As a result, as described above, four types of pattern images (I(S)) corresponding to different phases are acquired (S=1, 2, 3, 4) and stored in the storage unit 150.

In step S307, the shape calculation unit 114 calculates unidirectional shape data 86 based on the four types of pattern images I(S) using the phase shift method. This unidirectional shape data 86 is data representing the shape-related value Q regarding the three-dimensional shape of the component E mounted in the component mounting range Ref for each pixel PX (FIG. 11). Here, the pixel PX corresponds to, for example, a pixel of the individual imaging device 311, and since the imaging camera 31 captures an image while facing the substrate B from the Z direction, a plurality of the individual imaging devices 311 The pixels correspond to different positions (X coordinate, Y coordinate) on the substrate B. This shape-related value (Q) is a measured value (Qm) calculated as a height (Z coordinate) from four types of pattern images (I(S)) by a phase shift method, and the reliability (Qr) of the measured value (Qm). ) is composed of. In particular, in the example here, the shape-related value (Q) is given as the product of the measured value (Qm) and the reliability (Qr).

For reference, in the phase shift method, based on the difference in luminance shown by the pixels PX of four types of pattern images (I(S)) (S=1, 2, 3, 4), each pixel (PX) The reliability (Qr) can be calculated. Specifically, if the respective luminance values when four types of pattern images (I(S)) are imaged while shifting the phase by π/2 are d0, d1, d2, and d3, the phase shift angle α is expressed by the following equation

α=atan[(d2-d0)/(d3-d1)]

Calculated from, the reliability (Qr) is calculated from the following equation

Qr=[(d2-d0) ² +(d3-d1) ² ] ^1/2

It is calculated from

Additionally, the pixel PX when calculating the shape-related value Q does not need to correspond to the pixel of the individual imaging device 311. For example, when image processing is performed to convert the resolution of an image captured by the object imaging device 311, the shape-related value Q may be obtained from the pixel PX corresponding to the resolution after conversion.

In step S308, whether the count value R of the component mounting range Ref has reached the number Rmax of the component mounting range Ref formed on the board B, that is, for all component mounting ranges Ref formed on the board B It is confirmed whether acquisition of the pattern image I(S) (step S306) and calculation of the unidirectional shape data 86 (step S307) have been completed. If there is a component mounting range Ref in which steps S306 to S307 have not been performed (in the case of “NO” in step S308), the process returns to step S305. In this way, until the pattern image I(S) is acquired (step S306) and the unidirectional shape data 86 is calculated (step S307) for all component mounting ranges Ref (step S308, “YES”) ), steps S306 to S307 are repeated. As a result, unidirectional shape data 86 when the pattern light L(S) is projected from the P-th projector 32 is calculated for the components E in all component mounting ranges Ref.

In step S309, it is checked whether the count value P of the projectors 32 has reached the number Pmax (=4) of the projectors 32, that is, whether steps S304 to S308 have been completed for all projectors 32. If there is a projector 32 that has not performed steps S304 to S308 (in the case of “NO” in step S309), the process returns to step S303. In this way, steps S304 to S308 are repeated until steps S304 to S308 are executed for all projectors 32 (“YES” in step S309). As a result, when the pattern light L(S) is projected from the projector 32, the unidirectional shape data 86 calculated for the component E in each component mounting range Ref is distributed to all projectors 32. is acquired and stored in the storage unit 150.

In this way, when the omnidirectional shape data acquisition of the three-dimensional measurement in FIG. 9 (step S201 in FIG. 8) is completed, the first data integration process (step S202) is executed. In the first data integration process shown in FIG. 10, the count value R (=1, 2, 3, ...) identifying the component mounting range Ref is reset to zero (step S401), and the count value R is incremented. (Step S402). As a result, four unidirectional shape data ( 86) is specified. In other words, with respect to the component E in the R-th component mounting range Ref, four pieces of unidirectional shape data 86 corresponding to the four projectors 32 are specified.

Subsequently, the count value N (N=1, 2, 3,...) that identifies the pixel PX is reset to zero (step S403), and the count value N is incremented (step S404). Additionally, in step S405, the count value P (=1, 2, 3, 4) that identifies the projector 32 is reset to zero, and in step S406, the count value P is incremented. By this, the N-th pixel (PX) of the unidirectional shape data 86 corresponding to the P-th projector 32 is designated.

Then, the shape calculation unit 114 determines whether the N-th pixel PX designated in this way belongs to the shadow area As or the secondary reflection area Ar by using the poor-irradiation area information of the storage unit 150 ( 85), determining the weight coefficient W(P) multiplied by the shape-related value Q (=measured value Qm×reliability Qr) of the N-th pixel PX (step S407). (Steps S408, S409). That is, if it is determined by the poor irradiation area information 85 that the N-th pixel (PX) belongs to the shadow area (As) or the secondary reflection area (Ar) (“YES” in step S407), the N-th pixel The weight coefficient W(P) for the shape-related value Q of (PX) is determined to be "0", and the shape-related value Q is excluded (step S408). On the other hand, if it is determined by the poor irradiation area information 85 that the N-th pixel (PX) does not belong to either the shadow area (As) or the secondary reflection area (Ar) (“NO” in step S407), N The weight coefficient W(P) for the shape-related value Q of the th pixel PX is determined to be 1 (step S410).

The calculation in steps S407 to S409 for the shape-related value (Q) of the N-th pixel (PX) increments the count value P of the projector 32 (step S406), and the count value P is changed to Pmax (=4). ) (“YES” in step S410). By this, the weight coefficient W(P) with respect to the shape-related value Q of the Nth pixel PX corresponding to each of P=1, 2, 3, and 4 is determined. In the example of FIG. 11, it is determined that the Nth pixel (PX) corresponding to the projector 32 with P = 2 belongs to the shadow area (As) or the secondary reflection area (Ar), and the pixel (PX) The weight coefficient W(2) for the shape-related value (Q) is determined to be “0”. On the other hand, it is determined that the Nth pixel (PX) corresponding to the projector 32 with P = 1, 3, and 4 does not belong to any of the shadow area (As) and the secondary reflection area (Ar), and these pixels The weight coefficients W(1), W(3), and W(4) for the shape-related value (Q) of (PX) are determined to be “1.”

In step S411, the shape calculation unit 114 calculates the shape-related value Q of the N-th pixel PX corresponding to each projector 32 with P=1, 2, 3, and 4, and performs step S407 to S409. By executing an operation to obtain a weighted average using the weight coefficient W (P) determined in , an integrated value (H) that integrates these shape-related values (Q) is obtained. The weighted average equation for calculating the integrated value (H) is as shown in the “Conversion Formula” column of FIG. 11. As a result, in the example of FIG. 11, the integrated value (H) of the Nth pixel (PX) becomes 159.

The calculations in steps S405 to S411 increment the count value N until the count value N reaches the number Nmax of the pixels (PX) constituting the unidirectional shape data 86 (“YES” in step S412) ( Step S404) is repeated. In this way, integrated shape data 87 representing for each pixel PX an integrated value H that integrates four shape-related values Q corresponding to different projectors 32 is calculated, and stored in the storage unit 150. It is saved in This integrated shape data 87 is data that integrates the four unidirectional shape data 86 corresponding to the four projectors 32 acquired for the component E mounted in the R-th component mounting range Ref. It corresponds to and represents the three-dimensional shape of E above.

The calculations in steps S403 to S412 are repeated while incrementing the count value R (step S402) until the count value R reaches the number Rmax of the component mounting range Ref (“YES” in step S413). In this way, integrated shape data 87 is calculated for the components E in all component mounting ranges Ref.

In the embodiment described above, a component mounting range (Ref) in which the component (E) is mounted on the board (B) and a component model (83) representing the external shape of the component (E) mounted in the component mounting range (Ref). (reference model) is acquired (step S105). In addition, projection direction information 84 (irradiation direction information) indicating the projection direction D in which the pattern light L(S) is projected onto the part E in the three-dimensional measurement device 1, and the part model 83 Based on this, the shadow area As and the secondary reflection area Ar (poorly irradiated area) are calculated (step S107). In this way, in measuring the three-dimensional shape of the measurement object J based on the pattern image I(S) acquired by imaging the pattern light L(S) irradiated on the measurement object J, the pattern light It is possible to acquire the shadow area (As) and the secondary reflection area (Ar) where the illumination defect of (L) occurs.

In addition, the component model acquisition unit 111 (reference model acquisition unit) provides inspection data representing a standard for inspecting the adequacy of the positional relationship between the component E mounted in the component mounting range Ref and the component mounting range Ref. and CAD data representing the structure of the substrate, a component model 83 is produced from at least one of the data (step S105). In this configuration, a part model 83 is created using existing data such as inspection data or CAD data of the substrate B, and the shadow area As and secondary reflection area are created based on this part model 83. (Ar) can be calculated.

In addition, a plurality of projectors 32 (pattern irradiation units) are installed to irradiate pattern light L(S) to the component E from different projection directions D, and the projection direction information 84 is provided by the projector ( 32), the projection direction D in which light is irradiated to the component E is shown for each of the plurality of projectors 32. In this configuration, when the projector 32 projecting the pattern light L(S) changes between the plurality of projectors 32, the shadow area As and the secondary reflection area Ar also change. In response to this, the area calculation unit 112 calculates the shadow area As and the secondary reflection area Ar for each of the plurality of projectors 32 (steps S102 and S107). As a result, the shadow area As and the secondary reflection area Ar that are generated when light is projected onto the component E from each of the plurality of projectors 32 can be acquired.

Furthermore, based on the pattern image I(S) acquired by imaging the pattern light L(S) irradiated from the projector 32 to the substrate B by the imaging camera 31 (imaging unit), The shape calculation unit 114 performs an operation (step S307) to calculate unidirectional shape data 86 representing the shape-related value Q, which is a value related to the three-dimensional shape, for each pixel PX. This calculation (step S307) is performed for each of the plurality of projectors 32 (step S303), and a plurality of unidirectional shape data 86 corresponding to the different projectors 32 are calculated. In addition, the shape calculation unit 114 integrates the plurality of unidirectional shape data 86 by calculating the weighted average of the shape-related value Q represented by each of the plurality of unidirectional shape data 86 for each pixel PX, and performs measurement. A first integrated operation process (FIG. 10) that calculates the three-dimensional shape of the object J is executed. In this first integrated calculation process, among the shape-related values (Q) indicated by the unidirectional shape data 86, the shape-related value (Q) of the pixel (PX) included in the shadow area (As) or the secondary reflection area (Ar) ), the three-dimensional shape of the measurement object J is calculated by a weighted average of multiplying the weight coefficient W(P) of less than 1 and greater than or equal to 0 (step S411).

In this configuration, based on the pattern image I(S) acquired by imaging the pattern light L(S) irradiated from the projector 32 to the substrate B by the imaging camera 31, a three-dimensional shape is formed. Unidirectional shape data 86 representing the shape-related value Q for each pixel PX is calculated (step S307). In other words, the unidirectional shape data 86 relates to a three-dimensional shape calculated based on the pattern image I(S) obtained by imaging the pattern light L(S) irradiated to the component E from one projector 32. It's data. The calculation for calculating this unidirectional shape data 86 (step S307) is performed for each of the plurality of projectors 32 (step S303). As a result, unidirectional shape data 86 when the pattern light L(S) is irradiated to the component E from each of the plurality of projectors 32 is acquired. The plurality of unidirectional shape data 86 acquired in this way are based on the pattern image I(S) acquired while irradiating the pattern light L(S) to the part E from different projection directions D, respectively. . Therefore, the pixel PX corresponding to the shadow area As or the secondary reflection area Ar in one unidirectional shape data 86 is a well-illuminated area in the other unidirectional shape data 86. It may apply to Therefore, by calculating the average of the shape-related values Q represented by each of the plurality of unidirectional shape data 86 for each pixel PX, the plurality of unidirectional shape data 86 are integrated, and the three-dimensional shape of the measurement object J is determined. It can be calculated (Figure 10). In addition, in the first integrated operation process in FIG. 10, the shape of the pixel PX included in the shadow area As or the secondary reflection area Ar among the shape-related values Q indicated by the unidirectional shape data 86. The three-dimensional shape of the measurement object J is calculated by a weighted average of multiplying the relevant value Q by a weight coefficient W(P) less than 1 and greater than or equal to 0 (steps S407 to S410). This makes it possible to appropriately calculate the three-dimensional shape of the measurement object J while suppressing the influence of the shadow area As or the secondary reflection area Ar.

In particular, in the first integrated operation process of FIG. 10, a weight coefficient W (P) of 0 is applied to the shape-related value (Q) of the pixel (PX) included in the shadow area (As) or the secondary reflection area (Ar). By multiplying the weighted average, the three-dimensional shape of the measurement object J is calculated (steps S407 to S410). In this way, the operation of multiplying the weight coefficient W(P) of 0 is an operation excluding the shadow area (As) and the secondary reflection area (Ar). This makes it possible to appropriately calculate the three-dimensional shape of the measurement object J while excluding the influence of the shadow area As and the secondary reflection area Ar.

FIG. 12 is a flowchart showing an example of calculation for correcting the poorly irradiated area calculated in FIG. 7. The flow chart in FIG. 12 is executed by calculations of the main control unit 110 in parallel with the acquisition of the omnidirectional shape data in FIG. 9. In step S501, the count value P of the projector 32 is reset to zero, and in step S502, the count value P is incremented. Additionally, in step S503, the count value R of the component mounting range Ref is reset to zero, and in step S504, the count value R is incremented. As a result, the shadow area As and the secondary reflection area Ar calculated for the component E in the R-th component mounting range Ref are specified, corresponding to the P-th projector 32.

In step S505, the shadow area As and the secondary reflection area Ar specified in steps S502 and S504 are corrected based on the position of the substrate B detected in step S301 of omnidirectional shape data acquisition in FIG. 9. do. That is, the position of the substrate B detected in step S301 is a positional deviation amount Δa with respect to the abnormal position of the substrate B assumed in calculating the shadow area As and the secondary reflection area Ar. It is assumed that there is a discrepancy of that amount. In this case, by correcting the shadow area As and the secondary reflection area Ar according to the positional deviation amount Δa of the substrate B, the actual position of the substrate B carried into the three-dimensional measuring device 1 is determined. Therefore, the shadow area (As) and secondary reflection area (Ar) can be accurately calculated.

In step S506, the shadow area As and the secondary reflection area Ar specified in steps S502 and S504 are corrected based on the position of the component E actually mounted in the R-th component mounting range Ref. . The details of this are as follows.

In control using poor irradiation area correction in FIG. 12, in step S306 of omnidirectional shape data acquisition in FIG. 9, a two-dimensional image of the part E is acquired in addition to the pattern image I(S). Specifically, while the illumination 33 radiates illumination light to the component E within the imaging field of view V31, the imaging camera 31 captures the component E from above, thereby obtaining a two-dimensional image of the component E. do. Additionally, in step S306, a component position indicating the positional relationship between the component E actually mounted in the component mounting range Ref of the board B carried into the three-dimensional measurement device 1 and the component mounting range Ref is determined. It is calculated based on the two-dimensional image of the part E and stored in the storage unit 150.

In contrast, the positional relationship between the component mounting range (Ref) and the component (E) indicated by the component position calculated in step S306 is assumed in the calculation of the shadow area (As) and the secondary reflection area (Ar). Regarding the positional relationship, it is assumed that there is a shift by the positional shift amount Δb. In such a case, the shadow area As and the secondary reflection area Ar are corrected according to the positional deviation Δb of the component E with respect to the component mounting range Ref, so that the Depending on the actual position of the component (E) mounted in the component mounting range (Ref) of the board (B), the shadow area (As) and secondary reflection area (Ar) can be accurately calculated.

In step S507, it is determined whether the count value R of the component mounting range Ref has reached the number Rmax of the component mounting range Ref formed on the board B, that is, for all the component mounting ranges Ref formed on the board B. It is confirmed whether the correction in steps S505 to S506 has been completed. If there is a component mounting range Ref for which the correction in steps S505 to S506 has not been performed (in the case of “NO” in step S507), the process returns to step S504. In this way, steps S505 to S506 are repeated until the correction in steps S505 to S506 is performed for all component mounting ranges Ref (“YES” in step S507). As a result, the illumination defect area (shadow area As and secondary reflection area) for each component mounting range Ref that occurs when the pattern light L(S) is projected from the P-th projector 32. (Ar)) is corrected.

In step S508, it is checked whether the count value P of the projectors 32 has reached the number Pmax (=4) of the projectors 32, that is, whether steps S503 to S507 have been completed for all projectors 32. If there is a projector 32 that has not performed steps S503 to S507 (in the case of “NO” in step S508), the process returns to step S502. In this way, steps S503 to S507 are repeated until steps S503 to S507 are executed for all projectors 32 (“YES” in step S508). As a result, when the pattern light L(S) is projected from each projector 32, the illumination defect area (shadow area As) generated due to the component E in each component mounting range Ref ·Secondary reflection area (Ar) is corrected.

Then, in the first data integration process in Fig. 10, steps S407 to S411 are executed based on the shadow area As and the secondary reflection area Ar corrected in this way. Accordingly, based on the accurately set shadow area As and secondary reflection area Ar for the measurement object J (substrate B and component E) actually brought into the three-dimensional measurement device 1. Thus, integrated shape data 87 can be calculated.

In the example of FIG. 12 , the area calculation unit 112 further relies on the results of recognizing the position of the substrate B brought into the three-dimensional measurement device 1 and supported on the conveyor 2 (object support portion) in step S301. Thus, the shadow area (As) and the secondary reflection area (Ar) are calculated (step S505). In this configuration, the shadow area As and the secondary reflection area Ar can be accurately calculated according to the actual position of the substrate B loaded into the three-dimensional measurement device 1.

In particular, the position of the substrate B supported on the conveyor 2 is recognized based on the results of imaging the fiducial mark attached to the substrate B with the imaging camera 31 (step S301). In this configuration, the shadow area As and the secondary reflection area Ar can be accurately calculated according to the position of the substrate B loaded into the three-dimensional measurement device 1.

In addition, the area calculation unit 112 calculates the shadow area As and the secondary reflection area Ar based on the result of detecting the position of the component E mounted in the component mounting range Ref in step S306. Do it (step S506). In this configuration, the shadow area As and the secondary reflection area Ar can be accurately calculated according to the actual position of the component E mounted in the component mounting range Ref.

Additionally, the position of the component E mounted in the component mounting range Ref is detected based on a two-dimensional image captured of the component E (step S306). In this configuration, the shadow area As and the secondary reflection area Ar can be accurately calculated according to the actual position of the component E mounted in the component mounting range Ref.

FIG. 13 is a flowchart showing a second example of three-dimensional measurement, FIG. 14 is a flowchart showing a second data integration process executed in the three-dimensional measurement of FIG. 13, and FIG. 15 is a flowchart of an operation performed in the three-dimensional measurement of FIG. 13. It is a drawing that schematically represents the contents. Here, a scene is assumed in which a plurality of substrates B are sequentially loaded into the three-dimensional measurement device 1 and three-dimensional measurement is performed on each substrate B.

As shown in Fig. 13, in the second example of three-dimensional measurement, omnidirectional shape data acquisition (step S601) and first data integration processing (step S602) are performed similarly to the first example. Additionally, in the second example, the second data integration process (FIG. 14) is executed in step S603. Additionally, the execution order of the first data integration process and the second data integration process is not limited to the example in FIG. 13 and may be the reverse of the example in FIG. 13.

As shown in FIG. 14, the second data integration process differs from the first data integration process in that instead of steps S405 to S411 of the first data integration process, the simple average (average with an overall weight coefficient of 1) is calculated in step S414. The point is to run . That is, in step S414 of the second data integration process, the simple average of the shape-related value Q of the Nth pixel PX obtained for each of the four projectors 32 is calculated. In this way, the integrated value H is calculated by a simple average rather than a weighted average, and the integrated shape data 87 is acquired.

In this way, when the three-dimensional measurement is completed for one board B (steps S601 to S603), it is determined whether steps S601 to S603 have been completed for a predetermined number of boards B (one or more sheets) (steps S601 to S603). S604). If the execution is not complete (“NO” in step S604), it is determined in step S605 whether to end the measurement. When measurement is terminated (in the case of “YES” in step S605), the three-dimensional measurement in Fig. 13 is terminated, while when the measurement is not terminated (in the case of “NO” in step S605), the process returns to step S601.

In step S604, if it is determined that steps S601 to S603 have been completed for a predetermined number of substrates B (YES), the integrated shape data 87 (three-dimensional shape) acquired in the first data integration process in step S602 and , the difference between the integrated shape data 87 (three-dimensional shape) acquired in the second data integration process in step S603 is calculated. For example, for the same component mounting range (Ref) on the same board (B), the integrated shape data 87 (three-dimensional shape) acquired in the first data integration process and the integrated shape acquired in the second data integration process The average value (i.e., mean square error) of the squares of the differences of the data 87 (three-dimensional shape) is calculated. In addition, when one board B has a plurality of component mounting ranges (Ref), the mean square error may be calculated for each component mounting range (Ref), and the mean square error may be calculated for one representative component mounting range (Ref). You may calculate the mean square error. Additionally, the mean square error is calculated for each of the predetermined number of substrates B. The average value, median value, or maximum value of each mean square error calculated in this way is calculated as the difference. Additionally, the specific calculation method for the difference is not limited to this example, and any calculation method that can evaluate the difference between two data can be adopted.

In step S607, the necessity of the first data integration process is determined based on the difference. Specifically, when the difference is more than a predetermined threshold, it is determined that the first data integration process is necessary (YES), and when the difference is less than the threshold, it is determined that the first data integration process is unnecessary. For example, as shown in each waveform (three-dimensional shape) of the column “Influenced by irradiation defects” in Figure 15, when there is an influence of irradiation defects, the peak noise generated in the three-dimensional shape obtained in the second data integration process is , does not appear in the three-dimensional shape obtained in the first data integration process. This indicates that a first data integration process is necessary to eliminate the influence of survey defects. On the other hand, as shown in each waveform (three-dimensional shape) in the “No effect of irradiation defect” column in FIG. 15, when there is no influence of irradiation defect, the three-dimensional shape obtained in the first data integration process and the three-dimensional shape obtained in the second data integration process There is no significant difference between three-dimensional shapes. This indicates that the first data integration process is unnecessary.

If it is determined that the first data integration process is necessary (“YES” in step S607), each time one substrate B is loaded into the three-dimensional measurement device 1, omnidirectional shape data is acquired (step S608), 1 Data integration processing (step S609) and determination of completion of measurement (step S610) are performed. In this way, the integrated shape data 87 is calculated by the first data integration process rather than the second data integration process (step S609). On the other hand, if it is determined that the first data integration process is unnecessary (“NO” in step S607), omnidirectional shape data is acquired each time one substrate B is loaded into the three-dimensional measurement device 1 (step S611). , second data integration processing (step S612) and determination of completion of measurement (step S613) are performed. In this way, the integrated shape data 87 is calculated by the second data integration process rather than the first data integration process (step S612).

In the example of FIG. 13 , the shape calculation unit 114 executes both the first data integration process (step S602) and the second data integration process (step S603) on the same measurement object J. In this second data integration process, the shape calculation unit 114 calculates the shape-related value (Q) indicated by the plurality of unidirectional shape data 86 without performing weighting according to the shadow area (As) and the secondary reflection area (Ar). By calculating the average (simple average) for each pixel PX, the plurality of unidirectional shape data 86 are integrated, and the three-dimensional shape (integrated shape data 87) of the measurement object J is calculated. In addition, the shape calculation unit 114 calculates the difference between the integrated shape data 87 (three-dimensional shape) of the measurement object J calculated in each of the first data integration process (step S602) and the second data integration process (step S603). An evaluation process (step S606) for evaluating is performed on a predetermined number of measurement objects J, which are the objects of acquisition of the pattern image I(S), in the three-dimensional measurement device 1. Based on this evaluation process, a determination of whether the first data integration process is necessary or not is performed (step S607). Then, in the necessity judgment of step S607, after it is determined that the first data integration process is unnecessary, the shape calculation unit 114 does not perform the first data integration process, but measures the measurement object (J) through the second data integration process. ) of the integrated shape data 87 (three-dimensional shape) is calculated (steps S611 to S613). In this configuration, when the shadow area As and the secondary reflection area Ar are not a problem, the integrated shape data of the measurement object J is obtained by the second data integration process, which is simpler than the first data integration process. 87) can be calculated accurately.

Additionally, in this example, the "predetermined number of sheets" determined in step S604 may be configured so that the user can set it. In this case, the user performs an operation to set a predetermined number of sheets in the UI 200, and the UI 200 accepts the predetermined number of sheets input by the user and sets it to be used in the three-dimensional measurement in FIG. 13. In this configuration, the user can appropriately adjust the period (steps S601 to S604) during which both the first data integration process (step S602) and the second data integration process (step S603) are executed.

In this way, in the above embodiment, the three-dimensional measurement device 1 corresponds to an example of the “three-dimensional measurement device” of the present invention, and the control device 100 corresponds to an example of the “three-dimensional measurement arithmetic device” of the present invention, The part model acquisition unit 111 corresponds to an example of the “reference model acquisition unit” of the present invention, the area calculation unit 112 corresponds to an example of the “area calculation unit” of the present invention, and the shape calculation unit 114 corresponds to an example of the “shape calculation unit” of the present invention, the conveyor 2 corresponds to an example of the “object support unit” of the present invention, and the imaging camera 31 corresponds to an example of the “image capturing unit” of the present invention. Correspondingly, the projector 32 corresponds to an example of the “projector” of the present invention, the plurality of projectors 32 corresponds to an example of the “pattern irradiation unit” of the present invention, and the part model 83 corresponds to an example of the “pattern irradiation unit” of the present invention. Corresponds to an example of a “reference model”, projection direction information 84 corresponds to an example of “irradiation direction information” of the present invention, and unidirectional shape data 86 corresponds to an example of “unidirectional shape data” of the present invention. , the server computer 91 corresponds to an example of the “recording medium” of the present invention, the three-dimensional measurement program 92 corresponds to an example of the “three-dimensional measurement program” of the present invention, and the UI 200 corresponds to an example of the “three-dimensional measurement program” of the present invention. It corresponds to an example of the “setting operation section”, the secondary reflection area (Ar) corresponds to an example of the “secondary reflection area” and the “poor irradiation area” of the present invention, and the shadow area (As) corresponds to an example of the “shadow area” of the present invention. corresponds to an example of a “region” and a “poor irradiation area”, the substrate B corresponds to an example of a “substrate” of the present invention, the component E corresponds to an example of a “part” of the present invention, and the pattern image (I(S)) corresponds to an example of the “pattern image” of the present invention, the measurement object J corresponds to an example of the “measurement object” of the present invention, and the pattern light L(S) corresponds to an example of the “measurement object” of the present invention. corresponds to an example of "light", the pixel PX corresponds to an example of the "pixel" of the present invention, the shape-related value (Q) corresponds to an example of the "shape-related value" of the present invention, and the measured value (Qm) corresponds to an example of the “calculated value” of the present invention, reliability (Qr) corresponds to an example of the “reliability” of the present invention, and the component mounting range (Ref) corresponds to an example of the “component mounting range” of the present invention. Corresponding to an example, step S606 corresponds to an example of the “evaluation process” of the present invention, step S607 corresponds to an example of the “decision of suitability” of the present invention, and the first data integration process in FIG. 10 corresponds to the “evaluation process” of the present invention. It corresponds to an example of the “first integrated arithmetic process”, and the second data integration process in FIG. 14 corresponds to an example of the “second integrated arithmetic process” of the present invention.

Additionally, the present invention is not limited to the above-described embodiments, and various changes can be made to the above-described embodiments without departing from the spirit thereof. For example, it is not necessary to calculate both the secondary reflection area (Ar) and the shadow area (As) as the poorly irradiated area. For example, if the influence of one of the secondary reflection area Ar and the shadow area As is large and the influence of the other is small, even if only one of the secondary reflection areas Ar and the shadow area As is calculated as the poor irradiation area in step S107 of FIG. good night.

In addition, in the correction of the irradiation defective area in FIG. 12, only one of correction based on the position of the substrate B (step S505) and correction based on the position of the component E (step S506) is performed and the other is performed. It's okay if you don't do it.

Additionally, the specific content of the shape-related value (Q) may be changed as appropriate. For example, the measured value Qm may be calculated as the shape-related value Q without calculating the reliability Qr.

Additionally, in step S408, the weight coefficient W(P) multiplied corresponding to the defective irradiation area does not need to be 0, but can be a value of 0 or more and less than 1.

In addition, by executing the three-dimensional measurement program 92, each of the functional units 111, 112, 113, and 114 comprised in the main control unit 110 must be connected to the control device 100 provided in the three-dimensional measurement device 1. There is no need to configure it. Accordingly, each of the functional units 111, 112, 113, and 114 may be configured in a processor provided in a computer installed separately from the three-dimensional measurement device 1.

Additionally, in the second data integration process in FIG. 1, in step S414, instead of calculating the average value (representative value) by simple average, the median value (representative value) may be calculated.

1: Three-dimensional measurement device
100: Control device (operation device for three-dimensional measurement)
111: Part model acquisition unit (standard model acquisition unit)
112: Area calculation unit (area calculation unit)
114: Shape calculation unit
2: Transfer conveyor (object support)
31: Imaging camera (imaging unit)
32: Projector (pattern investigation department)
83: Part model (base model)
84: Projection direction information (irradiation direction information)
86: One-way shape data
91: Server computer (recording media)
92: Three-dimensional measurement program
200: UI (Settings Control Panel)
Ar: Secondary reflection area (poorly irradiated area)
As: Shadow area (poorly irradiated area)
B: substrate
E: Parts
I(S): Pattern image
J: Measurement object
L(S): pattern light (light)
PX: pixel
Q: Shape-related values
Qm: measured value (calculated value)
Qr: Reliability
Ref: Component mounting range

Claims (17)

an object support unit that supports a measurement object having a substrate and components mounted on the substrate, a pattern irradiation unit that irradiates light of a predetermined pattern to the measurement object supported on the object support unit, and irradiation of light from the pattern irradiation unit to the measurement object. A calculation device for three-dimensional measurement that performs calculations for measuring the three-dimensional shape of the measurement object based on the pattern image acquired by a three-dimensional measurement device having an imaging unit that acquires a two-dimensional pattern image by imaging light, comprising:
a reference model acquisition unit that acquires a reference model representing a component mounting range in which the component is mounted on the board and an external appearance of the component mounted in the component mounting range;
A shadow area where the light irradiated from the imaging unit to the component is obscured by the component, thereby creating a shadow of the component on the substrate, and a shadow area where the light irradiated from the imaging unit and reflected from the component is incident on the substrate. A calculation device for three-dimensional measurement including an area calculation unit that calculates at least one poor irradiation area among secondary reflection areas based on the reference model and irradiation direction information indicating a direction in which light is irradiated from the three-dimensional measurement device to the component. . According to claim 1,
The reference model acquisition unit includes at least one of inspection data representing a standard for inspecting the appropriateness of the positional relationship between the component mounted in the component mounting range and CAD (Computer-Aided Design) data representing the configuration of the board. A three-dimensional measurement computing device that produces the reference model from one piece of data. The method of claim 1 or 2,
The area calculation unit calculates the irradiation defective area based on a result of recognizing the position of the substrate carried into the three-dimensional measurement device and supported on the object support unit. According to claim 3,
The position of the substrate supported on the object support portion is recognized based on a result of imaging a fiducial mark attached to the substrate by the imaging unit. The method according to any one of claims 1 to 4,
The area calculation unit calculates the irradiation defective area based on a result of detecting the position of the component mounted in the component mounting range. According to claim 5,
A three-dimensional measurement computing device in which the position of the component mounted in the component mounting range is detected based on a result of imaging the component. The method according to any one of claims 1 to 6,
The pattern irradiation unit has a plurality of projectors that irradiate light to the component from different directions,
The irradiation direction information indicates a direction in which light is irradiated from the projector to the component, for each of the plurality of projectors,
The area calculation unit calculates the illumination defective area for each of the plurality of projectors. According to claim 7,
A calculation for calculating unidirectional shape data representing a shape-related value, which is a value related to a three-dimensional shape, for each pixel, based on the pattern image acquired by imaging the light irradiated from the projector to the substrate by the imaging unit, It further includes a shape calculation unit that calculates a plurality of unidirectional shape data by executing it for each of the projectors,
The shape calculation unit integrates the plurality of unidirectional shape data by calculating an average of the shape-related values represented by each of the plurality of unidirectional shape data for each pixel, and performs a first integrated operation process to calculate a three-dimensional shape of the measurement object. run,
In the first integrated operation processing, among the shape-related values indicated by the unidirectional shape data, the shape-related values of the pixels included in the poorly irradiated area are multiplied by a weight coefficient that is less than 1 and greater than or equal to 0, by weighted average, A three-dimensional measurement computing device that calculates the three-dimensional shape of the measurement object. According to claim 8,
In the first integrated calculation process, the three-dimensional shape of the measurement object is calculated by a weighted average in which the shape-related values of the pixels included in the illumination defect area are multiplied by a weight coefficient of 0. According to claim 8 or 9,
An arithmetic device for three-dimensional measurement in which the shape-related value is given as a product of a calculated value of a three-dimensional shape calculated based on the pattern image and a reliability of the calculated value. According to claim 8 or 9,
The shape-related value is a calculation device for three-dimensional measurement that is a calculated value of a three-dimensional shape calculated based on the pattern image. The method according to any one of claims 8 to 11,
The shape calculation unit integrates the plurality of unidirectional shape data by calculating a representative value of the shape-related value represented by the plurality of unidirectional shape data for each pixel without performing weighting according to the irradiation defect area, and thereby integrates the plurality of unidirectional shape data into three dimensions of the measurement object. Both a second integrated calculation process for calculating a shape and the first integrated calculation process are performed on the same measurement object, and the three dimensions of the measurement object calculated in each of the first integrated calculation process and the second integrated calculation process are performed. Executing an evaluation process for evaluating differences in shape on a predetermined number of measurement objects that are targets of acquisition of the pattern image by the three-dimensional measurement device, thereby performing a desirability judgment to determine the desirability of the first integrated calculation process,
After the first integrated calculation process is determined to be unnecessary in the necessity determination, the shape calculation unit calculates a three-dimensional shape of the measurement object by the second integrated calculation process without performing the first integrated calculation process. Computation device for three-dimensional measurement. According to claim 12,
An arithmetic device for three-dimensional measurement, further comprising a setting operation unit that accepts an operation for setting the predetermined number. A three-dimensional measurement program that causes a computer to function as the three-dimensional measurement computing device according to any one of claims 1 to 13. A recording medium that records the three-dimensional measurement program according to claim 14 in a computer-readable manner. an object support portion that supports a measurement object having a substrate and components mounted on the substrate;
a pattern irradiation unit that irradiates light of a predetermined pattern to the measurement object supported on the object support unit;
an imaging unit that acquires a two-dimensional pattern image by imaging light irradiated from the pattern irradiation unit to the measurement object;
A three-dimensional measurement device comprising the arithmetic device for three-dimensional measurement according to any one of claims 1 to 13, which performs calculations for measuring the three-dimensional shape of the measurement object based on the pattern image. an object support unit that supports a measurement object having a substrate and components mounted on the substrate, a pattern irradiation unit that irradiates light of a predetermined pattern to the measurement object supported on the object support unit, and irradiation of light from the pattern irradiation unit to the measurement object. A calculation method for three-dimensional measurement that performs calculations for measuring the three-dimensional shape of the measurement object based on the pattern image acquired by a three-dimensional measurement device having an imaging unit that acquires a two-dimensional pattern image by imaging light, comprising:
A process of acquiring a reference model representing a component mounting range in which the component is mounted on the board and an external appearance of the component mounted in the component mounting range;
A shadow area where the light irradiated from the imaging unit to the component is obscured by the component, thereby creating a shadow of the component on the substrate, and a shadow area where the light irradiated from the imaging unit and reflected from the component is incident on the substrate. A calculation method for three-dimensional measurement comprising a step of calculating at least one illumination defective area of a secondary reflection area based on the reference model and irradiation direction information indicating the direction in which light is irradiated from the three-dimensional measurement device to the component.