JPH06201338A - Outer appearance state inspection - Google Patents

Abstract

PURPOSE:To provide an inspection method of especially a soldering state after mounting an electronic part and a board flexibly corresponding to various conditions concerning an outer appearance inspection method of general parts. CONSTITUTION:An inspected article is irradiated by luminous means 11-17, its image is picked up by a sensor of an image-up camera 2 and others and converted to an electrical signal. Additionally, this electrical signal is, by picture image processing means 3, 4, formed into a pattern of a shape information code data and a pattern of a variable density contrast distribution data. It is constituted by means of a neurocomputer having a learning recognizing ability to accurately grasp, inspect and judge an outer appearance state of the inspected article.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for inspecting the appearance of general components, and more particularly to a method for inspecting the soldering state of electronic components after they are mounted on a board.

[0002]

2. Description of the Related Art As an example of an object to be inspected as a three-dimensional shape, a conventional apparatus has a plurality of different light source light sources for a soldering portion to be inspected, as described in JP-A-61-41906. Of the soldering surface, the change in the value of the reflecting surface obtained each time is obtained by a light receiving element or a television camera to obtain image information of several inclined surfaces of the soldering surface (hereinafter referred to as a step illumination device). ). A judgment standard is created by numerical calculation processing of these various image information, and advanced software is used to sort the solder state, for example, good soldering, insufficient soldering, excessive soldering, poor soldering, none, lead floating, lead deviation. ,
It was classified as abnormal and judged.

[0003] The above-mentioned prior art does not take into consideration, for example, the variety of solder surface states, the progress thereof, and the flexibility, as an example of an object to be inspected as a three-dimensional shape. In other words, the electronic components mounted on the mounting board are further densified and miniaturized, the soldering technology is improved, and the quality standard of soldering is changed each time depending on the destination of the board. Therefore, it is necessary to change the content of the numerical calculation processing of the image information every time.

However, these studies are very time-consuming and very difficult. Further, setting this judgment criterion requires a high level of analysis know-how because various know-how needs to be accumulated, and setting the judgment criterion is very difficult and the judgment is delicate. For this reason, there is a problem that the determination is insufficient, and the inspection accuracy sometimes deteriorates due to a large variation in the inspection result from time to time. Further, the determination processing time becomes long, which is a problem. On the other hand, as a method for solving these problems, an inspection apparatus applying a neural network has been proposed. As an example thereof, there is Japanese Patent Application No. 3-81206. This invention is an inspection device using step illumination,
The three-dimensional code data is processed by a neuro computer. Therefore, the learning function of the neural network makes it possible to set delicate criteria.

In these conventional methods, grasping the state of a finer plane requires an increase in the number of stages of illumination, which not only complicates the configuration of the apparatus but also requires a longer processing time. Also becomes longer and there are many practical problems.

[0005]

In order to solve the above problems, the present invention irradiates an article to be inspected with a light emitting means,
An image is picked up by a sensor such as an image pickup camera and converted into an electric signal. Further, a pattern of shape information code data (that is, three-dimensional shape data) and a pattern of grayscale gradation distribution data are created from this electric signal by the image processing means.
Based on the respective pattern data, it is constituted by means of a neuro computer having a learning and recognition function for accurately grasping, inspecting and judging the appearance state of the article to be inspected.

[0006]

The shape information code data of the article to be inspected (that is, three-dimensional shape data) obtained by the light emitting means and the sensor.
The pattern and the grayscale distribution data pattern correspond to the relationship between the judgment standard of the article to be inspected and the detection data in the neurocomputer, and various data of various levels of the appearance state of the article to be inspected. The network in the neuro-computer is weighted and configured so as to have an optimum decision structure by learning by. As described above, the neuro computer handles the subtle change in the shape by the two data of the shape information and the gradation distribution, and makes an optimum decision.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT An embodiment of a visual inspection apparatus in a soldered state will be described in detail below as an example of a three-dimensionally shaped object to be inspected according to the present invention. FIG. 1 is a schematic configuration diagram of a main part of a soldering state inspection device. In the figure, a plurality of annular illumination devices 11, 13, 15, 17 are provided above the mounting board 1 on which electronic components are soldered as a light source for changing the irradiation angle to the soldered portion to be inspected.
Are arranged. Further, the image pickup camera 2 or 2 ′ is arranged above, and the image pickup camera 2 or 2 ′ is the image processing device 3,
4 and the recognition determination device 5. Each lighting device is kept at a constant brightness set at the beginning. Incidentally, a side cross-sectional view of the lead portion 21 and the soldering portion 22 of the electronic component is shown on the upper surface of the mounting board 1.

FIG. 2 is an enlarged side view of the soldering portion of the mounting board 1 shown in FIG. In the figure, 1
a, 3a, 5a, and 7a are lighting devices 11, 13, and 1 for each stage.
Incident light when switching 5 and 17 sequentially, 1b, 3b,
Similarly, 5b and 7b are reflected lights. The inclination angle of the solder noodle can be obtained from the relationship between the incident light and the reflected light.

FIG. 3 shows a plan view of the lead portion 21 and the soldering portion 22 of the electronic component. The substantially square frame 23 represented by the chain double-dashed line in the figure is the imaging camera 2 or 2 '.
The range of the window frame to be inspected for capturing images is shown. Note that FIG.
Nos. 3 to 3 are described in detail in Japanese Patent Application No. 3-81206.

FIG. 4 shows an example of the shape information code in the soldering state obtained from the image processing apparatus 3 and the grayscale gradation distribution data thereof. In the figure, for the gradation distribution data, relative values are calculated by the image processing device 3 for the photographed image obtained from the image pickup camera 2 or 2 ′ by sequentially switching the multistage lighting devices 11, 13, 15, and 17. Then, after normalization, a region of a portion having a relatively high reflectance from the solder surface is extracted, and each image data is synthesized and reconstructed by the image processing device 4. Shape code data number Nc for each pixel
And the grayscale data value M are represented by [Nc-M].

For example, [1c-numerical value] indicates a shape information code number of a region having a relatively high reflectance and its grayscale data value when it is illuminated by the illuminating device 11, and [3c
[Numerical value] indicates a shape information code number of a region having a relatively high reflectance when illuminated by the illuminating device 13 and its grayscale data value. In this way, the image processing device 4 composites and creates the respective imaging data corresponding to each illumination. Therefore, FIG. 4 shows an example of the shape information code number and the grayscale gradation distribution data viewed from the upper surface of the soldering. In [Nc-M], the shape information code number 2c is the intermediate value code number of the multi-stage multi-illuminating devices 11 and 13, 4c is the intermediate value of the multi-stage multi-illuminating devices 13 and 15, and the code number 6c is the multi-stage. The intermediate values of the multi-illumination devices 15 and 17 are obtained and set by the image processing devices 3 and 4. At this time, the grayscale gradation data value M is represented by the sum of both of the multistage multiilluminators.

FIG. 5 shows a structure of a recognition judgment device 5 for inputting the result of detailed data (that is, FIG. 4) shown from the upper surface of soldering, performing recognition judgment processing, classifying according to various soldering states and outputting. The figure is shown. In the figure, the configuration of the recognition determination device 5 is a neuro computer 32 which is a learning type hierarchical perceptron which is a kind of the neural network 31.

The input data, that is, the shape information code data and the grayscale distribution data are separately input. Note that the grayscale gradation distribution data in this case is the same as the multistage lighting device 1.
The gradation data values (corresponding to 3c or more of the shape information code data) obtained from the illumination / reflection of 3, 15, and 17 are selected and input. On the contrary, the gradation data value (shape information code data) obtained from the illumination / reflection of the multi-stage illuminators 11 and 13 from directly above, which captures the characteristics of the relatively flat portion that is not soldered. -(Corresponding to 3c or less) is selected and input.

The structure of the neural network 31 is an input layer, an intermediate layer, and an output layer. Here, the neuron unit j of each layer shown in FIG. 5 stores synaptic connections s shown in FIG. 6 with each other, that is, each synapse s stores one coefficient and stores the signal from another neuron unit j. Then, after being connected through the operation of multiplying the coefficient,
Take in Ron n. The signal output by each neuron n is determined by the level of the sum of the signals captured by the neuron unit j. The stored coefficient is the neural network.
This is the knowledge of Ku 31. Since the hardware of the neuron unit is a well-known technique, it will not be described here.

Here, back propagation is used for learning of the neural network 31. That is (details will be described later), first, detailed data (learning teaching data) is input to the input layer I, the intermediate layer H is passed, and the output layer O is answered, that is, classification by soldering state is obtained. The neural network 31 compares this answer (output) with the ideal answer (only the target output is 1, and other outputs are 0), and the synapse coupling is performed from the output layer O to the intermediate layer H and the input layer so that the difference becomes small. Go back to I and make your own changes. By repeating this learning using a large amount of data, the neural network 31 can output accurate answers to all data. That is, the network in the neural network 31 is configured so that the weighting of the connection is carried out so that the network has the optimum determination configuration.

The operation formula of the above neural network 31 will be described. First, in the forward-propagation operation, when the pattern P is input to the input layer I, each unit j of the intermediate layer H accumulates the internal potential Uj shown in the equation (1). Let W be the weight value of the connection and f be the sigmoid function. Uj = ΣWji · Ii + θj −−−−− (1) Uj: Internal potential of unit of intermediate layer H Wji: Coupling coefficient from unit of input layer I to unit of intermediate layer H Ii: Output of unit of input layer I θj : Offset coefficient of unit of intermediate layer H Hj = f (Uj) ------------- (2) Hj: Output of unit of intermediate layer H This is because the output of the unit of intermediate layer H is the input layer. It means that it is obtained by the sum of the output of the unit I and the coupling coefficient. The internal potential Uj becomes the output Hj of the unit of the intermediate layer H by the monotonically non-decreasing sigmoid function f (x), for example. Similarly, each unit k of the output layer O is also calculated by the following equation. Sk = ΣVkj · Hj + γk --- (3) Sk: Internal potential of unit of output layer O Vkj: Coupling coefficient from unit of intermediate layer H to unit of output layer O γj: Offset coefficient of unit of output layer O Ok = F (Sk) ------- (4) Ok: Output of the unit of the output layer O Next, the back propagation operation of the coupling coefficient based on the learning teaching data that is the target of the ideal answer is , Output O that output layer O answers
The correction value of the coupling coefficient is calculated so that k matches the learning teaching data Tk. As an evaluation of learning, the error function Ep
And Consider Ep = g (Tpk-Okp) --- (5). Update value ΔVkj of the coupling coefficient between the intermediate layer and the output layer
Is an effect on the error function when each coupling coefficient is slightly changed. If it is changed to a minimum value by an amount proportional to δEp / δVkj, ΔVkj = −α · δEp / δVkj −− (6) α: constant The influence of the coupling coefficient on the error function Ep is δEp / δVkj = −θk · Hj −− (7) θk: Error The new coupling coefficient Vkj is ΔVkj = α · θk · Hj −−−−− (8) Vkj = Vkj (previous time) + ΔVkj-(9) Similarly, the updated value ΔWji of the coupling coefficient between the input layer and the intermediate layer
Is an effect on the error function when each coupling coefficient is slightly changed. If changed to the minimum value by an amount proportional to δEp / δWji, ΔWji = −β · δEp / δWji −− (10) β: constant The influence of the coupling coefficient on the error function Ep is δEp / δWji = −ξj · Ii −− (11) ξj: Error The new coupling coefficient Wij is ΔWji = β · ξj · Ii −−−−− (12) Wji = Wji (previous time) + ΔWji- (13). This is a learning method in which the error is brought closer to 0 by repeating the operation of updating the weight value of the connection.

In the embodiment of the present invention, as shown in FIG. 5, as a concrete configuration example of the neural network 31, the input layers I, I ', the intermediate layer H, and the double intermediate layer are provided.
A neuro computer 32 with three layers of H'and output layer O.
Are configured. Further, a group of the input layer I and the intermediate layer H is connected to the shape information code data side. Input layer I ′,
The other group of the intermediate layer H'is connected to the grayscale gradation distribution data side.

The input layer I of the neurocomputer 32,
I'corresponds to the detailed data state diagram shown in FIG. 4, and corresponds to the pixel (after high-redundancy, if necessary, at a constant interval) fresh data m. × horizontal data n = m
n neuron units j are provided respectively. (FIG. 6 shows the configuration of one neuron unit) Data from the image processing device 4 is input to the input layers I and I '.

The intermediate layers H and H'corresponding to the input layers I and I'are experimentally provided with neuron units j of M and N.

The output layer O is provided with a neuron unit j of the detailed classification S of the recognition judgment category result. For example, as a preset recognition determination category, each soldering state is classified into: a: good, b: insufficient, c: excessive, d: poor wetting, e: none, f: lead floating, g: lead. Misalignment,
h: Classified as abnormal.

First, in the learning stage, the coefficient (knowledge)
Will be corrected to an appropriate value. First, in order to automatically classify the soldering state, shape information code data for each shape inclination angle, which is a characteristic parameter of the soldering surface whose detailed classification is known in advance, is input layer I and gray scale. The tone distribution data is input to the input layer I ', and is the vertical data of the pixels n * side data m = n.
It is input to the m-unit unit j. The data in the soldering state at this time includes shape data such as the actual area of soldering, steepness, flatness, and symmetry with respect to the lead position. For learning, prepare data that is an example of a model of learning teaching for each classification and correct output values, input the examples to input layers I and I ′, obtain intermediate layers H and H ′, and output layer O. So that the difference (error) between the output of
The coefficient stored in each synapse is updated. With respect to the intermediate layers H and H ′ that are not output to the outside, a direct error with respect to the model cannot be measured. Therefore, the error that appears in the output layer O is back-propagated to correct the error. Specifically, the error of the neuron unit j of the subsequent layer is multiplied by the coefficient of the synapse s, and the coefficient stored in the neuron unit n of the preceding layer H connected to the synapse is updated and learned. . That is, for example, the learning teaching data
In the learning when the appearance state is “a: good”, the output data of the neuron unit a, which is the ideal answer of the output layer O, is set to “1.000”, and the neuron unit b and the following ones are output. The other output data is set to "0.000" and the input layer I,
The shape information code data (1) which is the input data of the appearance state is used as the data of a number of non-defective samples corresponding to the learning teaching data in the input layer mn of the neuron unit j of I '. ~
8) and the grayscale distribution data (1 to 256) are input, the intermediate layers H and H'are obtained, and the output of the inspection response output by the output layer O (not necessarily "1.000" and "0.000"). Not 0
The coefficient stored in each synapse is updated and learned so that the difference (error) between the correct output value and the correct output value becomes small (≈0). Similarly, a large number of learning teaching data is input to other recognition determination categories, the corresponding output value is set to 1, and the other output is set to 0, and learning is successively performed.

After that, if an image of an arbitrary soldering state is picked up, image processing is performed, recognition judgment is performed, the result of the soldering state is extracted, and a clear inspection response is given. According to the above embodiment, without using a simple configuration and high-grade software,
When necessary, the level of the recognition determination category can be learned and the optimum inspection standard can be set by determining the soldering state only by inputting the detailed data state diagram, that is, the image distribution data. When used, it can be extremely rapidly and accurately judged. It is easy to change the soldering condition by learning.

In the embodiment, as the grayscale distribution data, the data for each illuminating device is used for each pixel in correspondence with the shape information code data. Accordingly, the grayscale distribution data may be normalized for each pixel by using the sum of the grayscale distribution data for each lighting device that captures the shape. If necessary, for each pixel, the grayscale distribution data may be a single grayscale distribution data obtained by an illumination device that captures a part of the shape, or a grayscale distribution data obtained by the illumination device. The data may be normalized by using some sums.

Further, in the embodiment, the explanation was made by using the illumination type, but
The ray may be used. In this case, for example, a laser light source is placed at the position of the image pickup camera shown in FIG. 1 and an annular sensor is provided at each position of the multistage illuminating device to input the image distribution data of angle or height. It is possible to obtain the effect of.

Furthermore, the multi-stage lighting device can accelerate the extraction of the image distribution data by emitting a unique color corresponding to each stage, that is, each irradiation angle of light. For example, when the illuminating means has a plurality of hues such as red, blue, and green, the irradiation angle can be specified by the difference or change in the hue of the reflected light. Therefore, the light source is switched (in the case of a multi-stage light source) or moved with time. It is not necessary to change the irradiation angle by moving the light source).

Further, as shown in FIG. 7, a fuzzy processing unit 33 is combined with the neural network 31 of the recognition determining unit 5 to form a neuro-fuzzy computer 34, and a three-dimensional shape is obtained. Fuzzy recognition judgment inspection of the appearance condition of the inspection target product. That is, the neuron unit j of the detailed classification S of the recognition judgment category result of the output layer O of the neural network 31 is finely provided, and the fuzzy processing unit 33 is connected to this output layer O. In this case, the output layer O has various recognition states as preset recognition classification states: a: good, b: insufficient, c: excessive, d: poor wetting, e: none, f: lead floating. , G: lead deviation, h: abnormal, etc.
More good: a ', less good: a ", more lacking:
b ', less shortage: b ", more excess: c', less excess: c", more wetting failure: d ', less wetting failure: d ", more lead floating (non-contact floating) ): F ', less lead float (joint float): f ", more lead shift: g', less lead shift: g", etc. Additional detailed classification S classification. The output data is fuzzy processed based on the membership function provided in the fuzzy processing unit 33, and the final recognition recognition category is set as
Various soldering status, good, shortage, excess, poor wetting,
It is judged as none, lead floating, lead deviation, abnormality, or the like. Here, as a specific example, as the fuzzy output data of the output layer O, the value of the non-defective product is rather good: a '(0.03
0), good: a (0.040), a little good: a ″ (0.
700), the value of the deficit is a large deficit: b '(0.20
0), shortage: b (0.020), small shortage: b "
(0.010) etc., the fuzzy processing device 33
Is determined to be non-defective.

Further, as shown in FIG. 8, a fuzzy processing function is additionally provided in the neural network 31 of the recognition / judgment device 5 to provide a neuro-fuzzy computer.
With the configuration of the computer 35, the appearance state of the inspection object having a three-dimensional shape is accurately inspected by the fuzzy recognition judgment. That is, the output layer O of the neural network 31 is replaced with the intermediate layer H ″, another output layer O is provided, and the input layer I, the input layer I ′, the intermediate layer H, the intermediate layer H ′, An intermediate layer H ″ and an output layer O are provided. Here, the neuron unit j of the intermediate layer H ″ is provided in more detail with the neuron unit j of the detailed classification S of the recognition determination category result. For example, as the preset recognition determination category, for each soldering state, a: good,
b: insufficient, c: excessive, d: poor wetting, e: none, f: lead floating, g: lead deviation, h: abnormal, and the like: more good: a ', less good: a ", a little lack: b ',
Less insufficient: b ", more excessive: c ', less excessive: c", more wetting failure: d', less wetting failure:
d ", more lead float: f ', less lead float:
f ", a large amount of lead shift: g ', a small amount of lead shift:
Further detailed classification S is classified into g "and the like. Here, the combination of the intermediate layer H" and the output layer O is fuzzy combined, and the recognition judgment category results of the output layer O are a: good, b: insufficient, c. : Excessive, d: Wetting failure, e: None, f: Lead floating, g: Lead deviation, h: Abnormal, etc., and the data of the intermediate layer H ″ and the output layer O are fuzzy processed Specifically, the values of non-defective products of the intermediate layer H ″ are, as fuzzy data examples, a large number of good: a ′ (0.100), a good: a (0.200), and a small number of good: The value of the shortage of the a ″ (0.500) intermediate layer H ″ is a fuzzy data and a large shortage: b ′ (0.15
0), shortage: b (0.050), small shortage: b "
(0.000) or the like, and the recognition determination category result of the output layer O processed fuzzy is a: good.

As described above, according to the present invention, the appearance state of a three-dimensional inspection target product is illuminated by a multi-stage lighting device, each image is picked up by a sensor such as an image pickup camera, and the image processing device is provided. In the composition, the necessary image distribution data is divided into shape information code data and grayscale distribution data, and in the recognition judging device, a learning type hierarchical pattern which is a kind of neural network. Input layer I,
A group of the intermediate layer H is connected to the shape information code data side,
The other group of the input layer I'and the intermediate layer H'is the grayscale gradation distribution data.
A neuro computer that is connected to the output side and is output to the output layer O is configured, and the appearance state of the inspection object having a three-dimensional shape is quickly and accurately recognized and inspected. Further, a fuzzy processing device is combined with the neural network of the recognition determination device used in the present invention to form a neuro-fuzzy computer, and the appearance state of the inspection object having a three-dimensional shape is accurately fuzzy. Check the recognition judgment. Furthermore, a fuzzy processing function is also configured in the neural network of the recognition determination apparatus used in the present invention to form a neuro-fuzzy computer, and the appearance state of the inspection object having a three-dimensional shape is accurately determined. Inspect fuzzy recognition judgment.

[Brief description of drawings]

FIG. 1 is a schematic configuration diagram of a main part of a visual inspection device in a soldered state which is an inspection target product according to the present invention.

FIG. 2 is an enlarged side sectional view of a soldering portion.

FIG. 3 is a plan view of a lead portion and a soldering portion of an electronic component.

FIG. 4 is a solder shape information code data according to the present invention;
The detailed data state diagram showing the gradation data.

FIG. 5 is a block configuration diagram of a recognition determination device used in the present invention.

FIG. 6 is an explanatory diagram of a neural unit according to an embodiment of the present invention.

FIG. 7 is a simplified configuration diagram of a main part showing another embodiment of the present invention.

FIG. 8 is a simplified configuration diagram of a main part showing another embodiment of the present invention.

[Explanation of symbols]

 1 Mounting board 11, 13, 15, 17 Illumination device 1a, 3a, 5a, 7a Incident light 1b, 3b, 5b, 7b Reflected light 2 Imaging camera 3,4 Image processing device 5 Recognition determination device I, I'input layer H , H'Intermediate layer O Output layer

Claims (5)

[Claims] 1. Shape information composed of image array data of gray scale distribution obtained by irradiating an inspection object with a multi-stage multi-illuminating device and capturing respective images by an image pickup means. The code data pattern and the grayscale distribution data pattern are input to a neuro computer means in which a learning type hierarchical perceptron is formed by an input layer, an intermediate layer and an output layer. An appearance state inspection method characterized by outputting the appearance state of the inspection target product to an output layer of the means to perform a recognition determination inspection. 2. A shape information code formed by scanning and irradiating an object to be inspected with a laser beam, receiving a reflected light by a sensor, and image arrangement data of a necessary extracted image distribution by an image processing device. A neurocomputer in which a data pattern and a grayscale distribution data pattern are created, and these two data patterns form a learning type hierarchical perceptron by an input layer, an intermediate layer, and an output layer. -Inputting to the output means of the means, and outputting the appearance state of the inspection object product to the output layer of the means,
Appearance state inspection method characterized by performing recognition judgment inspection 3. The appearance state inspection method according to claim 1, wherein the respective stages of the multi-stage illumination means have different hues, and the image pickup means is a color camera. 4. The appearance state inspection method according to claim 1 or 2, further comprising fuzzy processing means in an output layer of the recognition determining means. 5. The appearance state inspection method according to claim 1 or 2, further comprising fuzzy processing means in an intermediate layer of the recognition determining means.