JP2013065809A - Inspection device, inspection method, and inspection program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To properly determine the quality of solder mounting without setting an inspection parameter for each insection object.SOLUTION: An inspection device includes: an extraction part which extracts multiple partial images including solder joining parts from an image including the multiple solder joining parts; a creation part which creates a reference image serving as reference for determining the quality of the mounting of the solder joining part using the multiple partial images extracted by the extraction part; a comparison part for comparing each partial image with the reference image created by the creation part; and a determination part for determining the quality of the mounting of each solder joining part on the basis of the comparison result of the comparison part.

Description

  The present invention relates to an inspection apparatus, an inspection method, and an inspection program for inspecting the quality of soldering mounting.

  In mounting inspection of a printed circuit board, currently, after performing an automatic inspection, a person is visually inspecting an NG candidate. This is because, in terms of overall accuracy and reliability, the performance of the human eye still exceeds automatic inspection.

  However, in the visual inspection by humans, there are problems such as oversight, variations in inspection standards, and labor costs, and further automation is required.

  In the mounting inspection of the printed circuit board, it is necessary to inspect whether the soldering state is normal or defective as well as the mounting position and orientation of the components. If the solder joint state is poor, it may appear as a product defect later. The solder bonding state can be determined by the appearance based on the shape and wetness of the solder fillet.

  Among printed circuit board components, multi-pin connectors are highly dependent on visual inspection. This is because the number of leads is narrow and the number of points to be inspected is concentrated. In addition, the shape of the lead is various, it is difficult to apply a specific inspection standard, and individual teaching is required.

  Therefore, it is particularly required to increase the accuracy of such automatic inspection of connectors and reduce the amount of visual inspection work without increasing the teaching effort.

  The appearance inspection of the solder joint portion is a particularly difficult inspection among the mounting inspections of the printed circuit board because the surface of the solder fillet having a complicated three-dimensional shape reflects the metal and appears on the image. A method of avoiding the problem of gloss by using structured illumination is also used, but only the three-dimensional shape of the portion irradiated with the structure pattern is known, and the apparatus becomes complicated and the cost is likely to increase.

  Here, as a conventional technique for inspecting the soldering state, there is a color highlight method. Regarding this color highlight method, for example, there is a technique of automatically generating a color condition, which is one of inspection parameters, using a non-defective product image with a normal component mounting position and a defective product image with a non-normal mounting position.

  Further, as another inspection technique, among a plurality of soldering portions having the same pattern, information indicating the shape of a certain soldering portion is compared with information indicating the shape of another soldering portion. There is a technique for judging pass / fail.

JP 2006-86451 A JP 2008-191106 A

  However, in the conventional color light system, it is necessary to prepare a DB (Data Base) for storing images of mounted parts of non-defective products and defective products. Therefore, if new parts increase, it is necessary to prepare a DB for the parts, and the inspection parameters must be changed for each part to be inspected. In other inspection techniques, it can not be said that the quality of soldering mounting can be properly judged because the quality judgment largely depends on the shape of a predetermined soldering portion as a basis for comparison.

  Therefore, there is a need for a technique that can appropriately determine the quality of soldering mounting without setting inspection parameters for each inspection target.

  Accordingly, the disclosed technique has an object to provide an inspection apparatus, an inspection method, and an inspection program that can appropriately determine whether soldering is mounted or not without setting inspection parameters for each inspection object. .

  The inspection apparatus according to an aspect of the disclosure uses an extraction unit that extracts a plurality of partial images including the solder joints from an image including a plurality of solder joints, and a plurality of partial images extracted by the extraction unit, A creation unit that creates a reference image that serves as a reference for determining whether or not the solder joint is mounted, a comparison unit that compares the reference image created by the creation unit with each partial image, and a comparison result by the comparison unit And a determination unit that determines whether each solder joint is mounted or not.

  According to the disclosed technique, it is not necessary to set an inspection parameter for each inspection target, and it is possible to appropriately determine whether soldering is mounted or not.

FIG. 2 is a block diagram illustrating an example of hardware of the inspection apparatus according to the first embodiment. FIG. 2 is a block diagram illustrating an example of functions of the inspection apparatus according to the first embodiment. The figure which shows an example of a test object. The figure for demonstrating the process outline | summary of a quality determination. The figure for demonstrating the detection of a lead space | interval. The figure which shows the histogram of a brightness | luminance difference. The figure for demonstrating extraction of a partial image. The figure for demonstrating the production method of a reference | standard image. The figure which shows the correlation value of a reference | standard image and each partial image. 3 is a flowchart illustrating an example of inspection processing according to the first embodiment. 5 is a flowchart illustrating an example of partial image extraction processing according to the first exemplary embodiment. 5 is a flowchart illustrating an example of a reference image creation process (method 1) according to the first exemplary embodiment. 6 is a flowchart illustrating an example of a reference image creation process (method 2) according to the first exemplary embodiment. 9 is a flowchart illustrating an example of a reference image creation process (method 3) according to the first exemplary embodiment. FIG. 9 is a block diagram illustrating an example of hardware of an inspection apparatus according to a second embodiment. FIG. 9 is a block diagram illustrating an example of functions of an inspection apparatus according to a second embodiment. The figure which shows an example of the positional relationship of an imaging part and lighting equipment. The figure which shows an example of each reference | standard image produced | generated from each test | inspection image. 10 is a flowchart illustrating an example of inspection processing according to the second embodiment. FIG. 9 is a block diagram illustrating an example of functions of an inspection apparatus according to a third embodiment. The figure for demonstrating parallax detection based on a stereo matching. The figure for demonstrating the measurement of the lead height based on parallax. The figure for demonstrating the quality determination of solder. The figure for demonstrating the quality determination based on the height of a lead. 10 is a flowchart illustrating an example of inspection processing according to the third embodiment. The figure for demonstrating the process outline | summary in the case of partial image area sharing (the 1). The figure for demonstrating the process outline | summary in the case of partial image area sharing (the 2). FIG. 10 is a block diagram illustrating an example of functions of an extraction unit according to a fourth embodiment. The figure for demonstrating the detection of a common lead space | interval. The figure for demonstrating the processing efficiency improvement of a read space | interval. The figure which shows an example of the test | inspection image containing a metal fitting part. The figure for detecting the perpendicular range of a common partial image area. 10 is a flowchart illustrating an example of inspection processing according to the fourth embodiment. 10 is a flowchart illustrating an example of common partial image region detection processing according to the fourth embodiment.

  Each embodiment will be described below with reference to the accompanying drawings.

[Example 1]
<Hardware>
FIG. 1 is a block diagram illustrating an example of hardware of the inspection apparatus 10 according to the first embodiment. In the example illustrated in FIG. 1, the inspection apparatus 10 is connected to the illumination device 11, the imaging unit 12, and the display unit 13.

  The illumination device 11 is a device that has, for example, an issuing element and illuminates the inspection target. The imaging unit 12 is a device having an imaging element such as a camera. The imaging unit 12 captures an inspection target and generates, for example, a monochrome image. The display unit 12 is, for example, a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), or the like, and displays the inspection result acquired from the inspection apparatus 10.

  An inspection apparatus 10 illustrated in FIG. 1 includes a control unit 101, a main storage unit 102, an auxiliary storage unit 103, a communication unit 104, and a drive device 105. Each unit is connected via a bus so that data can be transmitted / received to / from each other.

  The control unit 101 is a CPU (Central Processing Unit) that performs control of each device, calculation of data, and processing in a computer. The control unit 101 is an arithmetic device that executes a program stored in the main storage unit 102 or the auxiliary storage unit 103. The control unit 101 receives data from the input device or the storage device, calculates, processes the output device, the storage device, and the like. Output to the device.

  The main storage unit 102 is, for example, a ROM (Read Only Memory) or a RAM (Random Access Memory). The main storage unit 102 is a storage device that stores or temporarily stores programs and data such as an OS and application software that are basic software executed by the control unit 101.

  The auxiliary storage unit 103 is, for example, an HDD (Hard Disk Drive) or the like, and is a storage device that stores data related to application software. For example, the auxiliary storage unit 103 stores an image acquired from the imaging unit 12.

  The communication unit 104 performs data communication with peripheral devices by wire or wireless. The communication unit 104 acquires an image including characters via a network, for example, and stores it in the auxiliary storage unit 103.

  The drive device 105 reads a predetermined program from the recording medium 106, for example, a flexible disk or a CD (Compact Disc), and installs it in the storage device.

  A predetermined program is stored in the recording medium 106, and the program stored in the recording medium 106 is installed in the inspection apparatus 10 via the drive device 105. The installed predetermined program can be executed by the inspection apparatus 10.

  In addition, although the inspection apparatus 10 illustrated in FIG. 1 has the illumination device 11, the imaging unit 12, and the display unit 13 as separate configurations, the inspection device 10 may include any one or a plurality of configurations.

<Function>
FIG. 2 is a block diagram illustrating an example of functions of the inspection apparatus 10 according to the first embodiment. In the example illustrated in FIG. 2, the inspection apparatus 10 includes an acquisition unit 201, an image processing unit 202, and a determination unit 203. The acquisition unit 201, the image processing unit 202, and the determination unit 203 can be realized by, for example, the control unit 101 and the main storage unit 102 as a work memory.

  The acquisition unit 201 acquires an image including a plurality of solder joints to be inspected from the imaging unit 12. Hereinafter, this image is referred to as an inspection image. The solder joint portion is a portion including a solder fillet formed by soldering a lead. The acquisition unit 201 outputs the acquired inspection image to the image processing unit 202.

  The image processing unit 202 performs image processing on the inspection image input from the acquisition unit 201 to generate and generate an image used to determine whether the soldering state or shape of each solder joint is good or not A determination parameter is calculated from the image. Hereinafter, the quality of the soldering state or shape is also referred to as the quality of soldering mounting. The determination parameter is a parameter used to determine whether the solder joint is mounted or not, and is, for example, a correlation value between images.

  The image processing unit 202 includes an extraction unit 211, a creation unit 212, a storage unit 213, and a comparison unit 214 in order to calculate determination parameters.

  The extraction unit 211 detects the interval between the leads included in the inspection image, and extracts a partial image including the solder joint based on the lead interval. One partial image is an image including one solder joint. The extraction unit 211 outputs the extracted partial images to the creation unit 212. Further, the extraction unit 211 exchanges data for extracting partial images with the storage unit 213. Specific extraction processing of the partial image will be described later.

  The creation unit 212 creates a reference image to be used for soldering mounting quality determination using a plurality of partial images acquired from the extraction unit 211. The reference image is, for example, an average image of a plurality of partial images. The creation unit 212 stores the created reference image in the storage unit 213 and notifies the comparison unit 214 of creation completion. Specific processing for creating the reference image will be described later.

  The storage unit 213 stores the reference image created by the creation unit 212. The reference image is written by the creation unit 212 and read by the comparison unit 214.

  Upon receiving the reference image creation completion notification from the creation unit 212, the comparison unit 214 reads the reference image from the storage unit 213. The comparison unit 214 compares the acquired reference image with each partial image included in the inspection image.

  The comparison unit 214 includes a calculation unit 221. The calculation unit 221 calculates a correlation value between the reference image and the partial image of the inspection image. The calculation unit 214 outputs this correlation value to the determination unit 203 as a comparison result. The calculation unit 221 may calculate the similarity between images and use the similarity as a comparison result.

  When the determination unit 203 obtains a comparison result between the reference image and the partial image in the inspection image from the image processing unit 202, the determination unit 203 determines whether the solder joint portion in the partial image is mounted. The determination unit 203 determines pass / fail for each partial image in the inspection image, and determines that the mounting is defective when it is determined that even one is defective.

  For example, when the correlation value that is the acquired determination result is smaller than the threshold value, the determination unit 203 determines that it is defective. The threshold value is set to 0.85, for example, and an appropriate value may be set as the threshold value through experiments or the like. The determination unit 203 outputs the determination result to the display unit 13. The display unit 13 displays the determination result.

<Inspection target>
FIG. 3 is a diagram illustrating an example of an inspection target. In the example shown in FIG. 3, the semiconductor integrated circuit 20 is mounted on the surface of the substrate 23. The mounted semiconductor integrated circuit 20 is covered with a resin, for example, and includes a plurality of leads 21. Each lead is soldered to a land formed on the surface of the substrate 23. As a result, a solder joint portion 22 is formed that indicates a portion where the lead and the land are joined by solder. The solder joint portion 22 has, for example, a solder fillet.

  The imaging unit 12 images a plurality of leads and solder joints as shown in FIG. 3 from above, and generates an inspection image.

  Note that the inspection object shown in FIG. 3 is an example, and the present embodiment is applicable to an inspection object having a plurality of solder joints, for example, for appearance inspection of a multi-pin connector surface-mounted on a substrate. Applicable.

<Process overview>
FIG. 4 is a diagram for explaining the outline of the pass / fail judgment process. In the example illustrated in FIG. 4, the inspection image 30 is an image including an inspection object imaged by the imaging unit 12. The partial image 31 is an image of a rectangular area including the lead 21 and the solder joint portion 22. The reference image 32 is an image that is created using a plurality of partial images and serves as a reference for determining whether soldering is mounted or not.

As shown in FIG. 4, the quality determination of the soldering mounting is performed according to the following procedure.
(1) Lead interval detection (2) Partial image extraction (3) Reference image creation (4) Pass / fail judgment based on the comparison result between the reference image and the partial image Hereinafter, each process (1) to (4) will be specifically described. To do.

(1) Lead interval detection Since the solder fillet portion and the lead surface on which the solder is placed have slightly different surface shapes and cause metal reflection, the brightness distribution appears slightly different from each other. Therefore, the extraction unit 211 detects a lead interval that can be obtained stably. In the following description, it is assumed that the leads are arranged in the horizontal direction, but even if the leads are arranged in the vertical direction, the same processing may be performed by reversing the horizontal and vertical directions.

  Since the lead portion is bright and the other portions appear dark, the extraction unit 211 searches for a portion where bright regions are arranged horizontally at equal intervals, extracts the lead region, and determines the repetition cycle of the bright region as the lead interval. Asking.

  Considering at the pixel level, the interval between bright pixels or the interval between dark pixels corresponds to the read interval, and therefore the read interval is determined depending on how far the pixels having the same luminance are separated. However, since the brightness of the solder portion varies widely and there is a lot of noise when viewed from each pixel, the extraction unit 211 can accurately detect the brightness difference between pixels at various intervals at various positions. Find the correct lead interval.

FIG. 5 is a diagram for explaining the detection of the lead interval. As illustrated in FIG. 5, the extraction unit 211 takes two pixels on a horizontal line, and obtains an interval (number of pixels) d and a luminance difference ΔV. For example, the luminance value at the position i in the horizontal direction is Vi, and the luminance value at the position j is Vj. At this time, the luminance difference ΔV is obtained by the following equation (1).
ΔV = | Vi−Vj | Equation (1)
Regardless of the position of the pixel, if the distance d is close to the lead interval, the luminance difference ΔV becomes almost zero, and if d is shifted by a half cycle from the lead interval, the luminance difference becomes the largest. Using this, the extraction unit 211 changes the pixel pairs in various ways, that is, changes i and j, and accumulates the luminance difference ΔV for each interval d.

  FIG. 6 is a diagram showing a histogram of luminance differences. The graph shown in FIG. 6 shows an example of a luminance histogram when the extraction unit 211 accumulates the luminance difference ΔV for each interval d. The minimum value appearing first in the luminance histogram corresponds to the lead interval.

  If the range of the interval d is widened, a minimum value (d = m2, m3) corresponding to twice or three times the lead interval as the lead period appears. For example, by dividing m2 corresponding to a double read interval by 2, the extraction unit 211 can obtain an accurate read interval with subpixel accuracy. For example, when m2 is 31 pixels, m2 / 2 = 15.5, and the read interval can be expressed by a pixel in decimal units.

  Note that the extraction unit 211 determines that there is no connector lead on a horizontal line where such a maximum value and minimum value do not appear in the histogram as shown in FIG. When the position of the lead is unknown, the extraction unit 211 sets several such horizontal lines and finds a position where the lead interval can be calculated correctly.

(2) Partial Image Extraction The extraction unit 211 acquires the lead interval obtained in the process (1), and uses this lead interval to extract a rectangular region including a bright lead region as a partial image. The rectangular area includes a part of the lead and a solder joint.

  FIG. 7 is a diagram for explaining extraction of partial images. FIG. 7A shows an example of an inspection image. In the example illustrated in FIG. 7, the extraction unit 211 performs luminance integration on the inspection image in the horizontal direction and the vertical direction based on the horizontal line from which the lead interval is obtained and the lead interval. As a result, the integral value increases in the portion where the lead exists.

  However, the integration result in the vertical direction includes irregularities due to noise, and even if smoothing is performed, the peaks are not necessarily arranged at equal intervals. However, even if the peak positions vary, it can be expected that if all the leads are added together, the peak will be at the center of the lead.

  FIG. 7B shows an integrated value obtained by integrating the luminance of the inspection image in the vertical direction. The extraction unit 211 adds the integral values shifted in the horizontal direction by the distance of the lead interval m from the predetermined start point X0 shown in FIG. The extraction unit 211 performs this addition by shifting one pixel at a time from X0 to m. FIG. 7C is a graph showing the cumulative integral value for each interval m. FIG. 7C is also a graph in which integrated values cut out from the starting point X0 at intervals m are superimposed. A position where the peak value (maximum value) in the graph shown in FIG.

  The extraction unit 211 determines the position of X0max as the center position of the lead. When the center position of the lead is determined, the extraction unit 211 sets the predetermined range t on the left and right sides as the horizontal width of the partial image. FIG. 7D shows a predetermined range t of the partial image in the horizontal direction. For example, the predetermined range t may be set to a predetermined length from the peak position to the left and right. The predetermined range t may be set based on the length between the peak value and minimum value of the luminance integrated value in the vertical direction.

  FIG. 7E shows an integrated value obtained by integrating the luminance of the inspection image in the horizontal direction. For example, the extraction unit 211 determines a predetermined range s in the vertical direction by shifting a portion where the integral value exceeds a threshold value by a predetermined value in the downward direction of the image with the solder joint.

  The extraction unit 211 extracts a plurality of partial images of the rectangular area based on the predetermined range t in the horizontal direction and the predetermined range s in the vertical direction. For example, the extraction unit 211 extracts a plurality of partial images that are separated from the center position of the lead by a lead interval m.

(3) Creation of Reference Image The creation unit 212 creates a reference image that serves as a reference for determining whether soldering is mounted or not using a plurality of partial images acquired from the extraction unit 211. Here, three creation methods will be described.

  Note that, when the creation unit 212 includes a partial image that is suspected of being defective when creating the reference image, the quality of the reference image is degraded. Accordingly, the creation unit 212 creates a reference image by excluding partial images that are suspected of being defective. A partial image suspected of being defective may be detected using a correlation value between the partial images. The creation unit 212 creates a reference image by, for example, averaging the partial images at the positions where template matching or pattern matching is performed. The reference image becomes a reference template in the template matching for quality determination described later.

(Method 1)
The creation unit 212 performs template matching processing with another partial image on the basis of a specific one of the partial images (here, the left end), and obtains the position and the correlation value. Processing time is the shortest of the three methods. If only the reference partial image is defective, the creation unit 212 can determine that the reference partial image is defective because the correlation values with other partial images are all low.

  As a specific process, the creation unit 212 examines the correlation value of each partial image with the leftmost partial image, and if it is equal to or greater than a predetermined threshold, takes an average as the reference image. At this time, the creation unit 212 does not add a correlation value equal to or lower than a predetermined threshold to the reference image. The predetermined threshold is, for example, 0.85, but an appropriate value may be set by experiment or the like.

  This is because a partial image that is defective or suspected of being defective is not mixed with the reference image. The creation unit 212 also counts the number of partial images averaged as a reference image. If the number is equal to or less than a predetermined number, the creation unit 212 considers that there are some partial images that are defective or close to defective, and determines that the quality is NG To do. The predetermined number is, for example, half the value of the partial image, but may be other values.

  Even when only the lead at the left end is defective, the count number is 1 and it is determined as NG. The creation unit 212 may perform the above-described processing with another partial image as a reference when the pass / fail determination is NG.

(Method 2)
The creation unit 212 performs template matching processing on all partial images with the adjacent partial images. The creation unit 212 performs template matching processing at both ends of the partial images at both ends. As a result, two correlation values are obtained for all the leads.

  The creation unit 212 can determine that a partial image whose both correlation values are lower than a predetermined threshold is close to a defect, and can exclude the partial image from creation of the reference image. Thereby, Method 2 can improve the accuracy of the reference image more than Method 1.

(Method 3)
The creation unit 212 performs template matching processing for all the partial images as a round robin, assuming that the method 2 is expanded to the maximum. As a result, the creation unit 212 determines that a partial image having a correlation value lower than a predetermined threshold is defective and excludes it, and creates a reference image by taking the average of the remaining partial images. Method 3 can create a more accurate reference image although the processing time increases.

  FIG. 8 is a diagram for explaining a method of creating a reference image. FIG. 8A is a diagram for explaining the method 1. In Method 1, as shown in FIG. 8A, the creation unit 212 can determine that the correlation value with the sixth partial image is low by performing template matching processing with the leftmost partial image. Therefore, the creation unit 212 creates, as a reference image, an average image obtained by averaging the first to fifth and seventh to eighth partial images at the positions matched by template matching.

  FIG. 8B is a diagram for explaining the method 2. As shown in FIG. 8B, the creation unit 212 performs template matching processing on the adjacent partial images, and excludes the sixth partial image whose two correlation values are lower than a predetermined threshold. Therefore, the creation unit 212 creates, as a reference image, an average image obtained by averaging the first to fifth and seventh to eighth partial images at the positions matched by template matching.

  FIG. 8C is a diagram for explaining the method 3. As shown in FIG. 8C, the creation unit 212 performs a template matching process between all the partial images, and creates a matrix of correlation values. At this time, the creation unit 212 excludes the sixth partial image whose average correlation value is lower than a predetermined threshold. Therefore, the creation unit 212 creates, as a reference image, an average image obtained by averaging the first to fifth and seventh to eighth partial images at the positions matched by template matching.

  The creation unit 212 may create a reference image using any one of methods 1 to 3. Any one of the methods 1 to 3 may be set in advance, or may be made to be selected by the user every time of creation.

  Note that when creating the reference image, the creating unit 212 may perform weighting based on the difference in the correlation value and add the partial images to generate an average image.

(4) Pass / Fail Judgment Based on Comparison Result between Reference Image and Partial Image The comparison unit 214 performs template matching processing with each partial image using the reference image created by the creation unit 212, and a correlation value with each partial image. Ask for. The comparison unit 214 outputs the calculated correlation value to the determination unit 203.

  The determination unit 203 determines that the partial image having a low correlation value indicates that the solder joint portion is defective due to a large difference from the solder joint portion of the reference image.

  FIG. 9 is a diagram illustrating a correlation value between the reference image and each partial image. As shown in FIG. 9, the correlation value of the sixth partial image is smaller than the threshold value. Therefore, the determination unit 203 determines that the solder joint included in the sixth partial image is defective. This threshold value is set to 0.9, for example, but an appropriate value may be set by an experiment or the like.

  Here, the reason for performing the pass / fail judgment based on the average image is as follows. Since partial images including individual leads vary, the correlation values of individual pairs are not necessarily high even if they are non-defective products, and the variation is also large. For this reason, it is difficult to separate the non-defective product from the non-defective product.

  However, by creating an average image of a plurality of partial images, variations are offset and a partial image close to a true good image (ideal good image) is obtained. Therefore, if the correlation value with each partial image is calculated using the average image as a reference image, it is possible to appropriately determine whether each soldering is mounted.

  Accordingly, it is only necessary to set a threshold value in advance, so that parameter setting for each inspection object is not required, the teaching process is greatly simplified, and the quality determination of soldering mounting can be appropriately performed.

<Operation>
Next, the operation of the inspection apparatus 10 in the first embodiment will be described. FIG. 10 is a flowchart illustrating an example of the inspection process according to the first embodiment. In step S101 illustrated in FIG. 10, the imaging unit 12 captures an inspection target and generates, for example, a monochrome captured image. This captured image is acquired as an inspection image by the acquisition unit 201 and input to the extraction unit 211.

  In step S102, the extraction unit 211 determines whether or not the partial image including the solder joint portion has been successfully extracted from the inspection image. Details of the extraction process will be described with reference to FIG. If the extraction is successful (step S102—YES), the process proceeds to step S103. If the extraction fails (step S102—NO), the inspection process is terminated.

  In step S103, the creation unit 212 determines whether the reference image has been successfully created. The reference image creation process will be described with reference to FIGS. If the creation of the reference image is successful (step S103—YES), the process proceeds to step 104. If the creation of the reference image fails (step S103—NO), the inspection process is terminated.

  In step S104, the comparison unit 214 compares the reference image with each partial image. For example, the comparison unit 214 performs template matching between the lead and solder joint of the reference image and the lead and solder joint of the partial image, and calculates a correlation value. The correlation value may be a pixel similarity between images.

  In step S <b> 105, the determination unit 203 determines the quality of the soldering mounting of the solder joint included in each partial image based on the comparison result of the comparison unit 214. As a result, it is possible to create an appropriate reference image and appropriately perform the pass / fail determination.

  In the case of partial image extraction failure or reference image creation failure, it is suggested that the degree of mounting abnormality is too large, so the determination unit 203 determines that the mounting is defective.

  FIG. 11 is a flowchart illustrating an example of partial image extraction processing according to the first embodiment. In step S201 illustrated in FIG. 11, the extraction unit 211 selects a horizontal line in the inspection image. The extraction unit 211 selects a predetermined line (for example, a line several lines below the center) in advance.

  In step S202, the extraction unit 211 accumulates the luminance differences between the two pixels on the selected horizontal line, and creates a luminance difference histogram as illustrated in FIG.

  In step S203, the extraction unit 211 determines whether the minimum position of the luminance difference histogram can be detected. When there is no minimum position in the luminance difference histogram, there is a high possibility that there is no lead on the horizontal line. Therefore, if there is no minimum position (step S203—NO), the process returns to step S201, and the extraction unit 211 selects another horizontal line. If there is a minimum position (step S203—YES), the process proceeds to step S204.

  In step S204, the extraction unit 211 determines whether or not the lead interval can be detected based on the minimum position of the luminance histogram. If the read interval can be detected (step S204-YES), the process proceeds to steps S205 and S209. If the read interval cannot be detected (step S204-NO), the extraction process ends.

  In step S205, the extraction unit 211 performs luminance integration in the vertical direction of the inspection image (see FIG. 7B).

  In step S206, the extraction unit 211 accumulates the luminance integration value in the vertical direction at each read interval (see FIG. 7C).

  In step S207, the extraction unit 211 detects the peak of the accumulated value of the integrated luminance value within the lead interval as the center position in the lead.

  In step S208, the extraction unit 211 detects a predetermined range t based on the center position of the lead as the horizontal range of the lead at the left end of the inspection image.

  In step S209, the extraction unit 211 performs luminance integration in the horizontal direction of the inspection image (see FIG. 7E).

  In step S210, the extraction unit 211 detects the predetermined range s as the vertical range of the lead using the luminance integrated value in the horizontal direction.

  Note that steps S205 to S208 and steps S209 to S210 are out of order, and the processes may be performed in parallel, or one of them may be performed first.

  In step S211, the extraction unit 211 determines, as a partial image, an image of a rectangular area based on the detected predetermined range t of the leftmost horizontal range and the predetermined range s of the vertical range. This partial image includes a lead and a solder joint.

  When the leftmost partial image is determined in step S212, the extraction unit 211 can determine all partial images by shifting the center position of the partial image in the horizontal direction by the lead interval, for example. Thereby, all the partial images including the lead and the solder joint portion can be extracted from the inspection image.

  FIG. 12 is a flowchart illustrating an example of a reference image creation process (method 1) according to the first embodiment. In step S301 illustrated in FIG. 12, the creation unit 212 sets the leftmost partial image as a template matching target. Let the left end be n = 1.

  In step S302, the creation unit 212 initializes the parameter count to 0. In step S303, the creation unit 212 adds 1 to n.

  In step S304, the creation unit 212 searches for the vicinity of the n-th partial image based on the template matching target partial image. In the template matching process, a position matching the correlation value is calculated.

  In step S305, the creation unit 212 acquires a correlation value between two partial images and a matched position.

  In step S306, the creation unit 212 determines whether the acquired correlation value is greater than or equal to a threshold value. The threshold is 0.85, for example. If the correlation value is greater than or equal to the threshold value (step S306-YES), the process proceeds to step S307, and if the correlation value is less than the threshold value (step S306-NO), the process returns to step S303.

  In step S307, the creation unit 212 adds the matched partial image to the average image at the matched position. The average image is an image obtained by averaging partial images having a correlation value equal to or greater than a threshold value.

  In step S308, the creation unit 212 adds 1 to count.

  In step S309, the creation unit 212 determines whether n is N. N is the total number of partial images extracted by the extraction unit 211. If n = N (step S309—YES), the process proceeds to step S310, and if not n = N (step S309—NO), the process returns to step S303.

  In step S310, the creation unit 212 determines whether count is equal to or greater than a threshold value. This threshold is, for example, N / 2. It should be noted that an appropriate value may be set for the threshold value through experiments or the like. If count is greater than or equal to the threshold (step S310—YES), the process proceeds to step S311. If count is less than the threshold (step S310—NO), the process proceeds to step S312.

  In step S311, the creation unit 212 sets an average image obtained by averaging partial images having a correlation value equal to or greater than a threshold value as a reference image. The created reference image is stored in the storage unit 213.

  In step S312, the creation unit 212 determines that the determination is NG because there are many mounting defects in the soldering. If the creation unit 212 determines that the determination is NG, the generation unit 212 notifies the determination unit 203 of the determination.

  FIG. 13 is a flowchart illustrating an example of a reference image creation process (method 2) according to the first embodiment. In step S401 illustrated in FIG. 13, the creation unit 212 sets n = 1.

  In step S402, the creation unit 212 sets the nth partial image from the left end of the reference image as a partial image to be matched.

  In step S403, the creation unit 212 performs a matching search around the (n + 1) th partial image based on the partial image to be template matched. In the template matching process, the correlation value and the matched position are calculated. When n is N, matching with the first partial image is performed.

  In step S404, the creation unit 212 acquires the correlation value between the two partial images and the matched position. In step S405, the creation unit 212 adds 1 to n.

  In step S406, the creation unit 212 determines whether n> N. N is the total number of partial images extracted by the extraction unit 211. If n> N (step S406—YES), the process proceeds to step S407, and if n ≦ N (step S406—NO), the process returns to step S402.

  In step S407, the creation unit 212 initializes count, n to 0. In step S408, the creation unit 212 adds 1 to n.

  In step S409, the creation unit 212 calculates the average of the two (n−1) th and nth correlation values. This average is called an average correlation value.

  In step S410, the creation unit 212 determines whether the average correlation value is greater than or equal to a threshold value. This threshold is, for example, 0.85. If the average correlation value is greater than or equal to the threshold (step S410—YES), the process proceeds to step S411, and if the average correlation value is less than the threshold (step S410—NO), the process returns to S408. The threshold is set to 0.85, for example, but an appropriate value may be set by experiment or the like.

  In step S411, the creation unit 212 adds the nth partial image to the average image at the matched position. In step S412, the creation unit 212 adds 1 to count.

  In step S413, the creation unit 212 determines whether n = N. If n = N (step S413-YES), the process proceeds to step S414, and if not n = N (step S413-NO), the process returns to step S408.

  Since the process after step S414 is the same as the process after step S310 shown in FIG. 12, the description is abbreviate | omitted.

  FIG. 14 is a flowchart illustrating an example of a reference image creation process (method 3) according to the first embodiment. The processes in steps S501 to S502 shown in FIG. 14 are the same as the processes in steps S401 to S402 shown in FIG.

  In step S503, the creation unit 212 searches for the vicinity of the (n + 1) th to Nth partial images based on the template matching target partial images.

  In step S504, the creation unit 212 acquires all the correlation values and the matching positions between the template matching target partial image and each of the (n + 1) th to Nth partial images.

  The processing in steps S505 to S508 is the same as the processing in steps S405 to S408 shown in FIG.

  In step S509, the creation unit 212 calculates the average of N-1 correlation values other than the nth, and sets this average as the average correlation value.

  Since the process after step S510 is the same as the process after step S410 shown in FIG. 13, the description thereof is omitted.

  As described above, according to the first embodiment, it is not necessary to set a parameter for each inspection target, and it is possible to appropriately determine whether soldering is mounted or not. Moreover, according to Example 1, even if the lighting condition changes, the quality determination process can be applied without performing teaching again. Further, according to the first embodiment, it is not necessary to perform teaching individually for each type of connector. In addition, according to the first embodiment, it can be applied to a new similar component as it is without performing teaching again.

[Example 2]
Next, the inspection apparatus in Example 2 will be described. In the second embodiment, the accuracy of the determination is improved by performing the pass / fail determination based on the inspection image acquired by changing the position of the lighting device or imaging the inspection object from a plurality of viewpoints.

<Hardware>
FIG. 15 is a block diagram illustrating an example of hardware of the inspection apparatus 10 according to the second embodiment. In the example illustrated in FIG. 15, the hardware of the inspection apparatus 10 is the same as that in the first embodiment, but there are a plurality of imaging units. In the example illustrated in FIG. 15, two imaging units are connected to the inspection apparatus 10. The first imaging unit 12-1 is an apparatus that images an inspection object from the left viewpoint, for example, and the second imaging unit 12-2 is an apparatus that images the inspection object from the right viewpoint, for example. Thereby, the inspection object can be imaged from various angles.

  Since the other configuration is the same as that of the first embodiment, the same reference numerals as those shown in FIG.

<Function>
FIG. 16 is a block diagram illustrating an example of functions of the inspection apparatus 10 according to the second embodiment. The inspection apparatus 10 illustrated in FIG. 16 includes an acquisition unit 300, a setting unit 301, an image processing unit 302, and a determination unit 303. The acquisition unit 300, the setting unit 301, the image processing unit 302, and the determination unit 303 can be realized by the control unit 101 and the main storage unit 102 as a work memory, for example.

  The acquisition unit 300 outputs the captured images acquired from the first imaging unit 12-1 and the second imaging unit 12-2 to the extraction unit 311. This captured image is treated as an inspection image.

  The setting unit 301 sets the position of the lighting device 11. For example, the position of the illumination device 11 is adjusted so that the illumination device 11 comes to the upper left, right above, and upper right of the inspection object. The lighting device 11 is movably controlled by the setting unit 301.

  In Example 2, the position of the illumination device 11 is not adjusted, but a plurality of illumination devices 11 are provided, and each illumination device is illuminated in order to illuminate the inspection object from each direction. Also good. Further, the setting unit 301 may change the illumination direction instead of changing the illumination position of the illumination device 11.

  The image processing unit 302 determines whether each solder joint is mounted or not by using inspection images from each illumination direction and each viewpoint for one inspection object.

  The functions of the extraction unit 311, the creation unit 312, the storage unit 313, and the comparison unit 314 of the image processing unit 302 are basically the same as those in the first embodiment. The difference from the first embodiment is that processing is performed on a plurality of inspection images.

  The determination unit 303 acquires a comparison result for each of the plurality of inspection images. Based on the comparison result, when there is no defect in any of the inspection images, the determination unit 303 determines that there is no defect in the mounting of the solder joint of the inspection object.

  This is because the solder fillet causes metal reflection on the three-dimensional shape, so that the defective portion may not be clearly recognized depending on how it is seen. Thus, for example, imaging is performed by changing the viewpoint from the two imaging units on the left and right, and the imaging unit captures images by changing the position and direction of illumination, whereby a plurality of inspection images can be obtained. A determination process similar to that in the first embodiment is performed on the plurality of inspection images, and if any inspection image is not defective, the determination unit 303 determines that the shape of the solder fillet is not defective.

  FIG. 17 is a diagram illustrating an example of a positional relationship between the imaging unit and the lighting device. In the example illustrated in FIG. 17, the first imaging unit 12-1 images the inspection target from the left direction, and the second imaging unit 12-2 images the inspection target from the right direction. In addition, the illumination device 11 changes the illumination position to, for example, upper left, directly above, and upper right. In the example illustrated in FIG. 17, since there are two viewpoints and three illumination positions of the imaging unit, the inspection apparatus 10 can obtain six inspection images.

  Note that the first imaging unit 12-1 and the second imaging unit 12-2 are adjusted so as to image a dotted frame including a plurality of leads and solder joints extending from the integrated circuit 20, for example.

  FIG. 18 is a diagram illustrating an example of each reference image generated from each inspection image. The example illustrated in FIG. 18 illustrates an inspection image captured at the viewpoint position and the illumination position illustrated in FIG. In addition, a reference image created from each inspection image is shown. The solder fillet portion of the reference image varies in brightness depending on the direction of illumination.

  Thereby, the accuracy of the quality determination of the solder joint mounting can be improved by performing the quality determination with a plurality of inspection images having different viewpoint positions and illumination positions.

<Operation>
Next, the operation of the inspection apparatus 10 in the second embodiment will be described. FIG. 19 is a flowchart illustrating an example of inspection processing according to the second embodiment.

  In step S601 illustrated in FIG. 19, the setting unit 301 sets the position of the lighting device 11 to a predetermined position.

  In step S <b> 602, the first imaging unit 12-1 and the second imaging unit 12-2 image an inspection target and output an inspection image to the inspection apparatus 10. For example, left and right inspection images captured from the left and right directions are output to the inspection apparatus 10.

  In step S603, the extraction unit 311 determines whether the partial image including the solder joint portion has been successfully extracted from the left and right inspection images. The extraction process is the same as the process shown in FIG. If the extraction is successful (step S603—YES), the process proceeds to step S604. If the extraction fails (step S603—NO), the inspection process is terminated.

  In step S604, the creation unit 312 determines whether the reference image has been successfully created from the right inspection image. The reference image creation process uses one of the processes shown in FIGS. If the creation of the right reference image is successful (step 604-YES), the process proceeds to step 605, and if the creation of the right reference image fails (step S604-NO), the inspection process is terminated.

  In step S605, the creation unit 312 determines whether the reference image has been successfully created from the left inspection image. The reference image creation process uses one of the processes shown in FIGS. If the creation of the left reference image is successful (step 605-YES), the process proceeds to step 606. If the creation of the left reference image fails (step S605-NO), the inspection process is terminated. The order of steps S604 and S605 does not matter.

  In step S606, the setting unit 301 determines whether imaging has been performed under all illumination conditions. For example, when changing the illumination position in the order of right, center, and left, the setting unit 301 determines that imaging has been performed under all illumination conditions when the illumination device 11 is on the left.

  If the image is captured under all illumination conditions (step S606: YES), the process proceeds to step S607. If the image is not captured under all illumination conditions (step S606: NO), the process returns to step S601 to change the illumination condition.

  In step S607, the comparison unit 314 compares the reference image and each partial image for each of the plurality of inspection images. For example, the comparison unit 214 performs template matching between the lead and solder joint of the reference image and the lead and solder joint of the partial image, and calculates a correlation value.

  In step S <b> 608, based on the comparison result of the comparison unit 314, the determination unit 303 determines that the soldering mounting of the inspection object is defective when one of the plurality of inspection images is defective. When there is no defect in the plurality of inspection images, the determination unit 303 determines that the inspection object is mounted by soldering as a non-defective product.

  In the second embodiment, the example in which both the viewpoint position and the illumination position are changed has been described. However, only one of the positions may be changed.

  As described above, according to the second embodiment, since a complicated solder shape is imaged from different viewpoints and different illumination positions, there is a high possibility that an abnormal state will be imaged, an abnormality inspection failure is reduced, and the solder shape abnormality is robust. Can be detected.

[Example 3]
Next, the inspection apparatus in Example 3 will be described. In Example 3, the height of each lead is measured using a stereo image captured by the left and right imaging units. As a result, an abnormality in the height of the lead can be detected, and the accuracy of the quality determination of mounting in soldering can be increased.

<Hardware>
Since the hardware of the inspection apparatus and the equipment connected to the inspection apparatus in the third embodiment are the same as those in the second embodiment, description thereof is omitted.

<Function>
FIG. 20 is a block diagram illustrating an example of functions of the inspection apparatus 10 according to the third embodiment. The inspection apparatus 10 illustrated in FIG. 20 includes an acquisition unit 300, a setting unit 301, an image processing unit 401, and a determination unit 402. The acquisition unit 300, the setting unit 301, the image processing unit 401, and the determination unit 402 can be realized by the control unit 101 and the main storage unit 102 as a work memory, for example. In the configuration shown in FIG. 20, the same functions as those in the configuration shown in FIG. 16 are denoted by the same reference numerals, and the description thereof is omitted. The extraction unit 311 stores the extracted partial image in the storage unit 313.

  The image processing unit 401 includes a measurement unit 411. The measuring unit 411 detects the positions of the leads from the left and right inspection images, and obtains the three-dimensional positions of the left and right corresponding points based on the principle of triangulation. By this method, the lead height is obtained as a three-dimensional position as viewed from the reference plane.

  FIG. 21 is a diagram for explaining parallax detection based on stereo matching. Stereo matching uses a set of two images taken by two cameras arranged on the left and right. In addition, it is determined by area correlation calculation which part of the image captured by the left camera corresponds to which part of the image captured by the right camera. Next, the three-dimensional position of each point is estimated by triangulation using the correspondence. This method is stereo matching.

  First, the measurement unit 411 cuts out the image 51 of the lead portion of the left inspection image 50. The image of the lead portion is called a lead image.

  The measurement unit 411 obtains the center coordinates of the lead image 51 as the detection position of the left image. Using this lead image 51, a predetermined search range 54 of the right inspection image 52 is searched by template matching.

  The measurement unit 411 detects the lead image 53 on the right inspection image 52 at the position with the highest matching degree. The difference between these horizontal positions detected in the left and right inspection images is the parallax D. For example, the parallax D is a difference between the center of the lead image 51 and the center of the lead image 53.

  The distance from the lead detection position in the left and right camera coordinates to the corresponding point on the left and right is determined by the principle of triangulation.

  FIG. 22 is a diagram for explaining the measurement of the lead height based on the parallax. In the example illustrated in FIG. 22, when the optical axes of the left and right imaging units intersect, the distance is closer as the parallax is negatively increased, and the distance is longer as the parallax is positively increased.

  The measurement unit 411 obtains the height of the lead from the reference plane based on the distance to the corresponding point when the intersection of the central axes of the imaging units is the reference plane. The measurement unit 411 outputs the measured lead height to the determination unit 402.

  The determination unit 402 includes a solder quality determination unit 421 and a height quality determination unit 422. The solder quality determination unit 421 is the same as the determination unit 303 described in the second embodiment.

  FIG. 23 is a diagram for explaining determination of acceptability of solder. As illustrated in FIG. 23, when the solder quality determination unit 421 regards, for example, the correlation values based on six imaging conditions as a six-dimensional feature vector, the correlation value of each solder joint portion is 1 in the six-dimensional feature space. Represented by dots. The solder pass / fail judgment unit 421 judges that each lead is defective if there is a lead that falls below a preset pass / fail judgment plane. The pass / fail judgment surface represents, for example, a 0.9 surface in each dimension.

  Further, the solder quality determination unit 421 may adaptively adjust the quality determination surface based on the distribution of data in the 6-dimensional feature space. Moreover, since the correlation value can be sub-grouped by the left and right imaging units and the illumination position, the solder pass / fail judgment unit 421 can improve the judgment accuracy by setting a pass / fail judgment surface for each sub group. it can. These methods can also be applied in the second embodiment.

  The height acceptance / rejection determination unit 422 acquires the height of each lead from the measurement unit 411. The height acceptance / rejection determination unit 422 performs determination based on the following two relative criteria. As a first criterion, the height acceptance / rejection determination unit 422 determines that there is a floating lead if the variation in the lead height exceeds the threshold value.

  As a second criterion, the height acceptance / rejection determination unit 422 determines that the attachment of the connector is inclined as defective if the inclination of the distribution of the lead heights exceeds the threshold value.

  FIG. 24 is a diagram for explaining pass / fail determination based on the height of a lead. As shown in FIG. 24, the quality determination unit 422 measures the height of each lead included in the extracted partial image, and when the variation width exceeds the threshold, or an approximate straight line of the height If the slope of the curve exceeds the threshold, it is determined that there is a defect. In addition, the quality determination unit 422 may determine that the height is defective when there is a lead whose measured height is not within a predetermined range.

  The determination unit 402 determines that there is a mounting defect in soldering when any one of the solder quality determination unit 421 and the height quality determination unit 422 determines a failure. Thereby, the precision of the quality determination of the connector mounting including soldering and lead height can be improved.

<Operation>
Next, the operation of the inspection apparatus 10 in the third embodiment will be described. FIG. 25 is a flowchart illustrating an example of inspection processing according to the third embodiment. The processes in steps S701 to S705 shown in FIG. 25 are the same as the processes in steps S601 to S605 shown in FIG.

  In step S706, the measurement unit 411 measures the height of each lead from the left and right inspection images by stereo matching.

  The processing in steps S707 to S708 is the same as the processing in steps S606 to S607 shown in FIG.

  In step S709, the solder quality determination unit 421 determines whether the solder joint is mounted. The determination method is the same as in the second embodiment.

  In step S710, the quality determination unit 422 determines whether there is no floating lead or the connector is tilted based on the height of the lead.

  The determination unit 402 determines that the connector is not properly mounted if at least one is determined to be defective in steps S709 and S710.

  As described above, according to the third embodiment, since the height of each lead can be appropriately detected by measuring the height of each lead using a stereo image captured from the left and right viewpoints, Mounting defects can be detected with high accuracy.

[Example 4]
Next, an inspection apparatus according to Example 4 will be described. In the fourth embodiment, the positions and sizes of the partial images are made common to a plurality of inspection images captured by changing the illumination position and the viewpoint position. As a result, it is possible to make a pass / fail determination using partial images having a common position and size, and to efficiently detect mounting defects.

  In Example 4, since the partial images have the same size, it is possible to perform pass / fail determination using the same threshold value. As for sharing, it is possible to share the position and size of the partial image with respect to the inspection image of the same viewpoint, or to share the position and size of the partial image with respect to all the inspection images.

  First, an outline of processing in the fourth embodiment will be described. FIG. 26 is a diagram for explaining an outline of processing in the case of sharing a partial image area (part 1). In the example shown in FIG. 26, images are taken with the left and right cameras under illumination conditions 1 and 2, and four inspection images are acquired.

  In the example shown in FIG. 26, the partial image area is shared for the inspection image of the same viewpoint. Thereby, at the same viewpoint, partial images in the same region can be extracted, and pass / fail judgment can be performed using the same threshold value. Therefore, it is not necessary to extract partial areas independently from all inspection images or set a threshold value.

  Since the same object is imaged only at different illumination conditions at the same viewpoint, it is desirable that the positions and sizes of the partial image areas to be matched are unified.

  FIG. 27 is a diagram for explaining an outline of processing in the case of partial image area sharing (part 2). In the example shown in FIG. 27, the left and right cameras are imaged under illumination conditions 1 and 2, and four inspection images are acquired.

  In the example shown in FIG. 27, the partial image area is shared by all the inspection images. Thereby, the partial image of the same area | region can be extracted from all the test | inspection images, and a quality determination can be performed using a unified threshold value. Therefore, it is sufficient to set one threshold value for pass / fail judgment, and the pass / fail judgment can be performed efficiently.

  Even when inspection images are captured at multiple viewpoints or multiple illumination conditions, the same object is captured, so it is desirable that the position and size of the partial image areas to be matched are unified. . If a common optimal partial image area is detected by comprehensively using inspection images of multiple viewpoints or lighting conditions, a uniform matching score can be obtained for inspection images of any viewpoint or lighting conditions. An improvement in judgment accuracy can be expected. Below, Example 4 is described using the example shown in FIG.

<Hardware>
Since the hardware of the inspection apparatus and the equipment connected to the inspection apparatus in the fourth embodiment are the same as those in the second embodiment, the description thereof is omitted.

<Function>
Regarding the functional configuration in the fourth embodiment, the functions other than the extraction unit are the same as the functional configurations in the second and third embodiments. Therefore, the function of the extraction unit in the fourth embodiment will be described below. In addition, although the extraction part in Example 4 is applicable also in Example 2 or Example 3, the case where it applies to Example 3 is demonstrated below.

  FIG. 28 is a block diagram illustrating an example of the function of the extraction unit 501 according to the fourth embodiment. In the example illustrated in FIG. 28, the extraction unit 501 includes a region detection unit 510 and a partial image extraction unit 520.

  The region detection unit 510 detects a common partial image region for a plurality of inspection images having different illumination positions and viewpoint positions input from the acquisition unit 300. Note that the area detection unit 510 may detect a common partial image area for a plurality of images having different illumination positions or viewpoint positions.

  The region detection unit 510 includes a common interval detection unit 511 and a partial image region detection unit 512. The common interval detection unit 511 detects a common interval for a common partial image region based on the sum of luminance differences between pixels in the horizontal direction in each of the plurality of inspection images. Details of the common interval detection process will be described later.

  The partial image region detection unit 512 detects the position and size of the common partial image region based on the sum of the luminance integrals in the horizontal or vertical direction of each of the plurality of inspection images. Details of the common area detection processing will be described later.

  The partial image extraction unit 520 extracts a plurality of partial images in a common partial image region from each of the plurality of inspection images. The partial image extraction unit 520 outputs the extracted plurality of partial images to the creation unit 312 and the storage unit 313.

<Common processing>
Next, the sharing process of partial image areas will be described. First, a common lead interval is detected.

  FIG. 29 is a diagram for explaining the detection of the common lead interval. First, since the lead interval is constant even when the viewpoint and illumination conditions are different, the common lead interval is obtained by comprehensively processing a plurality of inspection images as shown in FIG. FIG. 29 shows an example in which the common lead interval is obtained for the inspection images captured by the left and right cameras for the sake of simplicity.

  FIG. 29A shows an inspection image captured by the left camera, and FIG. 29B shows an inspection image captured by the right camera. The common interval detection unit 511 creates a luminance difference histogram as shown in FIGS. 29C and 29D for the left and right inspection images. The common interval detection unit 511 performs processing similar to the processing described with reference to FIGS. 5 and 6 to create a luminance difference histogram.

  In the histograms shown in FIGS. 29C and 29D, as in FIG. 6, the minimum horizontal pixel interval corresponds to the connector lead interval. As shown in FIGS. 29C and 29D, the left and right histograms are somewhat distorted due to differences in the appearance of each lead, noise, etc., but if the left and right histograms are added and averaged, Such noise is canceled out.

  Therefore, as shown in FIG. 29E, the common interval detection unit 511 creates a histogram by averaging the left and right histograms. The common interval detection unit 511 can obtain an appropriate common lead interval by detecting the minimum position of the histogram shown in FIG.

  The inspection apparatus can improve the detection accuracy of the rectangular region of the lead in each of the left and right images by using the detected common lead interval.

  Even if conditions with different illumination conditions are added, the common interval detection unit 511 may add and average luminance difference histograms created from a plurality of inspection images with different illumination conditions and viewpoint positions.

  When the left and right cameras have different scales, the common interval detection unit 511 may convert the distance (number of pixels) in the screen between the left and right according to the scale difference obtained by camera calibration.

  Next, an example of improving the efficiency of processing for detecting the read interval will be described. FIG. 30 is a diagram for explaining the processing efficiency of the read interval. Here, the lead intervals of the connector parts are generally determined by the standards in each product field.

  For example, if it is known in advance that the lead interval is either 0.4 mm or 0.5 mm, the extraction unit narrows down to a pixel interval portion corresponding to the lead interval as shown in FIG. What is necessary is just to create a histogram. Thereby, the processing amount in the read interval can be reduced. In addition, this process efficiency improvement is applicable also about Examples 1-3.

  In addition, in addition to the solder joints of the leads, the connector housing fittings and the like may periodically appear in the inspection image. FIG. 31 is a diagram illustrating an example of an inspection image including a metal part. Here, a portion having a high luminance value other than the extracted lead and solder joint portion is referred to as a noise portion. The noise part is a metal part as shown in FIG. 31, for example.

When the extraction unit detects the lead interval, the following criterion can be used to remove the noise portion.
・ If the size of the bright part is within the specified range, it is considered as a noise part.
In particular, if the size in the vertical direction is small, it is considered as a metal fitting instead of a lead.
・ If the bright part has a predetermined shape, it is considered a noise part.
In particular, a horizontally long shape is considered to be a metal fitting instead of a lead.
-If the correlation value of the extracted area is greater than or equal to a predetermined value, it is considered as a bracket part.
The images of leads and solder fillets vary individually, but the metal parts are almost the same image, so the correlation values are concentrated on very high values.

  Therefore, the extraction unit removes a portion that satisfies any of the above conditions as a noise portion, and detects the lead interval at another position. This makes it possible to detect an appropriate lead interval when detecting the lead interval at various parts in the inspection image.

  Next, detection of a common partial image area will be described. The method of detecting the partial image area of each of the plurality of inspection images is as described with reference to FIG. Therefore, the partial image region detection unit 512 performs luminance integration in the horizontal and vertical directions for each inspection image, and detects a range equal to or greater than a predetermined luminance integration value as the partial image region.

  The partial image region detection unit 512 adds the horizontal and vertical luminance integral values obtained from the respective inspection images, and detects a common partial image region based on the added integral value.

  FIG. 32 is a diagram for detecting the vertical range of the common partial image area. FIG. 32 shows an example in which the vertical range of the common partial image region is obtained with respect to the inspection images captured by the left and right cameras for the sake of simplicity. The common partial image region refers to a partial image region that is used in common by a plurality of inspection images.

  FIG. 32A shows an inspection image of the left camera, and FIG. 32B shows an integration value obtained by integrating the inspection image of the left camera in the horizontal direction. FIG. 32C shows an inspection image of the right camera, and FIG. 32D shows an integration value obtained by integrating the inspection image of the right camera in the horizontal direction.

  The partial image area detection unit 512 obtains the sum of the luminance integral value shown in FIG. 32B and the luminance integral value shown in FIG. 32D (FIG. 32E), and based on the sum of the left and right integral values. To define the vertical range. The predetermined range s in the vertical direction is obtained in the same manner as the processing shown in FIG.

  FIG. 32 (F) shows an integration value obtained by integrating the inspection image of the left camera in the vertical direction, and FIG. 32 (G) shows the integration value in the vertical direction of the inspection image of the right camera. The integrated value is shown.

  Similar to the vertical direction, partial image region detection section 512 obtains the sum of the integral value shown in FIG. 32 (F) and the integral value shown in FIG. 32 (G), and based on the sum of the left and right integral values. Define the horizontal range. The predetermined range t in the horizontal direction is obtained in the same manner as the processing shown in FIG.

  Since the lead and the solder joint have a complicated three-dimensional shape, if the viewpoint or illumination position changes, the portion that appears bright changes, and even the solder joint may appear dark. In this case, the partial image region detection unit 512 may set a partial image region by regarding a bright portion of either one of the left and right inspection images as a lead portion. For example, the partial area detection unit 512 can perform a logical sum of luminance integration at corresponding vertical and horizontal positions on the left and right sides, or can perform a simple addition average.

  Thereby, a partial image area including a portion to be inspected can be detected. By using partial images in the vertical and horizontal ranges that are common to the left and right, a common partial image region can be obtained.

<Operation>
Next, the operation of the inspection apparatus in Example 4 will be described. FIG. 33 is a flowchart illustrating an example of inspection processing according to the fourth embodiment. The processes in steps S801 to S802 shown in FIG. 33 are the same as the processes in steps S601 to S602 shown in FIG.

  In step S803, the setting unit 301 determines whether imaging has been performed under all illumination conditions. For example, when changing the illumination position in the order of right, center, and left, the setting unit 301 determines that imaging has been performed under all illumination conditions when the illumination device 11 is on the left.

  If the image is captured under all illumination conditions (step S803-YES), the process proceeds to step S804. If the image is not captured under all illumination conditions (step S803-NO), the illumination condition is changed and the process returns to step S801.

  In step S804, the extraction unit 501 determines whether the common partial image region has been successfully extracted for all inspection images. The common partial area image detection process will be described with reference to FIG. If the extraction is successful (step S804-YES), the process proceeds to step S805, and if the extraction fails (step S804-NO), the inspection process is terminated.

  In steps S805 to S807, the processing of steps S704 to S706 shown in FIG. 25 is performed for each illumination condition.

  Steps S808 to S810 are the same as steps S708 to S710 shown in FIG.

  FIG. 34 is a flowchart illustrating an example of the common partial image region detection process according to the fourth embodiment. In step S901 illustrated in FIG. 34, the extraction unit 501 performs luminance integration in the horizontal direction over the entire screen of the inspection image.

  In step S902, the extraction unit 501 determines whether the luminance integration in the horizontal direction has been accumulated for all inspection images. If all the inspection images have been accumulated (step S902-YES), the process proceeds to step S903. If all the inspection images have not been accumulated (step S90-NO), the unprocessed inspection image is processed. Therefore, the process returns to step S901.

  The processing in steps S903 to S904 is the same as the processing in steps S201 to S202 shown in FIG.

  In step S905, the extraction unit 501 determines whether the creation of the luminance difference histogram has been completed for all inspection images. If creation has been completed for all inspection images (YES in step S905), the process proceeds to step S906. If creation has not been completed for all inspection images (NO in step S905), brightness for all inspection images is determined. The process returns to step S904 to create a difference histogram.

  The processing in steps S906 to S907 is the same as the processing in steps S203 to S204 shown in FIG.

  In step S908, the extraction unit 501 acquires a luminance integration value in the horizontal direction for the upper and lower vicinity of the horizontal position where the lead interval is detected.

  In step S909, the extraction unit 501 determines whether or not high luminance representing a luminance integral value equal to or greater than a predetermined value is continuous for a predetermined length. The predetermined length corresponds to the length of the lead portion, and an appropriate value is set by experiment or the like. If the high brightness continues for a predetermined length (step S909-YES), it is determined that a common lead interval can be detected from all inspection images, and the process proceeds to steps S910 and S914, and the high brightness must continue for a predetermined length. (Step S909—NO), the process returns to Step S903 to select another horizontal line. By this processing, for example, a noise part such as a metal fitting can be removed.

  In step S910, the extraction unit 501 performs luminance integration in the vertical direction for all inspection images.

  In step S911, the extraction unit 501 accumulates the luminance integration values in the vertical direction of all inspection images at each read interval.

  In step S912, the extraction unit 501 detects the peak of the accumulated value of the integrated luminance value within the lead interval as the center position in the lead.

  In step S913, the extraction unit 501 detects a predetermined range t based on the center position of the lead as the horizontal range of the lead at the left end of all inspection images.

  In step S914, the extraction unit 501 detects the predetermined range s as the vertical range of the lead by using the luminance integrated value in the horizontal direction of all inspection images.

  Note that steps S910 to S913 and step S914 are out of order, and the processes may be performed in parallel, or one of them may be performed first.

  In step S915, the extraction unit 501 detects, as a common partial image area, an image of a rectangular area that includes the detected predetermined range t of the leftmost horizontal range and the predetermined range s of the vertical range.

  In step S916, when the leftmost common partial image area is determined, the extraction unit 501 detects all the common partial image areas by, for example, shifting the center position of the common partial image area in the horizontal direction by the common lead interval. be able to.

  Thereby, the common partial image area including the lead and the solder joint can be detected from all the inspection images. Thereafter, the extraction unit 501 extracts an image included in the common partial image region as a partial image for all inspection images.

  As described above, according to the fourth embodiment, by sharing the position and size of the partial image with respect to the plurality of inspection images captured by changing the illumination position and the viewpoint position, the partial image having the common position and size can be obtained. It can be used to make a pass / fail judgment, and a mounting defect can be detected efficiently.

[Modification]
In addition, by recording a program for realizing the inspection process described in each of the above-described embodiments on a recording medium, the computer can execute the inspection process in each of the embodiments. For example, it is possible to record the program on a recording medium and cause the computer to read the recording medium on which the program is recorded, thereby realizing the above-described inspection processing.

  The recording medium is a recording medium that records information optically, electrically, or magnetically, such as a CD-ROM, flexible disk, magneto-optical disk, etc., and information is electrically recorded such as ROM, flash memory, etc. Various types of recording media such as a semiconductor memory can be used. This recording medium does not include a carrier wave.

  Although each embodiment has been described in detail above, it is not limited to a specific embodiment, and various modifications and changes can be made within the scope described in the claims. It is also possible to combine all or a plurality of the constituent elements of the above-described embodiments.

In addition, the following additional notes are disclosed regarding each of the above embodiments.
(Appendix 1)
An extraction unit for extracting a plurality of partial images including the solder joints from an image including a plurality of solder joints;
Using a plurality of partial images extracted by the extraction unit, a creation unit that creates a reference image serving as a reference for determining whether the solder joint is mounted or not,
A comparison unit that compares the reference image created by the creation unit with each partial image;
Based on the comparison result by the comparison unit, a determination unit that determines the quality of the mounting of each solder joint,
An inspection apparatus comprising:
(Appendix 2)
The extraction unit includes:
Extracting a plurality of the partial images for each of the plurality of images having different illumination positions;
The creating unit
Creating the reference image for each of the plurality of images,
The comparison unit includes:
The inspection apparatus according to appendix 1, wherein for each image, a reference image of the image is compared with each partial image of the image.
(Appendix 3)
The extraction unit includes:
Extracting a plurality of partial images for each of the plurality of images having different viewpoint positions of the imaging unit;
The creating unit
Creating the reference image for each of the plurality of images,
The comparison unit includes:
The inspection apparatus according to appendix 1 or 2, wherein for each image, a reference image of the image is compared with each partial image of the image.
(Appendix 4)
The plurality of images are images taken from two left and right viewpoints,
It further includes a measurement unit that performs three-dimensional measurement using the left and right images and measures a three-dimensional position of the inspection target component in the partial image,
The determination unit
The inspection apparatus according to supplementary note 3, wherein the quality of mounting of each solder joint portion is determined based on the three-dimensional position and the comparison result.
(Appendix 5)
The determination unit
The inspection apparatus according to appendix 4, which performs pass / fail determination of different mounting for each of the plurality of images.
(Appendix 6)
The determination unit
The inspection apparatus according to appendix 4, wherein quality determination is performed based on a variation width of the three-dimensional position of each partial image or an inclination of an approximate line obtained from the three-dimensional position of each partial image.
(Appendix 7)
The creating unit
The correlation image is calculated between the partial images for the plurality of partial images, and the reference image is created by averaging the partial images having a correlation value higher than a threshold value. Inspection device.
(Appendix 8)
The extraction unit includes:
An area detection unit for detecting a common partial image area for the plurality of images having different illumination positions and / or viewpoint positions;
The inspection apparatus according to appendix 3, further comprising: a partial image extraction unit that extracts a plurality of partial images in the common partial image region from each of the plurality of images.
(Appendix 9)
The region detection unit
The inspection apparatus according to appendix 8, wherein the position and size of the common partial image area are detected based on a sum of luminance integrals in the horizontal or vertical direction of each of the plurality of images.
(Appendix 10)
The region detection unit
The inspection apparatus according to appendix 9, wherein a common interval with respect to the common partial image region is detected based on a sum of luminance differences between horizontal pixels in each of the plurality of images.
(Appendix 11)
Extracting a plurality of partial images including the solder joints from an image including a plurality of solder joints,
Using a plurality of the extracted partial images, create a reference image that serves as a reference for determining the quality of the solder joint mounting,
Compare the created reference image with each partial image,
An inspection method in which a computer executes a process for determining whether or not each solder joint is mounted based on a comparison result.
(Appendix 12)
Extracting a plurality of partial images including the solder joints from an image including a plurality of solder joints,
Using a plurality of the extracted partial images, create a reference image that serves as a reference for determining the quality of the solder joint mounting,
Compare the created reference image with each partial image,
An inspection program for causing a computer to execute processing for determining whether or not each solder joint is mounted based on the comparison result.

DESCRIPTION OF SYMBOLS 10 Inspection apparatus 11 Illumination equipment 12 Image pick-up part 12-1 1st image pick-up part 12-2 2nd image pick-up part 13 Display part 20 Integrated circuit 21 Lead 22 Solder joint part 101 Control part 102 Main memory part 103 Auxiliary memory part 104 Communication part 105 Drive device 201, 300 Acquisition unit 202, 302, 401 Image processing unit 203, 303 Judgment unit 211, 311, 501 Extraction unit 212, 312 Creation unit 213, 313 Storage unit 214, 314 Comparison unit 221, 321 Calculation unit 301 Setting unit 411 Measuring unit 421 Solder quality determination unit 422 Height quality determination unit 510 Area detection unit 511 Common interval detection unit 512 Partial image area detection unit 520 Partial image extraction unit

Claims (10)

An extraction unit for extracting a plurality of partial images including the solder joints from an image including a plurality of solder joints;
Using a plurality of partial images extracted by the extraction unit, a creation unit that creates a reference image serving as a reference for determining whether the solder joint is mounted or not,
A comparison unit that compares the reference image created by the creation unit with each partial image;
Based on the comparison result by the comparison unit, a determination unit that determines the quality of the mounting of each solder joint,
An inspection apparatus comprising: The extraction unit includes:
Extracting a plurality of the partial images for each of the plurality of images having different illumination positions;
The creating unit
Creating the reference image for each of the plurality of images,
The comparison unit includes:
The inspection apparatus according to claim 1, wherein a reference image of the image is compared with each partial image of the image for each image. The extraction unit includes:
Extracting a plurality of partial images for each of the plurality of images having different viewpoint positions of the imaging unit;
The creating unit
Creating the reference image for each of the plurality of images,
The comparison unit includes:
The inspection apparatus according to claim 1 or 2, wherein a reference image of the image is compared with each partial image of the image for each image. The plurality of images are images taken from two left and right viewpoints,
It further includes a measurement unit that performs three-dimensional measurement using the left and right images and measures a three-dimensional position of the inspection target component in the partial image,
The determination unit
The inspection apparatus according to claim 3, wherein whether or not each solder joint is mounted is determined based on the three-dimensional position and the comparison result. The creating unit
5. The reference image is created by calculating a correlation value between the partial images for the plurality of partial images, and averaging the partial images having a correlation value higher than a threshold value. Inspection equipment. The extraction unit includes:
An area detection unit for detecting a common partial image area for the plurality of images having different illumination positions and / or viewpoint positions;
The inspection apparatus according to claim 3, further comprising: a partial image extraction unit that extracts a plurality of partial images in the common partial image region from each of the plurality of images. The region detection unit
The inspection apparatus according to claim 6, wherein the position and size of the common partial image area are detected based on a sum of luminance integrals in the horizontal or vertical direction of each of the plurality of images. The region detection unit
The inspection apparatus according to claim 7, wherein a common interval with respect to the common partial image region is detected based on a sum of luminance differences between horizontal pixels in each of the plurality of images. Extracting a plurality of partial images including the solder joints from an image including a plurality of solder joints,
Using a plurality of the extracted partial images, create a reference image that serves as a reference for determining the quality of the solder joint mounting,
Compare the created reference image with each partial image,
An inspection method in which a computer executes a process for determining whether or not each solder joint is mounted based on a comparison result. Extracting a plurality of partial images including the solder joints from an image including a plurality of solder joints,
Using a plurality of the extracted partial images, create a reference image that serves as a reference for determining the quality of the solder joint mounting,
Compare the created reference image with each partial image,
An inspection program for causing a computer to execute processing for determining whether or not each solder joint is mounted based on the comparison result.