JP5564349B2 - Image processing apparatus and appearance inspection method - Google Patents

Description

  The present invention relates to an image processing apparatus and an appearance inspection method for capturing a workpiece with a camera and performing measurement processing using the obtained image data.

  In many production sites such as factories, image processing devices have been introduced in order to automate and speed up inspections that depend on human visual inspection. The image processing apparatus captures an image of a workpiece flowing on a production line such as a belt conveyor with a camera, and executes measurement processing such as edge detection and area calculation of a predetermined region using the obtained image data. Then, based on the processing result of the measurement process, the workpiece is inspected for chipping or burrs, and the quality of the workpiece is determined. At the FA site, the image processing apparatus is used as one of FA sensors for determining the quality of a workpiece.

  When using such an image processing apparatus to check whether or not a figure of a predetermined shape is correctly formed on a work, for example, by printing or the like, first, a reference figure that is the same as the predetermined shape is previously stored in the image processing apparatus. Remember. Then, the workpiece is imaged by the camera, pattern matching processing (for example, normalized correlation calculation) using the reference graphic is performed on the obtained image data, and the quality of the workpiece is determined based on the processing result. However, in actual operation situations, it is necessary to judge the quality of a large amount of work at high speed, so instead of performing pattern matching processing on the entire area of the obtained image data, you want to inspect the obtained image data. An area (inspection area) including a figure is specified, and pattern matching processing is performed only within this inspection area. Thereby, the erroneous detection resulting from the figure which exists out of a test | inspection area | region can be prevented, and the quality determination accuracy of a workpiece | work can be improved. In addition, it is possible to reduce the amount of calculation of various measurement processes and pre-processes in the image processing apparatus such as pattern matching process and edge extraction process, thereby speeding up the work quality determination.

  Here, when the workpieces flowing on the production line are regularly arranged, the inspection area in the obtained image data may be fixed. Therefore, in this case, it is easy to specify the inspection area (for example, a rectangular area having a central portion of the obtained image data as an intersection of diagonal lines is used as the inspection area). However, the workpieces are not always regularly arranged, and the workpiece may flow in a state shifted from the original position or tilted from the basic posture. In such a case, since the figure to be inspected is also shifted from the original position or inclined from the basic posture, it is necessary to correct the position and inclination of the inspection area in the image data. . Thus, for example, a technique is known in which an alignment mark for positioning is formed in advance on a workpiece and the position and inclination of the inspection region are corrected by using this alignment mark (see, for example, Patent Document 1).

  Specifically, first, alignment marks (for example, cross marks) for positioning are formed on the workpiece. This alignment mark has a certain positional / angular relationship with the position where the figure to be inspected is formed on the workpiece. For example, in the obtained image data, the alignment mark is formed so that the alignment mark appears at a position of 500 pixels upward and 500 pixels to the left from the center of gravity of the figure to be inspected. In an actual operational situation, the position of the alignment mark is detected from the image data obtained by the camera, and the inclination from the reference posture is also detected. Based on the position of the detected alignment mark, the position of the inspection area (for example, a rectangular area) including the figure to be inspected is corrected, and the inclination of the inspection area is corrected based on the detected inclination of the alignment mark. To do. A pattern matching process is performed using the reference graphic within the inspection area determined through such a correction process, and the quality of the workpiece is determined based on the processing result.

  As described above, in order to correct the position and inclination of the inspection region in the obtained image data, it is important to detect the alignment mark formed in advance on the workpiece from the obtained image data.

Japanese Patent Laid-Open No. 07-270331

  However, the alignment mark formed on the workpiece may not be detected stably. For example, when the alignment mark is a cross mark drawn in a color (for example, black) different from the surrounding color (for example, white), it can be detected by edge processing or the like. However, when the alignment mark is a cross mark composed of slight unevenness by pressing, the shade of each pixel of the alignment mark portion in the obtained image data does not change greatly, so it is difficult to detect even by edge processing etc. There is a risk of becoming. If the alignment mark cannot be detected, the position and inclination of the inspection area cannot be corrected as described above, and as a result, the quality of the workpiece cannot be properly determined.

  In this regard, for example, it is conceivable to devise a method of applying illumination for illuminating the workpiece at the time of inspection to detect an alignment mark by detecting a shadow caused by unevenness. However, a clear shadow is not always detected depending on the surrounding lighting environment, and if it is necessary to devise how to apply the illumination, it takes time to install the illumination.

  The present invention has been made in view of the above points, and an object thereof is to provide an image processing apparatus and an appearance inspection method capable of stably specifying an inspection region.

  An image processing apparatus according to the present invention includes a camera that captures a workpiece, inspects a predetermined inspection range on the surface of the workpiece based on an image acquired from the camera, and determines whether the workpiece is acceptable. Each pixel has a grayscale image acquisition means for acquiring a grayscale image having a grayscale value corresponding to the amount of light received by the camera, a distance calculation means for calculating the distance from the camera to the workpiece surface using the image acquired from the camera, and A distance image generating means for generating a distance image having a gray value corresponding to the calculated distance, and an inspection area corresponding to the inspection range on one of the gray image and the distance image is specified on the other image Specific pattern detecting means for detecting a specific pattern to detect, and a specific pattern detected by the specific pattern detecting means in the other image of the grayscale image and the distance image Based on at least one of the position and the inclination angle, the inspection area specifying means for specifying the inspection area, the feature quantity calculating means for calculating the feature quantity from the specified inspection area, and the workpiece based on the calculated feature quantity Determination means for determining pass / fail.

  That is, a specific pattern is detected in one image of a general gray image and a distance image having a gray value corresponding to the distance from the camera to the workpiece surface, and the position or inclination angle of the specific pattern detected in the other image Since the inspection area is specified based on at least one of the above, the feature amount is calculated from the specified inspection area, and the quality of the work is determined, for example, the alignment mark as the specific pattern has slight unevenness due to the press. Even in such a case, it is possible to detect the alignment mark in the distance image where the unevenness can be detected. Accordingly, since the inspection area can be specified on the grayscale image based on at least one of the detected position and inclination angle of the alignment mark, the inspection area can be specified stably. If the alignment mark does not tilt due to restrictions on the production line through which the workpiece flows, the inspection area may be specified based only on the position of the alignment mark. Conversely, if the (centroid) position of the alignment mark is a fixed position, the inspection area may be specified based only on the inclination angle of the alignment mark.

  The image processing apparatus according to the present invention also accepts a setting of a reference inspection area having a certain relative positional relationship with a specific pattern on the other image when the inspection area is specified by the inspection area specifying means. The inspection area may be specified by correcting the position of the reference inspection area on the other image based on at least one of the position and the inclination angle of the specific pattern detected in (1). Thereby, since the user can set a desired reference inspection region, the usability of the image processing apparatus can be improved.

  Further, the image processing apparatus according to the present invention uses an edge pattern based on the edge extracted on one image as the specific pattern when specifying the inspection region by the inspection region specifying means, and the position and inclination angle of the specific pattern are determined. The inspection region may be specified by generating a region corresponding to the edge pattern on the other image based on at least one. Thereby, even if the user does not set the above-described reference inspection area, the detection area can be appropriately specified, and the setting work can be saved.

  In addition, the image processing apparatus according to the present invention can detect a detection condition for detecting a specific pattern (for example, the type of filter used for preprocessing, the magnitude of edge strength as a threshold used for detection, or a normalized correlation). A detection condition setting receiving unit that receives a setting of a reference image for detecting a specific pattern by calculation, and a detection condition storage unit that stores the detection condition, and the specific pattern detection and output unit stores the detected detection condition On the other hand, a configuration may be adopted in which a specific pattern for specifying an inspection area corresponding to the inspection range is detected on the other image. Thereby, by setting the optimal detection condition, the detection accuracy of the specific pattern can be increased, and as a result, the detection accuracy of the detection region can be increased.

  Further, the camera for acquiring the grayscale image and the camera used for generating the distance image may be the same. As a result, a grayscale image and a distance image can be obtained for the same region of the same work, so calibration for obtaining relative parameters inside and between cameras and correction of the detection position based on the obtained parameters There is no need to perform measurement, and the processing load of the measurement process can be reduced.

  Further, the inspection area specifying means specifies the inspection area based on at least one of the position and the inclination angle of the specific pattern detected by the specific pattern detection means in one of the gray image and the distance image, and the feature amount The calculating means calculates the first and second feature amounts from the inspection regions specified in the grayscale image and the distance image, respectively, and the determining means is based on at least one of the first and second feature amounts. A configuration may be adopted in which the quality of the workpiece is determined. As a result, it is possible to perform an appearance inspection using one of the first and second feature amounts, or to perform an appearance inspection using both the first and second feature amounts. Usability of the processing apparatus can be improved.

  Moreover, you may make it the structure which determines the quality of a workpiece | work based on both the 1st and 2nd feature-value. As a result, for example, the user allows one of the dirt detectable on the grayscale image and the dent detectable on the distance image, but does not allow the other (for example, some dents on the workpiece are allowed. In addition, it is possible to perform a variety of appearance inspections such as that the workpiece is not allowed to be contaminated), and the usability can be further improved.

  As described above, according to the present invention, it is possible to stably specify the inspection region, and consequently improve the appearance inspection accuracy.

1 is a diagram illustrating a system configuration example of an image system including an image processing apparatus according to an embodiment of the present invention. It is explanatory drawing for demonstrating a distance image. It is a block diagram which shows the hardware structural example of the image processing apparatus which concerns on this embodiment. It is a figure which shows the function structural example of the image processing apparatus which concerns on this embodiment. It is explanatory drawing regarding a mode that an inspection area | region is specified in the image processing apparatus which concerns on this embodiment. It is explanatory drawing regarding a mode that an inspection area | region is specified in the image processing apparatus which concerns on this embodiment. It is explanatory drawing regarding a mode that an inspection area | region is specified in the image processing apparatus which concerns on this embodiment. It is explanatory drawing regarding a mode that an inspection area | region is specified in the image processing apparatus which concerns on this embodiment. It is explanatory drawing regarding a mode that an inspection area | region is specified in the image processing apparatus which concerns on this embodiment. It is explanatory drawing regarding a mode that an inspection area | region is specified in the image processing apparatus which concerns on this embodiment. It is explanatory drawing regarding a mode that an inspection area | region is specified in the image processing apparatus which concerns on this embodiment. It is explanatory drawing regarding a mode that an inspection area | region is specified in the image processing apparatus which concerns on this embodiment. It is a figure which shows an example of the user interface screen displayed on a monitor in the image processing apparatus which concerns on this embodiment. It is a figure which shows an example of the user interface screen displayed on a monitor in the image processing apparatus which concerns on this embodiment. It is a figure which shows an example of the user interface screen displayed on a monitor in the image processing apparatus which concerns on this embodiment. It is a figure which shows an example of the user interface screen displayed on a monitor in the image processing apparatus which concerns on this embodiment. It is a figure which shows an example of the user interface screen displayed on a monitor in the image processing apparatus which concerns on this embodiment. It is a figure which shows an example of the user interface screen displayed on a monitor in the image processing apparatus which concerns on this embodiment. It is a figure which shows an example of the user interface screen displayed on a monitor in the image processing apparatus which concerns on this embodiment. 5 is a flowchart showing a processing operation of the image processing apparatus according to the present embodiment. It is a figure which shows the application specific example when the grayscale image acquisition means, the distance calculation means, and the distance image generation means are replaced in the functional structure shown in FIG. It is a figure which shows the application specific example which specifies an test | inspection area | region, without using an alignment mark. It is a figure which shows another example of a function structure of the image processing apparatus which concerns on this embodiment. It is an application specific example (appearance inspection example) showing a state in which an inspection area is specified (inspection area is generated). It is explanatory drawing for demonstrating an example of the method of producing | generating an edge pattern from a distance image or a grayscale image. It is an example of an application that performs an appearance inspection using both a grayscale image and a distance image. It is a figure which shows an example of the selection screen for calculating the 1st and 2nd feature-value by the feature-value calculation means, and using at least one of these. It is a figure which shows the application specific example which performs the quality determination of a workpiece | work using both the 1st and 2nd feature-value calculated by the feature-value calculation means.

  Hereinafter, an image processing apparatus according to an embodiment of the present invention will be specifically described with reference to the drawings.

[System configuration]
FIG. 1 is a diagram showing a system configuration example of an image system 1 including an image processing apparatus 10 according to an embodiment of the present invention.

  An image system 1 shown in FIG. 1 includes an image processing apparatus 10 that performs measurement processing such as edge detection and area calculation, a camera 30 that captures a workpiece (inspection object), a monitor 40 such as a liquid crystal panel, and a user. A console 50 for performing various operations on the monitor 40 and a lighting device 60 for illuminating the work are provided. The camera 30, the monitor 40, the console 50, and the illumination device 60 are detachably connected to the image processing device 10. Among these, the illuminating device 60 is used as an illuminating means for illuminating the workpiece in order to generate a distance image (details will be described later). Alternatively, it may be a pattern projector for projecting a sinusoidal fringe pattern onto the workpiece. Although omitted in FIG. 1, a general illumination device (for example, a ring illuminator) for performing bright-field illumination and dark-field illumination may be separately provided (the illumination device 60 includes a general illumination device). It is also possible to have a function).

  The image processing apparatus 10 executes image processing using image data obtained from the camera 30, and indicates a determination result such as the quality of a work to a control device such as an externally connected PLC (Programmable Logic Controller) 70. A determination signal is output as a signal.

  The camera 30 images the inspection object based on a control signal input from the PLC 70, for example, an imaging trigger signal that defines a timing for capturing image data from the camera 30. The monitor 40 is a display device for displaying the image data obtained by imaging the inspection object and the result of the measurement process using the image data. The console 50 is an input device for moving the focus position on the monitor 40 and selecting menu items. In general, the user can confirm the operating state of the image processing apparatus 10 during operation by viewing the monitor 40. In addition, as will be described later, the user can perform various settings and various edits as necessary by operating the console 50 while viewing the monitor 40.

  The image processing apparatus 10 can also be connected to a PC 80 for generating a control program for the image processing apparatus 10, and generates a processing order program that defines the processing order of image processing by software operating on the PC 80. can do. In the image processing apparatus 10, each image processing is sequentially executed according to the processing order. The PC 80 and the image processing apparatus 10 are connected via a communication network, and the processing order program generated on the PC 80 is transferred to the image processing apparatus 10 together with layout information that defines the display mode of the monitor 40, for example. Is done. Conversely, a processing order program, layout information, and the like can be taken from the image processing apparatus 10 and edited on the PC 80. This processing order program may be generated not only by the PC 80 but also by the image processing apparatus 10.

[Generate distance image]
A “distance image” obtained by using the camera 30 and the illumination device 60 shown in FIG. 1 refers to an image in which the gray value of each pixel changes according to the distance from the camera 30 that captures the workpiece to the workpiece. . In other words, it can be said that the gray value is determined based on the distance from the camera 30 to the workpiece, can be said to be a multi-value image having a gray value corresponding to the distance to the workpiece, or the height of the workpiece. It can also be said to be a multi-value image having a corresponding gray value. Furthermore, it can be said that it is a multi-value image in which the distance from the camera 30 is converted into a gray value for each pixel of the gray image. For example, as shown in FIG. 2A, consider a distance image of a work in which cylinders having different radii are stacked in two stages. The distance to the top surface S ₁ and the camera 30 of the workpiece is l _1, the distance to the middle plane S ₂ and the camera 30 of the workpiece is l _2, the distance to the surface S ₃ and the camera 30 mounting the workpiece l ₃ and the height of the workpiece is L (= l ₃ −l ₁ ). From such a workpiece, for example, a distance image as shown in FIG. 2B can be obtained. According to FIG. 2 (b), the darkest (e.g., black) is uppermost surface S ₁ of the work, the thinnest mounting surface S ₃ of the workpiece (for example, white), the middle plane S ₂ of these intermediate colors of the workpiece (e.g. Gray). That is, according to the distances l ₁ , l ₂ , and l ₃ (l ₁ <l ₂ <l ₃ ) between the workpiece and the camera 30, the top surface S ₁ , the middle surface S ₂ , and the placement surface S ₃ The gray value of the pixel is small. Thus, in the distance image, the gray value of each pixel changes according to the distance from the camera 30 to the workpiece.

  There are roughly two methods for generating a distance image. One is a passive method (passive measurement) that generates a distance image using an image captured under illumination conditions to obtain a normal image. The other method is an active method (active measurement method) in which a distance image is generated by actively irradiating light in order to measure in the height direction. A typical passive method is a stereo measurement method. This is because a distance image can be generated simply by preparing two cameras 30 and arranging these two cameras in a predetermined positional relationship, so that a distance can be obtained using a general image system for generating a gray image. Images can be generated, and system construction costs can be reduced. However, in the stereo measurement method, it is necessary to determine which point in the image obtained by one camera corresponds to the point in the image obtained by the other camera. There is a problem that it takes time. Also, it is difficult to specify corresponding points in all pixels, and this point is also not suitable for speeding up the appearance inspection. On the other hand, typical methods of the active method are a light cutting method and a pattern projection method. The light cutting method is the above-described stereo measurement method, in which one camera is replaced with an optical projector, a line-shaped laser beam is projected onto the workpiece, and the line light image is distorted according to the shape of the object surface. Restore the 3D shape of the workpiece from the condition. Since the light section method can easily determine the corresponding points, it can be expected to speed up the measurement process to some extent as compared with the stereo measurement method. However, since only one line can be measured in one measurement, it is necessary to project the laser beam multiple times to obtain the measurement value of all pixels, and the object or camera must be scanned. There is a limit to speedup. The pattern projection method captures a plurality of images by shifting the shape and phase of a predetermined pattern projected onto a workpiece, and restores the three-dimensional shape of the workpiece by analyzing the captured images. . There are several types of pattern projection methods, taking multiple images (at least 3 images) by shifting the phase of the sinusoidal fringe pattern, obtaining the phase of the sine wave for each pixel from the multiple images, Reconstruct 3D shape using phase shift method to obtain 3D coordinates on workpiece surface using the obtained phase and a kind of spatial frequency beat phenomenon that occurs when two regular patterns are synthesized. A typical example is the moire repography method.

  In the image processing apparatus 10 according to the present embodiment, a distance image is generated by the phase shift method described above. This can contribute to speeding up the appearance inspection. The present invention is not limited to generating a distance image by the phase shift method, and the distance image may be generated by the above-described method other than the phase shift method. Any method other than the method described above, such as an optical radar method (time-of-flight), a focusing method, a confocal method, a white light interferometry, can be considered. Absent.

[Hardware configuration]
FIG. 3 is a block diagram illustrating a hardware configuration example of the image processing apparatus 10 according to the present embodiment. As shown in FIG. 3, the image processing apparatus 10 includes a main control unit 11 that performs numerical calculation and information processing based on various programs, and controls each part of the hardware. The main control unit 11 includes, for example, a CPU 11a as a central processing unit, a work memory 11b such as a RAM that functions as a work area when the main control unit 11 executes various programs, a startup program, an initialization program, and the like. And a program memory 11c such as a stored ROM, a flash ROM, or an EEPROM.

  In addition, the image processing apparatus 10 is obtained by imaging with the illumination control unit 12 for projecting the sine wave stripe pattern pattern with a phase shift on the work and the camera 30 in order to generate the above-described distance image. An image is displayed on an image input unit 13 composed of an ASIC (Application Specific Integrated Circuit) that captures image data, an operation input unit 14 that receives an operation signal from the console 50, and a monitor 40 such as a liquid crystal panel. A display control unit 15 composed of a display DSP, etc., a communication unit 16 communicably connected to an external PLC 70, PC 80, etc., and an arithmetic DSP that performs measurement processing such as edge detection and area calculation And an image processing unit 17 configured. The image input unit 13 may include a frame buffer for buffering image data, and the image processing unit 17 may include a memory for storing image data for measurement processing. The display control unit 15 may include a video memory such as a VRAM that temporarily stores image data when an image is displayed. These pieces of hardware are communicably connected via an electrical communication path (wiring) such as a bus.

  In the program memory 11c in the main control unit 11, the illumination control unit 12, the image input unit 13, the operation input unit 14, the display control unit 15, the communication unit 16, and the image processing unit 17 are stored by commands of the CPU 11a. A control program for controlling is stored. The above-described processing order program, that is, the processing order program generated in the PC 80 and transferred from the PC 80 is stored in the program memory 11c.

  The communication unit 16 functions as an interface (I / F) that receives an imaging trigger signal from the PLC 70 when a trigger is input from a sensor (photoelectric sensor or the like) connected to the external PLC 70. It also functions as an interface (I / F) that receives the image processing program of the image processing apparatus 10 transferred from the PC 80, the layout information that defines the display mode of the monitor 40, and the like.

  When receiving the imaging trigger signal from the PLC 70 via the communication unit 16, the CPU 11 a of the main control unit 11 sends an imaging command (command) to the image input unit 13. Further, a command for instructing image processing to be executed is transmitted to the image processing unit 17 based on the processing order program. In addition, as a device for generating an imaging trigger signal, a trigger input sensor such as a photoelectric sensor may be directly connected to the communication unit 16 instead of the PLC 70.

  The operation input unit 14 functions as an interface (I / F) that receives an operation signal from the console 50 based on a user operation. On the monitor 40, the user's operation content using the console 50 is displayed. More specifically, the console 50 is provided with various parts such as a cross key, a determination button, and a cancel button for moving the cursor displayed on the monitor 40 up, down, left, and right. As a result, the user creates a flowchart for defining the processing order of the image processing on the monitor 40, edits the parameter value of each image processing, or sets the reference inspection area as will be described later. Or setting detection conditions for a specific pattern.

  The image input unit 13 takes in image data. Specifically, for example, when an imaging command for the camera 30 is received from the CPU 11 a, an image data capture signal is transmitted to the camera 30. And after image pick-up with camera 30, image data obtained by image pick-up is taken in. The captured image data is temporarily buffered (cached) and substituted for an image variable prepared in advance. The “image variable” is a variable that is a reference destination for measurement processing and image display by being assigned as an input image of a corresponding image processing unit, unlike a normal variable that handles numerical values.

  The image processing unit 17 performs a measurement process on the image data. Specifically, first, the image input unit 13 reads image data from the frame buffer while referring to the image variables described above, and performs internal transfer to the memory in the image processing unit 17. Then, the image processing unit 17 reads out the image data stored in the memory and executes measurement processing.

  The display control unit 15 transmits a control signal for displaying a predetermined image (video) on the monitor 40 based on a display command (display command) sent from the CPU 11a. For example, a control signal is transmitted to the monitor 40 in order to display image data before or after the measurement process. The display control unit 15 also transmits a control signal for causing the monitor 40 to display the user's operation content using the console 50.

[Function configuration]
FIG. 4 is a diagram illustrating a functional configuration example of the image processing apparatus 10 according to the present embodiment.

  As shown in FIG. 4, the image processing apparatus 10 is configured to have each means in terms of software by various programs stored in the CPU 11a and the program memory 11c described above. Specifically, the image processing apparatus 10 includes a grayscale image acquisition unit 110, a distance calculation unit 120, a distance image generation unit 130, a detection condition setting reception unit 140, a specific pattern detection unit 150, and a reference inspection area setting. A receiving unit 160, an inspection area specifying unit 170, a feature amount calculating unit 180, and a determining unit 190 are provided.

  The grayscale image acquisition unit 110 is a unit that acquires a grayscale image in which each pixel has a grayscale value corresponding to the amount of light received by the camera 30. For example, it can be realized by the functions of the main control unit 11 and the image input unit 13 shown in FIG. The generated grayscale image may be stored in the frame buffer of the image input unit 13 described above, or may be stored in another memory. The distance calculation means 120 is a means for calculating the distance from the camera 30 to the workpiece surface using the image acquired from the camera 30, and the distance image generation means 130 is used for each pixel according to the calculated distance. A means for generating a distance image having a value. For example, it can be realized by the main control unit 11, the illumination control unit 12, the image input unit 13, and the like shown in FIG. 3. The generated distance image may be stored in the frame buffer of the image input unit 13 described above or may be stored in another memory. Further, in the present embodiment that employs the phase shift method to generate a distance image, the illumination control unit 12 controls the illumination 60 so as to project a sinusoidal fringe pattern with a phase shift on the work, and the image The input unit 13 controls the camera 30 so as to capture a plurality of images in which the phase of the sine wave stripe pattern is shifted accordingly. And the image input part 13 calculates | requires the phase of a sine wave for every pixel from several images, and produces | generates a distance image using the calculated | required phase.

  In the present embodiment, the image input unit 13 performs the distance image generation process. However, for example, the main control unit 11 and the image processing unit 17 illustrated in FIG. 3 can also perform the distance image generation process. . In the present embodiment, the camera that acquires the grayscale image by the grayscale image acquisition unit 110 and the camera that calculates the distance to the workpiece surface by the distance calculation unit 120 are configured by the same camera 30. Thereby, since the gray image and the distance image can be obtained for the same region of the same work, it is not necessary to separately correct the position of the entire gray image and the entire distance image, and the processing load of the measurement process can be reduced. If the camera that acquires the grayscale image by the grayscale image acquisition unit 110 and the camera that calculates the distance to the workpiece surface by the distance calculation unit 120 are configured as separate cameras, from the positional relationship between these cameras. The correspondence between the position on one camera image and the position on the other camera image is obtained for an arbitrary part of the workpiece, and the position correction (calibration correction) may be performed using the correspondence. In this embodiment, the grayscale image acquisition unit 110 and the distance image generation unit 130 are separated from each other. However, it is of course possible to combine them in the form of one image acquisition unit or one image generation unit.

  The detection condition setting accepting unit 140 is a unit that accepts setting of a detection condition for detecting a specific pattern for specifying an inspection area corresponding to the inspection range of the workpiece. For example, it can be realized by the functions of the monitor 40 and the display control unit 15, the console 50 and the operation input unit 14, the main control unit 11, and the like shown in FIG. In the present embodiment, an alignment mark (such as a cross mark) for positioning the workpiece is assumed as the specific pattern. However, the present invention is not limited to this, and includes all patterns for specifying an inspection area corresponding to the inspection range of a workpiece. Specifically, as will be described later in the second embodiment, an edge pattern including an edge extracted from a distance image can be assumed. Various conditions can be considered as detection conditions for detecting the specific pattern. For example, the type of filter used for preprocessing (binarization filter, expansion filter, contraction filter, averaging filter, median filter, edge enhancement filter, Sobel filter, Laplacian filter, Gaussian filter, etc.) and the threshold used for detection The magnitude of the edge strength (gradation change) as a value (upper limit or lower limit of the edge strength) can be considered. Further, the detection condition in the present embodiment includes a pattern area set by the user so as to include at least a part of the specific pattern (the setting of the pattern area will be described later). In the present embodiment, user-desired preprocessing among the above-described preprocessing is performed on the set pattern area (the preprocessing can be omitted), and information on the above-described edge strength (the size of the edge strength and the edge Angle, etc.) are stored in advance as detection conditions. At this time, it is preferable to also store the upper and lower limit values of the edge strength that can be regarded as the feature amount representing the specific pattern. As described above, in the image processing apparatus 10 according to the present embodiment, information such as edge strength necessary for detecting a specific pattern is stored as a detection condition, so that generally large-capacity image data is stored. There is no need to keep it. As a result, the memory resources can be effectively utilized (utilized efficiently). Note that it is also possible to store image data as detection conditions stored in the detection condition storage means. For example, data of a reference image for detecting a specific pattern (which may be a reference image stored in advance in the image processing apparatus or may be a reference image input during a setting operation) may be stored. That is, when a specific pattern is detected by a normalized correlation calculation, a reference image used for the normalized correlation calculation is required. Therefore, a predetermined reference image can be set as a detection condition and stored. If the reference image can be read as appropriate, the pattern area once set can be freely edited.

  As shown in FIG. 4, the detection condition setting accepting unit 140 according to the present embodiment sets detection conditions from the user for the distance image generated by the distance image generating unit 130, that is, the above-described preprocessing type and edge. Although the magnitude of the intensity is received, it is of course possible to receive the setting of the detection condition for the gray image acquired by the gray image acquisition unit 110. For example, when the detection condition is fixed in the image processing apparatus 10, it is possible to accept the setting of the detection condition from the user, that is, to omit the detection condition setting accepting unit 140.

  The specific pattern detection unit 150 is a unit that detects a specific pattern on the grayscale image based on the detection condition set by the detection condition setting reception unit 140. For example, it can be realized by the functions of the main control unit 11 and the image input unit 13 shown in FIG.

  The reference inspection area setting receiving unit 160 includes a reference inspection area having a certain relative positional relationship with a specific pattern for specifying the inspection area corresponding to the inspection area of the workpiece in the grayscale image acquired by the grayscale image acquiring unit 110. It is a means for accepting settings. For example, it can be realized by the functions of the monitor 40 and the display control unit 15, the console 50 and the operation input unit 14, the main control unit 11, and the like shown in FIG. The reference inspection area has a certain relative positional relationship with the specific pattern. For example, the reference inspection area is separated by X pixels on the left and right and Y pixels on the top and bottom on the grayscale image, or the specific pattern (as will be described later in this embodiment). Relative to the pattern area) by a predetermined angle θ (generally 0 degrees). That is, the relative positional relationship in this embodiment includes not only the relative positional relationship in the X-axis direction and the Y-axis direction but also the relative angular relationship.

  The inspection region specifying unit 170 is a unit that specifies an inspection region corresponding to the inspection range of the workpiece based on at least one of the position and the inclination angle of the specific pattern detected by the specific pattern detection unit 150. For example, it can be realized by the functions of the main control unit 11 and the image processing unit 17 shown in FIG. In the present embodiment, the position of the reference inspection area set by the reference inspection area setting receiving means 160 is corrected based on at least one of the position and the inclination angle of the specific pattern detected by the specific pattern detection means 150, thereby performing inspection. The area is specified. The specification of the inspection area will be described in more detail with reference to FIG.

  The feature amount calculation unit 180 is a unit that calculates a feature amount from the specified inspection region, and the determination unit 190 is a unit that determines the quality of the workpiece based on the calculated feature amount. These can be realized by, for example, functions of the main control unit 11 and the image processing unit 17 shown in FIG. There are various ways of calculating the feature amount by the feature amount calculation unit 180. For example, an “area” for measuring the area of the measurement target detected from the inspection region (by binarization or the like), a specific pattern "Pattern search" to detect edge, "Edge position" and "Edge width" to measure edge position and width, "Scratch" to detect scratches by shading change, For pixels with the same density in binary image “Blob” that measures the number, area, center of gravity, etc. of the blob that is a set. Based on the calculated feature value, for example, the determination unit 190 determines OK when there is no blob larger than a predetermined area, and NG determination when there is a blob larger than a predetermined area. Judgment is made. Further, if a specific pattern exists by pattern matching, OK determination is performed, and conversely, NG determination is performed if it does not exist. The determination result is output as a determination signal by the communication unit 16 to an external device (PLC 70 or the like).

[Specify inspection area]
5-12 is explanatory drawing regarding a mode that an inspection area | region is specified in the image processing apparatus 10 which concerns on this embodiment. 5 to 12 are a part of screens displayed on the monitor 40. FIG.

  First, using FIG. 5 to FIG. 7, the specification of the inspection area in the conventional appearance inspection using only the conventional grayscale image will be described. As shown in FIG. 5, the grayscale image acquired via the camera 30 includes an alignment mark M <b> 1 composed of a cross mark and a character “ABC” to be inspected. The alignment mark M1 here is an example of the so-called specific pattern described above. The user uses the console 50 to set the pattern region R1 so as to surround the alignment mark M1. In addition, a reference inspection region, that is, a reference inspection region R2 is set so as to surround the character “ABC” to be measured. At this time, the reference inspection region R2 has a certain relative positional relationship with the alignment mark M1 in the pattern region R1, and this relative positional relationship is stored in the image processing apparatus 10 (for example, the main control unit 11 or the image processing unit 17). The The relative positional relationship between the reference inspection region R2 and the pattern region R1 may be constant.

  Next, during the operation of the image processing apparatus 10, the position and posture of the work may be shifted, and as a result, “ABC” may deviate from the position shown in FIG. 5 on the grayscale image. Even in such a case, if the alignment mark M1 is detected, the position can be corrected and the inspection area can be specified. For example, as shown in FIG. 6, even if the work is translated in the upper right direction in the figure on the grayscale image, the position of the alignment mark M1 (pattern region R1 ′) translated in the upper right direction is detected. Thus, based on the detected position of the alignment mark M1, the inspection region R2 ′ obtained by correcting the position of the reference inspection region R2 can be specified. Further, for example, as shown in FIG. 7, even if the work moves parallel to the right side of the figure on the grayscale image, moves slightly upward, and rotates counterclockwise by about 20 degrees, it moves accordingly. The position of the alignment mark M1 (pattern region R1 ″) detected can be detected, and the inspection region R2 ″ obtained by correcting the position of the reference inspection region R2 can be specified based on the detected inclination angle of the alignment mark M1.

  However, when the alignment mark M1 cannot be detected, it is difficult to specify the inspection area. For example, as shown in FIG. 8, the alignment mark M1 is formed with unevenness (for convenience of explanation, the outline portion of the alignment mark M1 formed with unevenness is indicated by a thick dotted line. The alignment mark M1 hardly appears on the gray image shown in FIG. In this case, for example, even if the pattern region R1 and the reference inspection region R2 as shown in FIG. 8 are set, the inclination angle of the position of the alignment mark M1 that moves to various positions is detected during the operation of the image processing apparatus 10. I can't. As a result, for example, as shown in FIG. 9, the position and inclination angle of the alignment mark M1 cannot be detected, and the position of the reference inspection region R2 cannot be corrected.

  Therefore, in the image processing apparatus 10 according to the present embodiment, the user can set the pattern region R1 so as to surround the alignment mark M1 in the distance image instead of the grayscale image. The manner in which the inspection area is specified in the image processing apparatus 10 according to the present embodiment will be described with reference to FIGS.

  As shown in FIG. 10, even when the contour portion of the alignment mark M1 is formed with irregularities, it clearly appears in the distance image. This is because the distance image has a gray value corresponding to the distance between the workpiece and the camera 30. In FIG. 10, the color of the recessed portion in the contour portion of the alignment mark M <b> 1 is lighter. In the distance image shown in FIG. 10, the user uses the console 50 to set the pattern region R1 so as to surround the alignment mark M1. Note that the character “ABC” to be measured is a character having a planar shape only printed on a workpiece and is not an uneven character, and therefore does not appear in the distance image. The reference inspection region R2 is set in a grayscale image in which the character “ABC” appears clearly as shown in FIG.

  When the operation of the image processing apparatus 10 is actually started after such setting work is completed, the alignment mark M1 is detected by using a distance image in which the alignment mark M1 clearly appears as shown in FIG. Can do. Then, by calculating the position and inclination angle (difference) between the alignment mark M1 shown in FIG. 10 and the alignment mark M1 detected during the operation shown in FIG. 11, the image processing apparatus 10 (image processing unit 17). Can recognize how much the position of the reference inspection region R2 shown in FIG. 8 should be corrected and how much the angle should be corrected. Therefore, as shown in FIG. 12, the position-corrected inspection region R <b> 2 ″ can be specified this time on the grayscale image. In the present embodiment, the pattern region R1 shown in FIG. 10 is set in FIG. 8, but this need not be performed in FIG. Conversely, in FIG. 10, the reference inspection region R2 shown in FIG. 8 is not set, but this may be performed in FIG. In short, the pattern area should be set at least on the distance image, and the reference inspection area should be set at least on the distance image. Further, in the present embodiment, when setting the pattern region, it is set so as to surround all of the alignment marks M1, but may be set so as to surround a part of the alignment marks M1.

  13 to 19 are diagrams illustrating examples of user interface screens displayed on the monitor 40 in the image processing apparatus 10 according to the present embodiment.

  As shown in FIG. 13, a setting edit 1001 is displayed inside the window name “setting 0000”. When the user selects the “window” button using the console 50, a further window named “add window” appears on the right side of the button. In this window, a window number 1002 and a measurement method 1003 are displayed. Here, “000” is selected as the window number 1002 and “pattern search” is selected as the measurement method 1003, and the OK button below it is clicked. Then, as shown in FIG. 14, a window detailed setting screen including window number “000” and window name “pattern search” appears (“W000: pattern search” is displayed on the upper bar of this setting screen. ) On this detailed setting screen, items desired by the user can be selected for each of the use image 1006, the search area 1007, the pattern area 1008, the pre-processing 1009, and the detailed setting 1010. The use image 1006 means an image used when setting the detection condition for detecting the alignment mark M1, and for example, an input image or a registered image can be selected by a check box. In FIG. 14, the registered image is checked, but this is a case where an image stored in advance in the image processing apparatus 10 is used. The registered image can be saved from an “image registration” button in the setting editing 1001 (details of the setting method are omitted). If a registered image is used as the use image, not only the pattern area and search area can be set, but also the pattern area and search area once set can be freely edited later. On the other hand, when the use image is checked, every time an attempt is made to set a pattern area or a search area, a grayscale image or a distance image must be acquired from the camera 30, but image registration becomes unnecessary. Memory resources can be used effectively. In addition, the user can select a grayscale image and a registered image using check boxes, and a distance image is selected in FIG. Therefore, first, the setting related to the distance image is performed. In addition, the user can select an image number, and when there are a plurality of input images and registered images, the user can select which image is the use image 1006. In FIG. 14, “000-2” is selected, which means the 000th registered image. “2” after the hyphen indicates that the registered image is a distance image of the grayscale image and the distance image. The search area 1007 and the pattern area 1008 will be described later. The preprocessing 1009 is preprocessing for facilitating extraction of the feature amount of the alignment mark M1 from the set pattern region R1 (see FIG. 10). For example, which filter is used among the various filters described above. Here, the “gaussin filter” is selected to be used. The detailed setting 1010 is another detailed setting, and includes, for example, the size of the edge strength described above.

  Next, in FIG. 14, when the OK button is clicked with the search area 1007 and the pattern area 1008 checked, the screen is switched to the screen shown in FIG. A search area 1007 indicates an area to be searched for the alignment mark M1 in the above-described registered image No. 000 (distance image). The pattern region 1008 is a region that is set so as to surround at least a part of the alignment mark M1 and is set for detecting the alignment mark M1. In FIG. 15, the search area S1 has already been set, and the pattern area R1 (also referred to as “window W000” in the present embodiment) is set so as to surround the entire alignment mark M1. When the setting of the search area 1007 and the pattern area 1008 is completed in this way and the OK button is clicked, the process for extracting the feature amount of the alignment mark M1 is performed as described above. Specifically, the main control unit 11 performs, for example, Gaussian filter processing to reduce noise and perform edge processing in the area of the window W000 of the registered image of the number 000, and sets the set edge strength. The position to be regarded as an edge is recognized, projection processing is performed in the X direction or Y direction as necessary, and the processing result is stored in the work memory 11b or the like as the feature amount of the alignment mark M1. As described above, in this embodiment, the image data in the window W000 is not stored, but the feature amount of the alignment mark M1 in the window W000 is stored. Thereby, the used memory capacity can be reduced. Note that the image data in the window W000 may be stored in the work memory 11b or the like.

  Next, in FIG. 14 again, this time, the gray image is selected, and the search area and the pattern area are set as described above. For example, as shown in FIG. 16, a pattern region R1 (also referred to as “window W001” in this embodiment) is set so as to surround “ABC” to be inspected. Thereby, the area surrounded by the window W001 becomes the reference inspection area. In other words, the user sets the pattern region R1 and repeats the operation of setting a window on the image (as a result, by setting a plurality of windows in the grayscale image or the distance image), as a specific pattern. Region setting for detecting the feature amount of the alignment mark M1 can be performed (W000 in FIG. 15), or region setting for detecting a character or the like to be inspected can be performed (FIG. 15). (W0001 in FIG. 16).

  Next, as shown in FIG. 17, position correction is selected from the setting editing 1001, and correction source 1011 and correction destination 1012 windows are selected. The correction source 1011 is a window serving as a reference when performing position correction, and the correction destination 1012 is a window subjected to position correction using the window of the correction source 1011. In FIG. 17, the window W000 is selected as the correction source 1011 and the window W0001 is selected as the correction destination 1012. In addition, a check box for selecting whether to perform position correction in the X direction, the Y direction, or the θ direction (rotation direction) is displayed between the window W000 and the window W001. In FIG. 17, since all the check boxes are checked, it is set to perform horizontal position correction and inclination angle correction.

  Finally, an output setting is selected from the setting editing 1001, and a determination target window is selected in the determination window 1013 displayed on the right side. That is, the window to be handled as the reference inspection area is selected from the set windows W000 and W001. In FIG. 18, the window W001 (set on the gray image) is selected as the reference inspection area. If the window W000 (set on the distance image) is selected as the reference inspection area, the character “ABC” that was the measurement object becomes the alignment mark, and the alignment mark M1 becomes the measurement object (inspection object). . In the upper right of FIGS. 13 to 18, “setting” 1004 is displayed, which indicates that the setting work before starting the operation of the image processing apparatus 10 is being performed. . That is, it can be said that the image processing apparatus 10 is an apparatus that has both a setting function for accepting various setting operations necessary for appearance inspection from the user or the PLC 70 and a driving function for determining the quality of the workpiece.

  FIG. 19 is a diagram illustrating an example of a user interface screen displayed on the monitor 40 during driving in the image processing apparatus 10 according to the present embodiment. In the upper right of FIG. 19, “1015 during operation” is displayed. When this display is made, the image processing apparatus 10 captures a workpiece flowing at a high speed on the production line with the camera 30, generates a distance image, specifies an inspection region (position correction), and performs various measurement processes. A series of processes of performing the above are performed at high speed, for example, at intervals of several tens of ms for a plurality of works. In addition, for example, by pressing a changeover switch (not shown) of the console 50 or by receiving an external signal from the PLC 70 or the like, the display 1015 during operation is displayed from the setting mode in which the setting 1004 is displayed. The operation mode can be switched.

  As shown in FIG. 19, a setting number is displayed at the upper left, and a measurement number, an NG number, a measurement time, a trigger interval, and a determination window 1013 are displayed below the setting number. This determination window 1013 is set to W001 and is a window set using FIG. A grayscale image acquired from the camera 30 is displayed from the center to the lower right on the screen. On the other hand, the generated distance image is not displayed, but the alignment mark M1 is detected on the back side, and the position of the reference inspection region R2 is corrected based on the detected position and inclination angle of the alignment mark M1, The inspection area R2 ″ is specified. In this way, according to the image processing apparatus 10 according to the present embodiment, it is possible to stably specify the inspection region.

  In FIG. 19, only the grayscale image is displayed. However, the grayscale image and the distance image may be displayed separately in one screen by operating the display switching 1014 button or the like. . Alternatively, the display of the grayscale image and the distance image may be switched alternately by the button operation of the display switching 1014. Here, the inspection method for both W000 and W001 is a pattern search, but W000 may be a scratch inspection, for example.

  FIG. 20 is a flowchart showing the processing operation of the image processing apparatus 10 according to the present embodiment.

  As shown in FIG. 20, the grayscale image acquisition unit 110 performs the grayscale image acquisition process (step S1), while the distance calculation unit 120 and the distance image generation unit 130 perform the distance image generation process (step S2). Is called. In this flowchart, the processing is performed simultaneously in parallel, but serial processing may be performed, for example, a distance image is generated after a grayscale image acquisition process.

  Next, the specific pattern is detected by the specific pattern detecting means 150 (step S3). In the present embodiment, the specific pattern is detected according to the detection conditions (pattern region, edge strength, etc.) set by the detection condition setting receiving unit 140 (see FIGS. 14 and 15).

  Next, the inspection area is specified by the inspection area specifying means 170 (step S4). In the present embodiment, the inspection region is specified according to the reference inspection region set by the reference inspection region setting receiving unit 160 and the alignment mark M1 (see FIG. 16). That is, in this embodiment, the reference inspection area set by the reference inspection area setting receiving unit 160 is position-corrected and angle-corrected based on the position and inclination angle of the specific pattern (alignment mark M1) detected in step S150. The

  Then, the feature amount calculation unit 180 calculates the feature amount (step S5), the determination unit 190 determines the quality of the workpiece (step S6), and the series of processing ends. Such a series of processing is performed for each workpiece flowing through the production line. That is, the processing operation shown in FIG. 20 is repeatedly executed at a high speed at a predetermined interval (for example, several tens of ms).

  As described above, for example, even when the alignment mark M1 as the specific pattern is configured by slight unevenness by pressing (see FIG. 8), the alignment mark can be detected in the distance image where the unevenness can be detected. Since M1 can be detected (see FIG. 19), the inspection area can be specified on the grayscale image based on at least one of the position and the inclination angle of the detected alignment mark M1, and thus stably inspected. An area can be specified.

[Modification]
In the above-described embodiment, an example in which the alignment mark M1 cannot be detected on the grayscale image because the alignment mark M1 is configured by slight unevenness by pressing has been described. For example, as illustrated in FIG. There may be a case where it cannot be detected on the distance image. FIG. 21 is a diagram showing a specific application example when the grayscale image acquisition unit 110, the distance calculation unit 120, and the distance image generation unit 130 are replaced in the functional configuration shown in FIG. Specifically, as shown in FIG. 21A, the alignment mark M2 is printed on the left end of the workpiece (to an extent that the unevenness can be ignored), and is formed at the correct position near the center of the workpiece. There are four convex portions to be inspected (whether or not they are present). In such a gray image of the workpiece, as shown in FIG. 21B, the alignment mark M2 appears clearly, but the four convex portions do not appear clearly. However, as shown in FIG. 21C, in the distance image, the alignment mark M2 does not appear at all, but the four convex portions appear clearly. Accordingly, in the grayscale image shown in FIG. 21B, the pattern region R1 as described with reference to FIG. 15 is set, and in the distance image shown in FIG. 21C, the reference as described with reference to FIG. Inspection areas R2-1 to R2-4 are set. Thereby, even if the position of the workpiece is shifted during the operation of the image processing apparatus 10, the alignment mark M2 is detected from the grayscale image, and the reference inspection areas R2-1 to R2-4 are based on the detected alignment mark M2. Can be corrected.

  Next, although the alignment mark is formed on the workpiece in the above-described embodiment, the present invention is not limited to this, and a case where the alignment mark is not formed on the workpiece is assumed as another modification. FIG. 22 is a diagram illustrating a specific example of an application that specifies an inspection region without using an alignment mark. Specifically, as shown in FIG. 22A, it is assumed that the character “ABC” to be inspected is printed on the upper stage (to the extent that the unevenness can be ignored) in the upper and lower works. In such a gray-scale image of the work, as shown in FIG. 22B, the character “ABC” appears clearly, but the upper and lower step portions do not appear (the influence of shadows and the like due to the step is ignored). In addition, R2 in Fig. 22B is a window indicating the reference inspection area displayed superimposed on the grayscale image, and the stepped portion does not appear as an image). On the other hand, in the distance image, as shown in FIG. 22C, the upper and lower step portions appear clearly as light and shade of the image. Therefore, even if the pattern region R1 described with reference to FIG. 15 is not set in advance, edge extraction is performed on the distance image shown in FIG. 22C, and a pattern including the extracted edge is defined as the edge pattern R1. It is also possible to specify the inspection region R2 by generating a region corresponding to the edge pattern R1 on the grayscale image shown in FIG. In short, it is also possible to specify the inspection area by generating the corresponding area based on the extracted edge pattern, instead of correcting the position of the preset reference inspection area as in the above-described embodiment. is there.

  FIG. 23 is a diagram illustrating another functional configuration example of the image processing apparatus 10 according to the present embodiment. The main difference from FIG. 4 is that the edge detection means 200 exists.

  The edge detection means 200 is a means for extracting an edge from a distance image in FIG. In this case, the edge extraction condition includes edge strength and the like, but the detection condition set by the detection condition setting receiving unit 140 can be used. Of course, by storing the edge extraction conditions in the image processing apparatus 10 in advance, it is possible to save the user from setting the detection conditions through the detection condition setting receiving unit 140.

  The inspection region specifying unit 170 generates a region R2 corresponding to the edge pattern on the grayscale image based on at least one of the position and the inclination angle of the edge pattern R1 including the edge detected by the edge detecting unit. This makes it possible to specify the inspection region R2. As described above, it is also possible to stably specify the inspection area without setting the reference inspection area.

  In FIG. 24, based on the functional configuration shown in FIG. 23, the reference inspection area setting receiving unit 160 does not set the reference inspection area, and the detection condition setting receiving unit 140 sets the pattern area of the detection conditions. It is an application specific example (appearance inspection example) showing a state in which an inspection region is specified (inspection region is generated) without any inspection.

  As shown in FIG. 24 (a), when the work has a two-step plane composed of the upper step surface A1 and the lower step surface A2, dirt is attached to both the upper step surface A1 and the lower step surface A2. It is desired to perform an appearance inspection only for whether or not dirt is attached to A1 (that is, to perform an appearance inspection that allows the dirt to adhere to lower surface A2). In such a case, as shown in FIG. 24B, it can be clearly seen that two spots of dirt are attached to the gray image of the work, whereas as shown in FIG. The distance image does not show any contamination. Instead, the upper surface A1 and the lower surface A2 appear clearly due to the difference in shading. As shown in FIG. 24D, if the workpiece is displaced as shown in FIG. 24D and the position of the upper step surface A1 moves toward the upper right on the grayscale image, as shown in FIG. In addition, the position of the upper surface A1 also moves to the upper right in the distance image.

  Therefore, as described with reference to FIGS. 22 and 23, the edge is detected from the distance image, and the region corresponding to the edge pattern on the grayscale image based on the position and the inclination angle of the edge pattern based on the extracted edge. As shown in FIG. 24F, the inspection region R3 can be identified stably. If the inspection area R3 can be identified stably, for example, by performing the feature amount calculation process in the inspection area R3 such as the blob calculation described above, the appearance inspection is performed only on whether or not the upper surface A1 is contaminated. Can do. Although the edge is detected from the distance image here, it is needless to say that the edge may be detected from the grayscale image.

  Here, an example of a method for generating an edge pattern from a distance image or a grayscale image will be described with reference to FIG. FIG. 25 is an explanatory diagram for describing an example of a method for generating an edge pattern from a distance image or a grayscale image.

  As shown in FIG. 25A, the edge detection areas E1 to E4 are set on the image. In the edge detection area, a window is set in an area that the user thinks includes an edge through the detection condition setting reception unit 140. As a result, the position information of the edge detection areas E1 to E4 is included in the detection conditions. During operation of the image processing apparatus 10, as shown in FIG. 25B, edges are detected in the edge detection areas E1 to E4, and a straight line including four edges is estimated based on the detected edges. (FIG. 25C). A square frame based on these four straight lines can be used as an edge pattern (FIG. 25D). In this way, an edge pattern can be generated. In the application example of FIG. 24, the image area inside the edge pattern generated from the distance image shown in FIG. 24E is the inspection target of the appearance inspection. For example, the distance image shown in FIG. An image area outside the edge pattern generated from the above may be used as an inspection target for appearance inspection. In this case, it is possible to inspect whether or not dirt is attached to the lower surface of the work.

FIG. 26 is an example of an application that performs an appearance inspection using both a grayscale image and a distance image. More specifically, this is an example in which an appearance inspection of a ball grid array (BGA) is performed. FIG. 26A is a BGA grayscale image, and FIG. 26B is a BGA distance image. As shown in FIG. 26 (a), an alignment mark (cross) appears clearly in the upper left corner of the work in the grayscale image, while in the distance image, the alignment mark appears in the grayscale image. Does not appear clearly. This is because the alignment mark is a planar cross mark. In such a case, as shown in FIG. 26C, the coordinates (x _p , y _p ) of the alignment mark are detected using the grayscale image. Then, as specifying the inspection area in each embodiment described above, when considering an inspection area of each convex portion of the BGA, the correction is based on the coordinate positions of the respective convex portions of the BGA coordinates (x _{p, y p)} in be able to. When the coordinate position of the corrected convex part of the BGA is (x _b , y _b ), (x _b , y _b ) − (x _p , y _p ) = (x ₀ , y ₀ ) is calculated. At the same time, the height z of the convex portion is calculated from the distance image based on the gray value at the coordinate position (x _b , y _b ). If the calculated (x ₀ , y ₀ ) and the height z are within a predetermined range, it is determined that the workpiece is non-defective. In this way, the position of the inspection area is stably corrected using both the grayscale image and the distance image, and whether or not each convex portion of the BGA is at the correct position and at the correct height. It becomes possible to perform a special appearance inspection such as.

  As described above, according to the image processing apparatus 10 according to the present embodiment, it is possible to perform an appearance inspection using both a grayscale image and a distance image. It should be noted that, for example, it may be possible to select which of the grayscale image and the distance image is used for the appearance inspection. FIG. 27 is a diagram illustrating an example of a selection screen for calculating at least one of the first and second feature amounts by the feature amount calculation unit 180 (for example, using the console 50 on the monitor 40). Selectable). As shown in FIG. 27, when the determination based on the feature value obtained from the window W000 is OK and the determination based on the feature value obtained from the window W001 is OK, whether the OK determination is issued (second step from the top in FIG. 27). ) When the determination based on the feature value obtained from the window W000 is OK and the determination based on the feature value obtained from the window W001 is NG, whether to make an OK determination (third stage from the top in FIG. 27) or from the window W000 When the determination based on the obtained feature quantity is NG and the judgment based on the feature quantity obtained from the window W001 is OK, the user can select whether to issue the OK judgment (fourth row from the top in FIG. 27). Also good.

  That is, the inspection region specifying unit 170 specifies the inspection region based on at least one of the position and the inclination angle of the specific pattern detected by the specific pattern detection unit 150 in both the grayscale image and the distance image. The amount calculation unit 180 calculates the first and second feature amounts from the inspection regions specified in the grayscale image and the distance image, respectively, and the determination unit 190 sets at least one of the first and second feature amounts. Based on this, the quality of the workpiece may be determined.

  In particular, if the quality of the workpiece is determined based on the first and second feature amounts as described above, a special appearance inspection can be performed. An example different from FIG. 26 will be described.

  For example, when considering a special appearance inspection that allows some dents in the workpiece (I want to make an OK determination) but does not allow contamination of the workpiece (I want to make an NG decision), an appearance inspection using only a gray image Then, due to the influence of the shadow generated in the dent portion, even if the dent portion is not actually dirty, it may appear that the dent portion is stained on the grayscale image. At this time, only the first feature amount (for example, the area of the dark portion) calculated from the inspection region specified on the grayscale image is determined as NG not only for the actually dirty portion but also for the above-described recessed portion. Therefore, some dents in the workpiece cannot be allowed. On the other hand, the contamination of the work cannot be determined as NG in the first place only by the second feature amount (for example, the area of the dark portion) calculated from the inspection region specified on the distance image. Therefore, according to the image processing apparatus 10 according to the present embodiment, one or a plurality of inspection areas in the inspection area that cause NG determination based on the first feature amount calculated from the inspection area specified on the grayscale image. While extracting the 1st specific location (namely, dirty location) and based on the 2nd feature-value calculated from the test area specified on a distance image, the position in the inspection field which becomes a factor of NG determination, or two or more It is possible to extract the second specific part (that is, the recessed part). Therefore, if the first specific location still remains after subtracting the second specific location from the first specific location, the remaining first specific location becomes a dirty but not recessed portion. By making an NG determination when the first specific location exists, an appearance inspection is performed to allow a slight dent of the workpiece (to make an OK determination) but not to allow the workpiece to be dirty (to make an NG determination). be able to.

  FIG. 28 is a diagram illustrating a specific example of an application that performs quality determination of a workpiece using both the first and second feature amounts calculated by the feature amount calculation unit 180.

  As shown in FIG. 28 (a), the upper surface of the work has dirt and dents. The gray image at this time is as shown in FIG. 28 (b), and the distance image at this time is as shown in FIG. 28 (c). On the grayscale image shown in FIG. 28 (b), it appears that the dents are stained with shadows. When the workpiece is displaced, the result is as shown in FIG. 28 (d) and FIG. 28 (e), respectively. Here, the inspection region R4 on the grayscale image is specified based on the edge pattern of the distance image, and the inspection region R5 is also specified on the distance image (FIG. 28 (f), FIG. 28 (g)). Then, if blob processing or the like is performed in the inspection region R4 and the inspection region R5, as shown in FIG. 28 (h) and FIG. 28 (i), the change in shading can be detected. Since the change in light and darkness detected in the region R5 is due to the dent, the corresponding dirt portion is removed from the light and dark image. As a result, as shown in FIG. 28 (j), the change in density (blob) still remains, so that it can be seen that the work is contaminated, and an NG determination is made. As described above, it is possible to perform an appearance inspection that allows a slight dent of the workpiece (want to make an OK determination) but does not allow the workpiece to be dirty (want to make an NG determination). The image processing apparatus 10 is provided with execution condition setting accepting means (consisting of the console 50, the operation input unit 14, the main control unit 11 and the like) for accepting the setting of the image processing execution condition in the appearance inspection from the user. When the inspection condition (image processing) in a certain window is OK, the execution condition that inspection (image processing) is performed in another window, or the inspection (image processing) in a certain window is NG Alternatively, the user may be able to set an execution condition such that the subsequent inspection is not performed and the inspection is terminated. In the example of FIG. 28, when an NG determination is made in the inspection region R5 using the distance image, the blob at the corresponding position in the inspection region R4 based on the position of the specific blob that is the target of the NG determination. May be deleted (ignored), and if a blob is detected in other cases (see FIG. 28 (i)), an execution condition such as performing an NG determination may be set. In addition, various contents are conceivable as the execution condition setting contents.

  As another application example, in a distance image, a portion having a certain height or more may be considered as a measurement region. As a result, even in a product that dynamically changes or deforms, or in a product that is unstable and noisy on a grayscale image, it is possible to set the inspection region to the edge of the contour. For example, when an unevenness inspection or a foreign matter inspection (shading change) is performed by image processing on a product whose product contour is not vertical but has an R surface, the contour is determined by setting a region having a certain height or more as the inspection region. It is possible to measure to the last minute including the vicinity. In addition, according to the setting that all of a certain height or more in the screen is set as the measurement region, without worrying about the positional deviation in the two-dimensional space, and without specifying the inspection region with the two-dimensional coordinates, It becomes possible to perform inspection in the target area. As a setting procedure, for example, (1) a parameter of a certain height or more (a certain gradation or more) on the distance image is set by the console 50 or the like, and (2) a portion having a certain height or more in the screen. (3) Perform a concavo-convex inspection by height information and a foreign matter (shading change) inspection by shading information, using a cut-out portion of a certain height or more as an inspection area. (4) Fix the cutting area. Instead, a portion of a certain height or more is cut out as a region for each inspection, so that a dynamic product shape change can be followed as the inspection region. When it is not necessary to follow each time, an area cut out from the master image (reference distance image) is used as a template and can be used as an inspection area.

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 11 Main control part 12 Illumination control part 13 Image input part 14 Operation input part 15 Display control part 16 Communication part 17 Image processing part 30 Camera 40 Monitor 50 Console 60 Illumination 70 PLC
80 PC
DESCRIPTION OF SYMBOLS 110 Gray image acquisition means 120 Distance calculation means 130 Distance image generation means 140 Detection condition setting reception means 150 Specific pattern detection means 160 Reference inspection area setting reception means 170 Inspection area specification means 180 Feature quantity calculation means 190 Determination means

Claims (8)

In an image processing apparatus that has a camera for imaging a workpiece, inspects a predetermined inspection range on the surface of the workpiece based on an image acquired from the camera, and determines whether the workpiece is good or bad,
Each pixel has a grayscale image acquisition means for acquiring a grayscale image having a grayscale value corresponding to the amount of light received by the camera;
Distance calculation means for calculating a distance from the camera to the workpiece surface using an image acquired from the camera;
A distance image generating means for generating a distance image in which each pixel has a gray value corresponding to the calculated distance;
Specific pattern detection means for detecting a specific pattern for specifying an inspection area corresponding to the inspection range on the other image in one of the grayscale image and the distance image;
In the other image of the grayscale image and the distance image, a reference inspection region setting receiving unit that receives a setting of a reference inspection region that is in a certain relative positional relationship with the specific pattern;
Based on at least one of the position and the inclination angle of the specific pattern detected in one of the gray image and the distance image by the specific pattern detection means, the specific pattern is set in the other image of the gray image and the distance image. Inspection area specifying means for specifying the inspection area by correcting the position of the reference inspection area ;
A feature amount calculating means for calculating a feature amount from the specified inspection region;
An image processing apparatus comprising: a determination unit that determines the quality of the workpiece based on the calculated feature amount. Edge extraction means for extracting an edge from one of the grayscale image and the distance image;
The inspection area specifying unit generates an area corresponding to the edge pattern on the other image based on at least one of the position and the inclination angle of the edge pattern based on the edge extracted by the edge extracting unit, The image processing apparatus according to claim 1, wherein the inspection area is specified. Detection condition setting accepting means for accepting setting of a detection condition for detecting the specific pattern;
Detection condition storage means for storing the received detection condition,
The specific pattern detection and output means detects a specific pattern for specifying an inspection area corresponding to the inspection range on the other image based on the detection condition stored in the detection condition storage means. The image processing apparatus according to claim 1 or 2 . The image processing apparatus according to claim 3 , wherein the detection condition includes at least an edge strength for detecting the specific pattern. The camera that acquires a grayscale image having a grayscale value corresponding to the amount of received light by the grayscale image acquisition unit and the camera that calculates the distance to the workpiece surface by the distance calculation unit are the same camera. the image processing apparatus according to any one of claims 1 to 4. The inspection area specifying unit specifies the inspection area based on at least one of a position and an inclination angle of the specific pattern detected by the specific pattern detection unit in one of the grayscale image and the distance image. ,
The feature amount calculating means calculates first and second feature amounts from the inspection regions specified in the grayscale image and the distance image, respectively.
Said determining means, said first and second on the basis of at least one of the features of the image processing apparatus according to any of claims 1, characterized in that to determine the quality of the work 5. The image processing apparatus according to claim 6 , wherein the determination unit determines the quality of the workpiece based on the first and second feature amounts. In an appearance inspection method using an image processing apparatus that has a camera for imaging a workpiece, inspects a predetermined inspection range on the surface of the workpiece based on an image acquired from the camera, and determines whether the workpiece is good or bad.
Each pixel obtaining a grayscale image having a grayscale value corresponding to the amount of light received by the camera;
Using the image acquired from the camera, calculating the distance from the camera to the workpiece surface;
Each pixel generating a distance image having a gray value according to the calculated distance;
Detecting a specific pattern for specifying an inspection region corresponding to the inspection range on the other image in one of the grayscale image and the distance image;
The grayscale image and the distance on the basis of at least one of the one position and the inclination angle of the specific pattern is detected in the image of the image, the grayscale image and the distance image other image have been the specific pattern and a constant setting at the Identifying the inspection area by correcting the position of a reference inspection area in a relative positional relationship ;
Calculating a feature amount from the identified inspection region;
And a step of determining the quality of the workpiece based on the calculated feature amount, and an appearance inspection method using the image processing apparatus.

Images