WO2024116674A1 - Inspection system - Google Patents

Inspection system Download PDF

Info

Publication number
WO2024116674A1
WO2024116674A1 PCT/JP2023/038734 JP2023038734W WO2024116674A1 WO 2024116674 A1 WO2024116674 A1 WO 2024116674A1 JP 2023038734 W JP2023038734 W JP 2023038734W WO 2024116674 A1 WO2024116674 A1 WO 2024116674A1
Authority
WO
WIPO (PCT)
Prior art keywords
article
surface shape
pixel
imaging device
detection area
Prior art date
Application number
PCT/JP2023/038734
Other languages
French (fr)
Japanese (ja)
Inventor
秋希良 藤井
一馬 原口
泰資 田中
翔馬 高橋
匠 羽根田
Original Assignee
パナソニックＩｐマネジメント株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by パナソニックＩｐマネジメント株式会社 filed Critical パナソニックＩｐマネジメント株式会社
Publication of WO2024116674A1 publication Critical patent/WO2024116674A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination

Definitions

  • This disclosure relates to an inspection system used for visual inspection of objects.
  • a known lighting device used for visual inspection of an object is configured to place a light-shielding filter between a surface light source that irradiates inspection light and a lens that focuses the inspection light toward the object, with the light-shielding filter positioned at the focal position on the incident side of the lens (see Patent Document 1).
  • the inspection light irradiated onto the object is reflected off the surface of the object, and the reflected light is captured by an imaging device, and the inclination of the surface of the object can be estimated based on the luminance value of the captured image, etc.
  • the inclination of the surface of an object is detected for each pixel based on the brightness value of each pixel in an image captured by an imaging device, and the height of each pixel is calculated from the detected inclination of each pixel and the width of that pixel and is then sequentially added up. This makes it possible to estimate the surface shape of the object.
  • the sequential integration of the height for each pixel is interrupted in the non-detection areas.
  • the surface shape estimated by sequentially integrating the height for each pixel in areas where the inclination of the surface can be detected has the problem that height information is lost.
  • the objective of this disclosure is to provide an inspection system that estimates the surface shape of an object by sequentially accumulating the height of each pixel, and that, when the surface shape of an object includes an area (non-detection area) where the inclination of the surface cannot be detected by an imaging device, is capable of complementing the surface shape estimated in an area (detection area) where the inclination of the surface can be detected with a surface shape that includes height information.
  • the inspection system includes an illumination device that irradiates an object with inspection light, an imaging device that receives reflected light from the object irradiated with the inspection light using an imaging element having a plurality of pixels to capture an image of the surface of the object, an estimation unit that detects the inclination of the object's surface for each pixel based on the luminance value or gray value of each pixel of the image captured by the imaging device and estimates the surface shape of the object by sequentially accumulating the height of each pixel calculated from the detected inclination of each pixel and the width of the pixel, and a complementation unit that complements the surface shape of the object estimated by the estimation unit.
  • the complementation unit externally acquires height information at any reference position within the detection area where the inclination of the object's surface can be detected, and complements the surface shape in the detection area to a surface shape including the height information based on the height information.
  • the non-detection area where the inclination of the object's surface cannot be detected, may be a step having a side perpendicular to the flat reference surface of the object, or an inclined surface having an angle of a certain degree or more.
  • the surface shape of the object is formed based on design information, and height information at any reference position may be obtained from the design information.
  • the inspection system further includes a display unit that displays the surface shape of the object estimated by the estimation unit, and the display unit may display at least one of the range of the detection area, the range of the non-detection area, the reference position, and the surface shape including the supplemented height information.
  • the estimation unit may detect the inclination of the surface of the object for each pixel by detecting the change in the optical axis of the reflected light from the object relative to the imaging surface of the imaging element as a change in the brightness value or gray value of each pixel of the image captured by the imaging device.
  • Another inspection system includes an illumination device that irradiates an object with inspection light, an imaging device that receives reflected light from the object irradiated with the inspection light using an imaging element having a plurality of pixels to capture an image of the surface of the object, an estimation unit that detects the inclination of the surface of the object for each pixel based on the luminance value or gray value of each pixel of the image captured by the imaging device and estimates the surface shape of the object by sequentially accumulating the height of each pixel calculated from the detected inclination of each pixel and the width of the pixel, and a complementation unit that complements the surface shape of the object estimated by the estimation unit.
  • the complementation unit externally acquires information on the surface shape including the height in the non-detection area, and complements the surface shape that could not be estimated in the non-detection area based on the surface shape information to a surface shape including height information.
  • an inspection system that estimates the surface shape of an object by sequentially accumulating the height of each pixel, when the surface shape of the object includes an area (non-detection area) where the inclination of the surface cannot be detected by an imaging device, an inspection system can be provided that can complement the surface shape estimated in an area (detection area) where the inclination of the surface can be detected with a surface shape that includes height information.
  • FIG. 1 is a block diagram showing a configuration of an inspection system according to an embodiment.
  • FIG. 2A is a diagram illustrating a schematic configuration of an inspection system according to an embodiment.
  • FIG. 2B is an enlarged view of the area enclosed by the dashed line in FIG. 2A.
  • FIG. 3 is a diagram showing an inspection light irradiated onto the surface of an article.
  • FIG. 4A is a diagram illustrating the color of reflected light captured by an imaging device.
  • FIG. 4B is a diagram illustrating the color of reflected light captured by the imaging device.
  • FIG. 5 is a diagram for explaining a method for estimating the surface shape of an article.
  • FIG. 6 is a diagram illustrating an estimated surface shape when an article includes a step portion.
  • FIG. 1 is a block diagram showing a configuration of an inspection system according to an embodiment.
  • FIG. 2A is a diagram illustrating a schematic configuration of an inspection system according to an embodiment.
  • FIG. 2B is an enlarged view of
  • FIG. 7 is a diagram illustrating a method for complementing an estimated surface shape of an article when the surface shape includes steps.
  • FIG. 8 is a diagram illustrating another method for complementing an estimated surface shape of an article when a step is included.
  • FIG. 9 is a diagram for explaining a method for complementing an estimated surface shape when an inclined surface that cannot be detected by an imaging device is included.
  • FIG. 10 is a diagram illustrating another method for complementing an estimated surface shape when an inclined surface that cannot be detected by an imaging device is included.
  • (Embodiment) 1 is a block diagram showing the configuration of an inspection system 100 according to an embodiment.
  • the inspection system 100 according to the present embodiment is an inspection system that inspects the surface shape of an article formed based on design information.
  • the inspection system 100 includes a support table 70, an illumination device 10, an imaging device (camera) 20, an estimation unit 30, a complementation unit 40, a memory unit 50, and a display unit 60.
  • the support table 70 places the object 200 on it.
  • the illumination device 10 irradiates the object 200 with inspection light.
  • the imaging device 20 receives reflected light from the object 200 and captures an image of the surface of the object 200.
  • the estimation unit 30 estimates the surface shape of the object 200.
  • the complementation unit 40 complements the estimated surface shape of the object 200.
  • the memory unit 50 stores the surface shape including height information of the object 200.
  • the display unit 60 displays the surface shape of the object 200.
  • inspection light is irradiated from the lighting device 10 onto the object 200 placed on the support table 70, and the light reflected from the surface of the object 200 is captured by the imaging device 20.
  • the surface shape of the object 200 is estimated based on the image captured by the imaging device 20.
  • FIG. 2A is a diagram showing a schematic configuration of an inspection system 100 according to an embodiment.
  • FIG. 2B is an enlarged view of the portion surrounded by the dashed line in FIG. 2A.
  • the illumination device 10 is composed of a surface light source 11, a color filter 12, a lens 13, and a half mirror 14.
  • the surface light source 11 emits white planar light as the inspection light.
  • the color filter 12 is disposed between the surface light source 11 and the lens 13, at the focal position on the incident side of the lens 13.
  • the color filter 12 has a predetermined color distribution within the incident plane of the inspection light. Therefore, the inspection light emitted from the surface light source 11 becomes planar light having a color distribution in a direction intersecting the traveling direction of the inspection light by passing through the color filter 12.
  • the lens 13 focuses the inspection light that has passed through the color filter 12 toward the object 200.
  • the inspection light that has passed through the lens 13 is reflected by the half mirror 14 and irradiated onto the surface of the object 200.
  • the illumination solid angle at point P1 which is on the optical axis of the inspection light and is the output focal position of lens 13, is uniquely determined by the diameter of the optical path of the inspection light in color filter 12 and the focal length of lens 13.
  • the "illumination solid angle” referred to here refers to a cone of any shape whose apex is a specific point on the optical path of the inspection light and indicates the range over which light is illuminated at that specific point.
  • the illumination solid angle at a position away from the center of lens 13 by the exit focal position of lens 13 will have the same shape and size as the illumination solid angle at point P1.
  • the illumination solid angle at a position farther than the exit focal position of lens 13 will also have the same shape and size as the illumination solid angle at point P1.
  • FIG. 3 is a diagram showing the inspection light irradiated onto the surface of the article. These illumination solid angles are maintained even when the inspection light is reflected by the half mirror 14. Therefore, as shown in FIG. 3, the inspection light is irradiated so as to have the same illumination solid angle IS at each point on the surface of the article 200. In other words, the lighting conditions are the same at any point on the surface of the article 200, regardless of the distance from the surface light source 11.
  • the half mirror 14 reflects the inspection light focused by the lens 13 toward the object 200, while transmitting the light reflected by the object 200.
  • the imaging device 20 receives the reflected light that has passed through the half mirror 14 using an imaging element with multiple pixels, and captures the surface of the object 200 as a color image.
  • the imaging device 20 is positioned within a range in which the imaging surface of the imaging element can receive the reflected light from the object 200.
  • the inspection light is reflected by the half mirror 14 so that the optical axis faces the flat surface of the object 200.
  • the light reflected from the surface of the object 200 passes through the half mirror 14 and enters the imaging device 20.
  • the light reflected from the flat area S1 of the object 200 shown in FIG. 2B enters the imaging device 20 as white light.
  • the flat area S1 of the object 200 is captured by the imaging device 20 as a white image.
  • color filter 12 has a color distribution within the plane of incidence of the inspection light.
  • the optical axis of the inspection light that has passed through the red region of color filter 12 differs in direction from the optical axis of the inspection light that has passed through the blue region of color filter 12.
  • the illumination solid angle IS is the same at each point on the surface of article 200, so if the surface of article 200 is tilted, the color of the reflected light changes in response to the difference in the direction of the optical axis of the inspection light.
  • FIG. 4A is a diagram explaining the color of reflected light captured by imaging device 20.
  • region S2 where the surface of object 200 is tilted to the left side of the paper as shown in FIG. 2B, the inspection light that has passed through the blue region is reflected toward the imaging element of imaging device 20 as shown in FIG. 4A. Meanwhile, part of the inspection light that has passed through the red region is reflected away from the imaging element of imaging device 20. As a result, region S2 on the surface of object 200 is captured by imaging device 20 as a bluish image.
  • FIG. 4B is a diagram explaining the color of reflected light captured by imaging device 20.
  • the inspection light that has passed through the red region is reflected toward the imaging element of imaging device 20, as shown in FIG. 4B. Meanwhile, part of the inspection light that has passed through the blue region is reflected away from the imaging element of imaging device 20. As a result, region S3 on the surface of article 200 is captured by imaging device 20 as a reddish image.
  • the inspection system 100 shown in FIG. 2A can estimate the inclination of the surface of the object 200 based on the luminance values (color gradation) of each color in the image captured by the imaging device 20.
  • a color filter 12 is used to impart a color distribution within the plane of incidence of the inspection light.
  • a shade filter having a shade distribution within the plane of incidence of the inspection light may also be used.
  • the inclination of the surface of the article 200 can be estimated based on the shade values (gray levels) in the image captured by the imaging device 20.
  • FIG. 5 is a diagram illustrating a method for estimating the surface shape of the article.
  • the estimation of the surface shape of the article 200 described below is performed by the estimation unit 30.
  • Fig. 5A is a plan view of an object 200 to be inspected.
  • Fig. 5B shows the surface shape of the object 200.
  • the object 200 has inclined surfaces S1 and S3 with different inclinations, and a flat surface S2 with no inclination.
  • 5C shows pixel rows A 1 to A 7 , B 1 to B 7 , and C 1 to C 7 in the imaging element of the imaging device 20.
  • the surface of the article 200 is imaged by pixel rows A 1 to A 7 , B 1 to B 7 , and C 1 to C 7 .
  • a method for estimating the surface shape of the article 200 based on the luminance values of the images captured by the pixel rows A 1 to A 7 will be described below.
  • a slope ⁇ 1 at pixel A1 of the inclined surface S1 is detected based on the luminance value of the image captured at pixel A1 .
  • a height h1 at pixel A1 of the inclined surface S1 is calculated from the detected slope ⁇ 1 and the width d of pixel A1 .
  • a height h1 at pixel A2 of the inclined surface S1 is calculated based on the luminance value of the image captured at pixel A2 .
  • the surface shape of the inclined surface S1 is estimated by multiplying the height h1 calculated at pixel A1 by the height h1 calculated at pixel A2 , as shown in (d) of FIG.
  • the inclination of the surface is detected as zero based on the luminance values of the images captured at pixels A3 and A4 .
  • the heights at pixels A3 and A4 are calculated as zero.
  • the height ( h1 + h1 ) calculated at pixel A2 is successively multiplied by the heights (zero) calculated at pixels A3 and A4 to estimate the surface shape of the flat surface S2 as shown in FIG. 5D.
  • the inclination ⁇ 2 of the inclined surface S3 is detected based on the luminance values of the images captured at the pixels A5 , A6 , and A7 .
  • the height h2 of each pixel is calculated from the detected inclination ⁇ 2 and the pixel width d.
  • the surface shape of the inclined surface S3 is estimated as shown in FIG. 5D by sequentially accumulating the height ( h1 + h1 ) accumulated at the pixel A4 by the height ( -h2 ) calculated at the pixels A5 , A6 , and A7 .
  • the surface shape of the article 200 corresponding to the pixel rows A1 to A7 is estimated.
  • the surface shapes of the article 200 corresponding to the pixel rows B 1 to B 7 and C 1 to C 7 are estimated using a similar method. This makes it possible to estimate the surface shape of the entire surface of the article 200 by capturing an image once using the imaging device 20.
  • the inclination of the surface of the object 200 is detected for each pixel based on the brightness value or gray value of each pixel in the image captured by the imaging device 20.
  • the detected inclination for each pixel and the height for each pixel calculated from the pixel width are sequentially integrated, so that the surface shape of the object 200 can be estimated.
  • the surface shape of the object 200 estimated by this method can be obtained as a three-dimensional shape including height information, as shown in FIG. 5(d). Therefore, this is particularly useful in visual inspections that check not only the relative height difference of the surface shape, but also the height of the surface shape relative to a reference surface.
  • FIG. 6 is a diagram for explaining an estimated surface shape when the article 200 includes a step portion.
  • Fig. 6(a) is a diagram for explaining the surface shape of the article 200.
  • the surface shape of the article 200 has flat surfaces S1 , S3 , S5 and inclined surfaces S2 , S4 .
  • Step portions M1 , M2 are formed between the flat surface S1 and the inclined surface S2 , and between the inclined surface S4 and the flat surface S5 .
  • the flat surfaces S1 , S3 , S5 form the reference surfaces of the article 200.
  • the surface shape of the article 200 includes step portions M 1 , M 2 having side surfaces perpendicular to the flat surfaces (reference surfaces) S 1 , S 3 , S 5 of the article 200, it is not possible to grasp the height difference of the step portions M 1 , M 2 from the flat surfaces S 1 , S 5. Therefore, when the heights of each pixel calculated from the inclination of each pixel and the width of the pixel are sequentially integrated, the integration is performed assuming that there are no step portions M 1 , M 2 between the flat surface S 1 and the inclined surface S 2 , and between the inclined surface S 4 and the flat surface S 5.
  • the inclined surfaces S 2 to S 4 are estimated as inclined surfaces S' 2 to S' 4 located below the flat surfaces S 1 and S 5 , as shown in FIG. 6B.
  • the relative height relationship of the estimated inclined surfaces S' 2 to S' 4 is correctly estimated.
  • FIG. 7 is a diagram illustrating a method for complementing the estimated surface shape of the article 200 when the surface shape includes a step portion. The complementation of the surface shape of the article 200 described below is performed by the complementation unit 40.
  • Fig. 7A is a diagram showing the surface shape of the article 200.
  • the surface shape of the article 200 includes step portions M1 and M2 of height H1 and step portions M3 and M4 of height H2 .
  • Fig. 7B is a diagram showing the surface shape estimated by the estimation unit 30 using the method described in Fig. 6B.
  • the imaging device 20 cannot grasp the step portions M1 , M2 , M3 , and M4 . Therefore, the inclined surfaces S2 to S4 and the inclined surfaces S6 to S8 are estimated as the inclined surfaces S'2 to S'4 and the inclined surfaces S'6 to S'8 , respectively, which are located below the flat surfaces S1 , S5 , and S9, as shown in Fig. 7B. Since the height H2 of the step portions M3 and M4 is lower than the height H1 of the step portions M1 and M2 , the flat surface S'7 is estimated to be located above the flat surface S'3 .
  • the inspection system of this embodiment when the object 200 includes an area (step portion) where the inclination of the surface of the object 200 cannot be detected by the imaging device 20, height information at any reference position within the detection area where the inclination of the surface of the object 200 can be detected is acquired from the outside, and the surface shape in the detection area estimated by the estimation unit 30 is complemented to a surface shape including the height information based on the acquired height information.
  • height information at any reference position within the detection area can be obtained from the design information.
  • the height information may be obtained from data obtained by measuring the surface shape of a good object 200.
  • Surface shape information including the height information of the object 200 is stored in the memory unit 50.
  • the reference position V1 is the position where the step portion M1 is located.
  • the height information H1 is the height of the step portion M1 .
  • the height at the reference position V1 (zero in Fig. 7B) is supplemented to the height information H1 acquired from the storage unit 50.
  • a step portion M1 having a height H1 is supplemented between the flat surface S1 and the inclined surface S2 .
  • the relative height relationships of the inclined surfaces S' 2 to S' 4 are correctly estimated. Therefore, by complementing the heights of the inclined surfaces S' 2 to S' 4 by the same height H 1 as the step portion M 1 , the estimated inclined surfaces S' 2 to S' 4 can be complemented to inclined surfaces S 2 to S 4 containing correct height information.
  • the step portion M 2 has the same height H 1 as the step portion M 1. Therefore, by complementing the inclined surfaces S 2 to S 4 , the step portion M 2 is automatically complemented to have the height H 1 .
  • FIG. 8 is a diagram for explaining another method of complementing the estimated surface shape of the article 200 when the surface shape includes steps M 1 and M 2.
  • the height information for complementation is acquired with the positions of the steps M 1 and M 2 as the reference positions V 1 and V 2.
  • the estimated inclined surfaces S' 2 to S' 4 can be complemented to inclined surfaces S 2 to S 4 including correct height information based on the acquired height information in a manner similar to that described in FIG. 7(c).
  • the detection areas are supplemented to the surface shape including the height direction for both the inclined surfaces S2 to S4 and the inclined surfaces S6 to S8 .
  • the imaging device 20 which captures an image of the reflected light from the article 200, is fixedly disposed at a position facing the surface of the article 200. For this reason, an inclined surface having an angle of a certain degree or more with respect to the flat reference surface of the article 200 cannot be detected by the imaging device 20.
  • the angle of a certain degree or more that cannot be detected by the imaging device 20 is uniquely determined by the angle range in which the inspection light from the illumination device 10 is incident on the reference surface of the article.
  • FIG. 9 is a diagram for explaining a method for complementing an estimated surface shape when an inclined surface that cannot be detected by the imaging device 20 is included.
  • FIG. 9(a) is a diagram showing the surface shape of an article 200.
  • the surface shape of the article 200 has inclined surfaces S2 , S6 , and S3 , S5 having different angles with respect to flat surfaces S1 , S4 , S7 .
  • the flat surfaces S1 , S4 , S7 form the reference surfaces of the article 200.
  • the inclined surfaces S2 , S6 having an angle ⁇ of a certain value or more with respect to the flat surfaces S1 , S7 cannot be detected by the imaging device 20.
  • the accumulation stops at the area A1 of the inclined surface S2 , as shown in Fig. 9B. Therefore, for the inclined surface S3 , the accumulation starts from the height of the flat surface S1 (zero in this case). Therefore, the surfaces S3 to S5 shown in Fig. 9A are estimated as surfaces S'3 to S'5 located below the flat surface S1 . The relative height relationship of the surfaces S'3 to S'5 is correctly estimated.
  • the integration is also interrupted in the area A2 of the inclined surface S6 .
  • the flat surface S7 forms a reference surface, the height of the flat surface S7 is correctly estimated.
  • height information at any reference position within the detection area where the inclination of the surface of the item 200 can be detected can be acquired from the outside. Based on the acquired height information, the surface shape in the detection area estimated by the estimation unit 30 can be complemented with a surface shape including the height information.
  • the height at reference position Q (here, a negative height) is complemented to the acquired height information (here, zero).
  • the relative height relationship of surfaces S'3 to S'5 is correctly estimated. Therefore, by complementing the heights of surfaces S'3 to S'5 by the same height as reference position Q, the estimated surfaces S'3 to S'5 can be complemented to S3 to S5 including correct height information.
  • the estimated surface shape (surfaces S'3 to S'5 ) in the detection region ( x2 ⁇ x ⁇ x3 ) where the inclination of the surface of the article 200 can be detected is complemented to S3 to S5 including correct height information.
  • the surface shape (inclined surfaces S2 , S6 ) that could not be estimated by the estimation unit 30 can be complemented to a surface shape including height information by a similar method.
  • FIG. 10 is a diagram for explaining another method of complementing an estimated surface shape when an inclined surface that cannot be detected by the imaging device 20 is included.
  • information on the surface shape (inclined surfaces S 2 , S 6 ) including the height of the non-detection region (x 1 ⁇ x ⁇ x 2 ; x 3 ⁇ x ⁇ x 4 ) is acquired from the outside (for example, design information).
  • the surface shape that could not be estimated in the non-detection region is complemented to a surface shape including height information.
  • the relative height relationship of the surfaces S′ 3 to S′ 5 is correctly estimated.
  • the inclined surfaces S′ 3 to S′ 5 can also be complemented to the inclined surfaces S 3 to S 5 with the correct height.
  • the non-detection region (regions x 3 to x 4 ) can also be complemented in a similar manner.
  • the non-detection areas where the inclination of the surface of the article 200 cannot be detected are steps that have sides perpendicular to the flat reference surface of the article 200, or inclined surfaces that have an angle of a certain degree or more.
  • the sequential accumulation of heights for each pixel is interrupted in that area. This causes a problem in that the surface shape estimated in the detection area loses height information.
  • the surface shape in the detection area estimated by the estimation unit 30 can be complemented with a surface shape that includes height information. Therefore, in the inspection system 100, the "non-detection area" also includes an area that is not to be detected intentionally.
  • height information at any reference position can be obtained from the design information.
  • surface shape information including the height of the non-detection area may be acquired from the outside. Based on the acquired surface shape information, the surface shape that could not be estimated in the non-detection area may be complemented with a surface shape including height information.
  • the inspection system in this embodiment may further include a display unit 60 that displays the surface shape of the item 200 estimated by the estimation unit 30.
  • a display unit 60 that displays the surface shape of the item 200 estimated by the estimation unit 30.
  • a lighting device equipped with an optical system that irradiates the inspection light so that each point on the surface of the article 200 has the same illumination solid angle IS is used.
  • this is not limited to this, and for example, a lighting device equipped with an optical system using a photometric stereo method with divided light emission lighting may be used.
  • the estimation unit detects the change in the optical axis of the reflected light from the object 200 relative to the imaging surface of the imaging element in the imaging device 20 as a change in the brightness value or gray value of each pixel of the image captured by the imaging device. This makes it possible to estimate the inclination of the surface of the object 200 for each pixel.
  • the technology disclosed herein is useful as a lighting device for visual inspection.

Landscapes

  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

This inspection system comprises: an illumination device that irradiates an article with inspection light; an imaging device that captures an image of the surface of the article by receiving reflected light from the article irradiated with the inspection light, using an imaging element having a plurality of pixels; an estimation unit that detects an inclination of the surface of the article for each pixel on the basis of a luminance value or a gray value of each pixel of the image captured by the imaging device, and estimates the surface shape of the article by accumulating heights that are calculated for the respective pixels from the detected inclinations of the respective pixels and the widths of the pixels; and a complementing unit that complements the estimated surface shape of the article. If the article includes a non-detection area in which the inclination of the surface of the article cannot be detected in the imaging device, the complementing unit externally acquires height information at a reference position in a detection area in which the inclination of the surface of the article can be detected, and complements the surface shape in the detection area on the basis of the height information to obtain a surface shape including the height information.

Description

検査システムInspection Systems
　本開示は、物品の外観検査に用いられる検査システムに関する。 This disclosure relates to an inspection system used for visual inspection of objects.
　従来、物品の外観検査に用いられる照明装置として、検査光を照射する面光源と、検査光を物品に向けて集光するレンズとの間に、遮光フィルターを配置し、この遮光フィルターをレンズの入射側の焦点位置に配置した構成が知られている（特許文献１を参照）。このような構成の照明装置を用いた検査システムでは、物品に照射された検査光が物品の表面で反射された反射光を撮像装置で撮像して、撮像された画像の輝度値などに基づいて、物品の面の傾きを推定することができる。 　Conventionally, a known lighting device used for visual inspection of an object is configured to place a light-shielding filter between a surface light source that irradiates inspection light and a lens that focuses the inspection light toward the object, with the light-shielding filter positioned at the focal position on the incident side of the lens (see Patent Document 1). In an inspection system using a lighting device configured in this way, the inspection light irradiated onto the object is reflected off the surface of the object, and the reflected light is captured by an imaging device, and the inclination of the surface of the object can be estimated based on the luminance value of the captured image, etc.
　従来の検査システムでは、撮像装置で撮像された画像の各画素における輝度値に基づいて、画素毎に物品の表面の傾きを検出し、検出された画素毎の傾き及び当該画素の幅から算出した画素毎の高さを順次積算する。これによって、物品の表面形状を推定することができる。 　In conventional inspection systems, the inclination of the surface of an object is detected for each pixel based on the brightness value of each pixel in an image captured by an imaging device, and the height of each pixel is calculated from the detected inclination of each pixel and the width of that pixel and is then sequentially added up. This makes it possible to estimate the surface shape of the object.
　しかしながら、物品の表面形状が、撮像装置で表面の傾きを検出できない領域（非検出領域）を含む場合、非検出領域において、画素毎の高さを順次積算することが途切れてしまう。そのため、表面の傾きを検出できる領域（検出領域）において、画素毎の高さを順次積算することによって推定された表面形状は、高さ情報が失われてしまうという問題がある。 However, if the surface shape of an object includes areas where the inclination of the surface cannot be detected by the imaging device (non-detection areas), the sequential integration of the height for each pixel is interrupted in the non-detection areas. As a result, the surface shape estimated by sequentially integrating the height for each pixel in areas where the inclination of the surface can be detected (detection areas) has the problem that height information is lost.
日本国特許第６４５１８２１号公報Japanese Patent No. 6451821
　本開示の目的は、画素毎の高さを順次積算することによって物品の表面形状を推定する検査システムにおいて、物品の表面形状が、撮像装置で表面の傾きを検出できない領域（非検出領域）を含む場合、表面の傾きを検出できる領域（検出領域）において推定された表面形状を、高さ情報を含む表面形状に補完することが可能な検査システムを提供することにある。 The objective of this disclosure is to provide an inspection system that estimates the surface shape of an object by sequentially accumulating the height of each pixel, and that, when the surface shape of an object includes an area (non-detection area) where the inclination of the surface cannot be detected by an imaging device, is capable of complementing the surface shape estimated in an area (detection area) where the inclination of the surface can be detected with a surface shape that includes height information.
　本開示に係る検査システムは、物品に検査光を照射する照明装置と、検査光が照射された物品からの反射光を、複数の画素を有する撮像素子で受光して、物品の表面を撮像する撮像装置と、撮像装置で撮像された画像の各画素における輝度値または濃淡値に基づいて、画素毎に物品の表面の傾きを検出するとともに、検出された画素毎の傾き及び画素の幅から算出した画素毎の高さを順次積算することにより、物品の表面形状を推定する推定部と、推定部で推定した物品の表面形状を補完する補完部とを備える。補完部は、物品が、撮像装置において、物品の表面の傾きを検出できない非検出領域を含む場合、物品の表面の傾きを検出できる検出領域内の任意の基準位置における高さ情報を外部から取得し、高さ情報に基づいて、検出領域における表面形状を、高さ情報を含む表面形状に補完する。 The inspection system according to the present disclosure includes an illumination device that irradiates an object with inspection light, an imaging device that receives reflected light from the object irradiated with the inspection light using an imaging element having a plurality of pixels to capture an image of the surface of the object, an estimation unit that detects the inclination of the object's surface for each pixel based on the luminance value or gray value of each pixel of the image captured by the imaging device and estimates the surface shape of the object by sequentially accumulating the height of each pixel calculated from the detected inclination of each pixel and the width of the pixel, and a complementation unit that complements the surface shape of the object estimated by the estimation unit. When the object includes a non-detection area where the imaging device cannot detect the inclination of the object's surface, the complementation unit externally acquires height information at any reference position within the detection area where the inclination of the object's surface can be detected, and complements the surface shape in the detection area to a surface shape including the height information based on the height information.
　物品の表面の傾きを検出できない非検出領域は、物品の平坦な基準面に対して垂直な側面を有する段差部、または、一定以上の角度を有する傾斜面であってもよい。 The non-detection area, where the inclination of the object's surface cannot be detected, may be a step having a side perpendicular to the flat reference surface of the object, or an inclined surface having an angle of a certain degree or more.
　物品の表面形状は、設計情報に基づいて形成されたものであり、任意の基準位置における高さ情報は、設計情報から取得してもよい。 The surface shape of the object is formed based on design information, and height information at any reference position may be obtained from the design information.
　本開示に係る検査システムは、推定部で推定した物品の表面形状を表示する表示部をさらに備え、表示部は、検出領域の範囲、非検出領域の範囲、基準位置、及び補完された高さ情報を含む表面形状のうち少なくとも一つを表示してもよい。 The inspection system according to the present disclosure further includes a display unit that displays the surface shape of the object estimated by the estimation unit, and the display unit may display at least one of the range of the detection area, the range of the non-detection area, the reference position, and the surface shape including the supplemented height information.
　推定部は、撮像素子の撮像面に対する物品からの反射光の光軸の変化を、撮像装置で撮像された画像の各画素における輝度値または濃淡値の変化として検知することによって、画素毎に物品の表面の傾きを検出してもよい。 The estimation unit may detect the inclination of the surface of the object for each pixel by detecting the change in the optical axis of the reflected light from the object relative to the imaging surface of the imaging element as a change in the brightness value or gray value of each pixel of the image captured by the imaging device.
　本開示に係る別の検査システムは、物品に検査光を照射する照明装置と、検査光が照射された物品からの反射光を、複数の画素を有する撮像素子で受光して、物品の表面を撮像する撮像装置と、撮像装置で撮像された画像の各画素における輝度値または濃淡値に基づいて、画素毎に物品の表面の傾きを検出するとともに、検出された画素毎の傾き及び画素の幅から算出した前記画素毎の高さを順次積算することにより、物品の表面形状を推定する推定部と、推定部で推定した物品の表面形状を補完する補完部とを備える。補完部は、物品が、撮像装置において、物品の表面の傾きを検出できない非検出領域を含む場合、非検出領域における高さを含む表面形状の情報を外部から取得し、表面形状の情報に基づいて、非検出領域において推定できなかった表面形状を、高さ情報を含む表面形状に補完する。 Another inspection system according to the present disclosure includes an illumination device that irradiates an object with inspection light, an imaging device that receives reflected light from the object irradiated with the inspection light using an imaging element having a plurality of pixels to capture an image of the surface of the object, an estimation unit that detects the inclination of the surface of the object for each pixel based on the luminance value or gray value of each pixel of the image captured by the imaging device and estimates the surface shape of the object by sequentially accumulating the height of each pixel calculated from the detected inclination of each pixel and the width of the pixel, and a complementation unit that complements the surface shape of the object estimated by the estimation unit. When the object includes a non-detection area where the inclination of the surface of the object cannot be detected by the imaging device, the complementation unit externally acquires information on the surface shape including the height in the non-detection area, and complements the surface shape that could not be estimated in the non-detection area based on the surface shape information to a surface shape including height information.
　本開示によれば、画素毎の高さを順次積算することによって物品の表面形状を推定する検査システムにおいて、物品の表面形状が、撮像装置で表面の傾きを検出できない領域（非検出領域）を含む場合、表面の傾きを検出できる領域（検出領域）において推定された表面形状を、高さ情報を含む表面形状に補完することが可能な検査システムを提供することができる。 According to the present disclosure, in an inspection system that estimates the surface shape of an object by sequentially accumulating the height of each pixel, when the surface shape of the object includes an area (non-detection area) where the inclination of the surface cannot be detected by an imaging device, an inspection system can be provided that can complement the surface shape estimated in an area (detection area) where the inclination of the surface can be detected with a surface shape that includes height information.
図１は、実施形態における検査システムの構成を示すブロック図である。FIG. 1 is a block diagram showing a configuration of an inspection system according to an embodiment. 図２Ａは、実施形態における検査システムの構成を模式的に示す図である。FIG. 2A is a diagram illustrating a schematic configuration of an inspection system according to an embodiment. 図２Ｂは、図２Ａの破線で囲まれた部分の拡大図である。FIG. 2B is an enlarged view of the area enclosed by the dashed line in FIG. 2A. 図３は、物品の表面に照射される検査光を示す図である。FIG. 3 is a diagram showing an inspection light irradiated onto the surface of an article. 図４Ａは、撮像装置で撮像される反射光の色を説明する図である。FIG. 4A is a diagram illustrating the color of reflected light captured by an imaging device. 図４Ｂは、撮像装置で撮像される反射光の色を説明する図である。FIG. 4B is a diagram illustrating the color of reflected light captured by the imaging device. 図５は、物品の表面形状を推定する方法を説明する図である。FIG. 5 is a diagram for explaining a method for estimating the surface shape of an article. 図６は、物品が段差部を含む場合の推定される表面形状を説明する図である。FIG. 6 is a diagram illustrating an estimated surface shape when an article includes a step portion. 図７は、段差を含む場合の推定された物品の表面形状を補完する方法を説明する図である。FIG. 7 is a diagram illustrating a method for complementing an estimated surface shape of an article when the surface shape includes steps. 図８は、段差を含む場合の推定された物品の表面形状を補完する他の方法を説明する図である。FIG. 8 is a diagram illustrating another method for complementing an estimated surface shape of an article when a step is included. 図９は、撮像装置で検出できない傾斜面を含む場合の推定された表面形状を補完する方法を説明する図である。FIG. 9 is a diagram for explaining a method for complementing an estimated surface shape when an inclined surface that cannot be detected by an imaging device is included. 図１０は、撮像装置で検出できない傾斜面を含む場合の推定された表面形状を補完する他の方法を説明する図である。FIG. 10 is a diagram illustrating another method for complementing an estimated surface shape when an inclined surface that cannot be detected by an imaging device is included.
　以下、本開示の実施形態を図面に基づいて詳細に説明する。本開示は、以下の実施形態に限定されるものではない。本開示は、本開示の効果を奏する範囲を逸脱しない範囲で、適宜変更が可能である。 Below, an embodiment of the present disclosure will be described in detail with reference to the drawings. The present disclosure is not limited to the following embodiment. The present disclosure can be modified as appropriate without departing from the scope of the effects of the present disclosure.
　（実施形態）
　図１は、実施形態における検査システム１００の構成を示すブロック図である。本実施形態における検査システム１００は、設計情報に基づいて形成された物品の表面形状を検査する検査システムである。 (Embodiment)
1 is a block diagram showing the configuration of an inspection system 100 according to an embodiment. The inspection system 100 according to the present embodiment is an inspection system that inspects the surface shape of an article formed based on design information.
　図１に示すように、検査システム１００は、支持台７０と、照明装置１０と、撮像装置（カメラ）２０と、推定部３０と、補完部４０と、記憶部５０と、表示部６０とを備える。支持台７０は、物品２００を載置する。照明装置１０は、物品２００に検査光を照射する。撮像装置２０は、物品２００からの反射光を受光して、物品２００の表面を撮像する。推定部３０は、物品２００の表面形状を推定する。補完部４０は、推定された物品２００の表面形状を補完する。記憶部５０は、物品２００の高さ情報を含む表面形状を記憶する。表示部６０は、物品２００の表面形状を表示する。 As shown in FIG. 1, the inspection system 100 includes a support table 70, an illumination device 10, an imaging device (camera) 20, an estimation unit 30, a complementation unit 40, a memory unit 50, and a display unit 60. The support table 70 places the object 200 on it. The illumination device 10 irradiates the object 200 with inspection light. The imaging device 20 receives reflected light from the object 200 and captures an image of the surface of the object 200. The estimation unit 30 estimates the surface shape of the object 200. The complementation unit 40 complements the estimated surface shape of the object 200. The memory unit 50 stores the surface shape including height information of the object 200. The display unit 60 displays the surface shape of the object 200.
　検査システム１００では、支持台７０に載置された物品２００に、照明装置１０から検査光を照射し、物品２００の表面で反射された反射光を撮像装置２０で撮像する。撮像装置２０で撮像された画像に基づいて、物品２００の表面形状が推定される。 In the inspection system 100, inspection light is irradiated from the lighting device 10 onto the object 200 placed on the support table 70, and the light reflected from the surface of the object 200 is captured by the imaging device 20. The surface shape of the object 200 is estimated based on the image captured by the imaging device 20.
　図２Ａは、実施形態における検査システム１００の構成を模式的に示す図である。図２Ｂは、図２Ａの破線で囲まれた部分の拡大図である。 FIG. 2A is a diagram showing a schematic configuration of an inspection system 100 according to an embodiment. FIG. 2B is an enlarged view of the portion surrounded by the dashed line in FIG. 2A.
　図２Ａに示すように、照明装置１０は、面光源１１と、カラーフィルター１２と、レンズ１３と、ハーフミラー１４とで構成される。面光源１１は、白色の平面光を検査光として出射する。 As shown in FIG. 2A, the illumination device 10 is composed of a surface light source 11, a color filter 12, a lens 13, and a half mirror 14. The surface light source 11 emits white planar light as the inspection light.
　カラーフィルター１２は、面光源１１とレンズ１３との間であって、レンズ１３の入射側の焦点位置に配置されている。カラーフィルター１２は、検査光の入射面内で所定の色分布を有している。このため、面光源１１から出射された検査光は、カラーフィルター１２を透過することで、検査光の進行方向と交差する方向に色分布を有する平面光となる。 The color filter 12 is disposed between the surface light source 11 and the lens 13, at the focal position on the incident side of the lens 13. The color filter 12 has a predetermined color distribution within the incident plane of the inspection light. Therefore, the inspection light emitted from the surface light source 11 becomes planar light having a color distribution in a direction intersecting the traveling direction of the inspection light by passing through the color filter 12.
　レンズ１３は、カラーフィルター１２を透過した検査光を、物品２００に向けて集光する。レンズ１３を透過した検査光は、ハーフミラー１４で反射されて、物品２００の表面に照射される。 The lens 13 focuses the inspection light that has passed through the color filter 12 toward the object 200. The inspection light that has passed through the lens 13 is reflected by the half mirror 14 and irradiated onto the surface of the object 200.
　図２Ａに示すように、検査光の光軸上であって、かつレンズ１３の出射側焦点位置である点Ｐ１における照射立体角は、カラーフィルター１２における検査光の光路の直径とレンズ１３の焦点距離により、一義的に決まる。ここで言う「照射立体角」とは、検査光の光路上の所定の点を頂点とし、当該所定の点に光が照射される範囲を示す任意形状の錐体を言う。 As shown in Figure 2A, the illumination solid angle at point P1, which is on the optical axis of the inspection light and is the output focal position of lens 13, is uniquely determined by the diameter of the optical path of the inspection light in color filter 12 and the focal length of lens 13. The "illumination solid angle" referred to here refers to a cone of any shape whose apex is a specific point on the optical path of the inspection light and indicates the range over which light is illuminated at that specific point.
　検査光の光軸から離れた位置であっても、レンズ１３の中心からレンズ１３の出射側焦点位置だけ離れた位置における照射立体角も、点Ｐ１における照射立体角と同じ形状で同じ大きさとなる。レンズ１３の出射側焦点位置よりも遠い位置における照射立体角も、点Ｐ１における照射立体角と同じ形状で同じ大きさとなる。 Even if the position is far from the optical axis of the inspection light, the illumination solid angle at a position away from the center of lens 13 by the exit focal position of lens 13 will have the same shape and size as the illumination solid angle at point P1. The illumination solid angle at a position farther than the exit focal position of lens 13 will also have the same shape and size as the illumination solid angle at point P1.
　図３は、物品の表面に照射される検査光を示す図である。検査光がハーフミラー１４により反射された場合にも、これらの照射立体角は維持される。したがって、図３に示すように、検査光は、物品２００の表面の各点において同じ照射立体角ＩＳを有するように照射される。つまり、物品２００の表面の任意の点において、面光源１１からの距離に依存せず、照明条件が同じとなる。 FIG. 3 is a diagram showing the inspection light irradiated onto the surface of the article. These illumination solid angles are maintained even when the inspection light is reflected by the half mirror 14. Therefore, as shown in FIG. 3, the inspection light is irradiated so as to have the same illumination solid angle IS at each point on the surface of the article 200. In other words, the lighting conditions are the same at any point on the surface of the article 200, regardless of the distance from the surface light source 11.
　ハーフミラー１４は、レンズ１３で集光された検査光を物品２００に向けて反射する一方、物品２００で反射された反射光を透過する。 The half mirror 14 reflects the inspection light focused by the lens 13 toward the object 200, while transmitting the light reflected by the object 200.
　撮像装置２０は、複数の画素を備えた撮像素子により、ハーフミラー１４を透過した反射光を受光し、物品２００の表面をカラー画像として撮像する。撮像装置２０は、撮像素子の撮像面が、物品２００からの反射光を受光できる範囲内に配置される。 The imaging device 20 receives the reflected light that has passed through the half mirror 14 using an imaging element with multiple pixels, and captures the surface of the object 200 as a color image. The imaging device 20 is positioned within a range in which the imaging surface of the imaging element can receive the reflected light from the object 200.
　次に、検査システム１００の動作原理について説明する。 Next, the operating principle of the inspection system 100 will be explained.
　図２Ａに示すように、検査光は、ハーフミラー１４により、光軸が物品２００の平坦面に向くように反射される。物品２００の表面で反射された反射光は、ハーフミラー１４を透過して、撮像装置２０に入射される。例えば、図２Ｂに示す物品２００の平坦な領域Ｓ１での反射光は、白色光として撮像装置２０に入射される。その結果、物品２００の平坦な領域Ｓ１は、白色の画像として撮像装置２０で撮像される。 As shown in FIG. 2A, the inspection light is reflected by the half mirror 14 so that the optical axis faces the flat surface of the object 200. The light reflected from the surface of the object 200 passes through the half mirror 14 and enters the imaging device 20. For example, the light reflected from the flat area S1 of the object 200 shown in FIG. 2B enters the imaging device 20 as white light. As a result, the flat area S1 of the object 200 is captured by the imaging device 20 as a white image.
　一方、カラーフィルター１２は検査光の入射面内で色分布を有している。その結果、例えば、カラーフィルター１２の赤色領域を透過した検査光の光軸は、カラーフィルター１２の青色領域を透過した検査光の光軸と方向が異なっている。上述したように、物品２００の表面の各点において、照射立体角ＩＳは同じであるため、物品２００の表面に傾きがある場合、検査光の光軸の方向が異なることに対応して反射光の色が変化する。 On the other hand, color filter 12 has a color distribution within the plane of incidence of the inspection light. As a result, for example, the optical axis of the inspection light that has passed through the red region of color filter 12 differs in direction from the optical axis of the inspection light that has passed through the blue region of color filter 12. As described above, the illumination solid angle IS is the same at each point on the surface of article 200, so if the surface of article 200 is tilted, the color of the reflected light changes in response to the difference in the direction of the optical axis of the inspection light.
　図４Ａは、撮像装置２０で撮像される反射光の色を説明する図である。例えば、図２Ｂに示すように、物品２００の表面が紙面の左側に傾いている領域Ｓ２では、図４Ａに示すように、青色領域を透過した検査光は、撮像装置２０の撮像素子に向かって反射される。一方、赤色領域を透過した検査光の一部は、撮像装置２０の撮像素子から離れるように反射される。その結果、物品２００の表面の領域Ｓ２は、青みがかった色の画像として撮像装置２０で撮像される。 FIG. 4A is a diagram explaining the color of reflected light captured by imaging device 20. For example, in region S2, where the surface of object 200 is tilted to the left side of the paper as shown in FIG. 2B, the inspection light that has passed through the blue region is reflected toward the imaging element of imaging device 20 as shown in FIG. 4A. Meanwhile, part of the inspection light that has passed through the red region is reflected away from the imaging element of imaging device 20. As a result, region S2 on the surface of object 200 is captured by imaging device 20 as a bluish image.
　図４Ｂは、撮像装置２０で撮像される反射光の色を説明する図である。図２Ｂに示すように、物品２００の表面が紙面の右側に傾いている領域Ｓ３では、図４Ｂに示すように、赤色領域を透過した検査光は、撮像装置２０の撮像素子に向かって反射される。一方、青色領域を透過した検査光の一部は、撮像装置２０の撮像素子から離れるように反射される。その結果、物品２００の表面の領域Ｓ３は、赤みがかった色の画像として撮像装置２０で撮像される。 FIG. 4B is a diagram explaining the color of reflected light captured by imaging device 20. As shown in FIG. 2B, in region S3 where the surface of article 200 is tilted to the right of the page, the inspection light that has passed through the red region is reflected toward the imaging element of imaging device 20, as shown in FIG. 4B. Meanwhile, part of the inspection light that has passed through the blue region is reflected away from the imaging element of imaging device 20. As a result, region S3 on the surface of article 200 is captured by imaging device 20 as a reddish image.
　以上説明したように、図２Ａに示す検査システム１００によれば、撮像装置２０での撮像画像における各色の輝度値（カラーグラデーション）に基づいて、物品２００の表面の傾きを推定することができる。 As described above, the inspection system 100 shown in FIG. 2A can estimate the inclination of the surface of the object 200 based on the luminance values (color gradation) of each color in the image captured by the imaging device 20.
　実施形態では、カラーフィルター１２を用いて、検査光の入射面内での色分布を付与する。しかし、検査光の入射面内で濃淡分布を有する濃淡フィルターを用いてもよい。この場合、撮像装置２０の撮像画像における濃淡値（グレーレベル）に基づいて、物品２００の表面の傾きを推定することができる。 In the embodiment, a color filter 12 is used to impart a color distribution within the plane of incidence of the inspection light. However, a shade filter having a shade distribution within the plane of incidence of the inspection light may also be used. In this case, the inclination of the surface of the article 200 can be estimated based on the shade values (gray levels) in the image captured by the imaging device 20.
　次に、図５を参照しながら、物品２００の表面形状を推定する方法について説明する。図５は、物品の表面形状を推定する方法を説明する図である。以下に説明する物品２００の表面形状の推定は、推定部３０により実行される。 Next, a method for estimating the surface shape of the article 200 will be described with reference to FIG. 5. FIG. 5 is a diagram illustrating a method for estimating the surface shape of the article. The estimation of the surface shape of the article 200 described below is performed by the estimation unit 30.
　図５の（ａ）は、検査対象となる物品２００の平面図である。図５の（ｂ）は、物品２００の表面形状を示す。物品２００は、図５の（ｂ）に示すように、傾きの異なる傾斜面Ｓ１、Ｓ３、及び傾きのない平坦面Ｓ２を有する。 Fig. 5A is a plan view of an object 200 to be inspected. Fig. 5B shows the surface shape of the object 200. As shown in Fig. 5B, the object 200 has inclined surfaces S1 and S3 with different inclinations, and a flat surface S2 with no inclination.
　図５の（ｃ）は、撮像装置２０の撮像素子における画素列Ａ１～Ａ７、Ｂ１～Ｂ７、Ｃ１～Ｃ７を示す。物品２００の表面は、画素列Ａ１～Ａ７、Ｂ１～Ｂ７、Ｃ１～Ｃ７で撮像される。 5C shows pixel rows A 1 to A 7 , B 1 to B 7 , and C 1 to C 7 in the imaging element of the imaging device 20. The surface of the article 200 is imaged by pixel rows A 1 to A 7 , B 1 to B 7 , and C 1 to C 7 .
　以下、画素列Ａ１～Ａ７で撮像された画像の輝度値に基づいて、物品２００の表面形状を推定する方法を説明する。 A method for estimating the surface shape of the article 200 based on the luminance values of the images captured by the pixel rows A 1 to A 7 will be described below.
　傾斜面Ｓ１では、画素Ａ１で撮像された画像の輝度値に基づいて、傾斜面Ｓ１の画素Ａ１における傾きα１が検出される。検出された傾きα１と画素Ａ１の幅ｄから、傾斜面Ｓ１の画素Ａ１における高さｈ１が算出される。同様に、画素Ａ２で撮像された画像の輝度値に基づいて、傾斜面Ｓ１の画素Ａ２における高さｈ１が算出される。これにより、画素Ａ１で算出した高さｈ１に、画素Ａ２で算出した高さｈ１を積算することにより、図５の（ｄ）に示すように、傾斜面Ｓ１の表面形状が推定される。 In the inclined surface S1 , a slope α1 at pixel A1 of the inclined surface S1 is detected based on the luminance value of the image captured at pixel A1 . A height h1 at pixel A1 of the inclined surface S1 is calculated from the detected slope α1 and the width d of pixel A1 . Similarly, a height h1 at pixel A2 of the inclined surface S1 is calculated based on the luminance value of the image captured at pixel A2 . As a result, the surface shape of the inclined surface S1 is estimated by multiplying the height h1 calculated at pixel A1 by the height h1 calculated at pixel A2 , as shown in (d) of FIG.
　平坦面Ｓ２では、画素Ａ３、Ａ４で撮像された各画像の輝度値に基づいて、面の傾きはゼロと検出される。その結果、画素Ａ３、Ａ４における高さはゼロと算出される。これにより、画素Ａ２で積算した高さ（ｈ１＋ｈ１）に、画素Ａ３、Ａ４で算出した高さ（ゼロ）を順次積算することにより、図５の（ｄ）に示すように、平坦面Ｓ２の表面形状が推定される。 For the flat surface S2 , the inclination of the surface is detected as zero based on the luminance values of the images captured at pixels A3 and A4 . As a result, the heights at pixels A3 and A4 are calculated as zero. As a result, the height ( h1 + h1 ) calculated at pixel A2 is successively multiplied by the heights (zero) calculated at pixels A3 and A4 to estimate the surface shape of the flat surface S2 as shown in FIG. 5D.
　傾斜面Ｓ３では、画素Ａ５、Ａ６、Ａ７で撮像された各画像の輝度値に基づいて、傾斜面Ｓ３の傾きα２が検出される。検出された傾きα２と画素の幅ｄから、画素毎の高さｈ２が算出される。これにより、画素Ａ４で積算された高さ（ｈ１＋ｈ１）に、画素Ａ５、Ａ６、Ａ７で算出した高さ（－ｈ２）を順次積算することにより、図５の（ｄ）に示すように、傾斜面Ｓ３の表面形状が推定される。その結果、画素列Ａ１～Ａ７に対応した物品２００の表面形状が推定される。 For the inclined surface S3 , the inclination α2 of the inclined surface S3 is detected based on the luminance values of the images captured at the pixels A5 , A6 , and A7 . The height h2 of each pixel is calculated from the detected inclination α2 and the pixel width d. As a result, the surface shape of the inclined surface S3 is estimated as shown in FIG. 5D by sequentially accumulating the height ( h1 + h1 ) accumulated at the pixel A4 by the height ( -h2 ) calculated at the pixels A5 , A6 , and A7 . As a result, the surface shape of the article 200 corresponding to the pixel rows A1 to A7 is estimated.
　画素列Ｂ１～Ｂ７、及びＣ１～Ｃ７についても、同様の方法により、画素列Ｂ１～Ｂ７、及びＣ１～Ｃ７に対応した物品２００の表面形状が推定される。これにより、撮像装置２０による１回の撮像で、物品２００の面全体の表面形状を推定することができる。 For pixel rows B 1 to B 7 and C 1 to C 7 , the surface shapes of the article 200 corresponding to the pixel rows B 1 to B 7 and C 1 to C 7 are estimated using a similar method. This makes it possible to estimate the surface shape of the entire surface of the article 200 by capturing an image once using the imaging device 20.
　このように、撮像装置２０で撮像された画像の各画素における輝度値または濃淡値に基づいて、画素毎に物品２００の表面の傾きを検出する。これとともに、検出された画素毎の傾き及び画素の幅から算出した画素毎の高さを順次積算することにより、物品２００の表面形状を推定することができる。 In this way, the inclination of the surface of the object 200 is detected for each pixel based on the brightness value or gray value of each pixel in the image captured by the imaging device 20. At the same time, the detected inclination for each pixel and the height for each pixel calculated from the pixel width are sequentially integrated, so that the surface shape of the object 200 can be estimated.
　このような方法により推定される物品２００の表面形状は、図５の（ｄ）に示すように、高さ情報を含む３次元形状として求めることができる。そのため、表面形状の相対的な高低差だけでなく、基準面に対する表面形状の高さを検査する外観検査において、特に有用である。 The surface shape of the object 200 estimated by this method can be obtained as a three-dimensional shape including height information, as shown in FIG. 5(d). Therefore, this is particularly useful in visual inspections that check not only the relative height difference of the surface shape, but also the height of the surface shape relative to a reference surface.
　［段差部を含む表面形状］
　図６は、物品２００が段差部を含む場合の推定される表面形状を説明する図である。図６の（ａ）は、物品２００の表面形状を説明する図である。物品２００の表面形状は、平坦面Ｓ１、Ｓ３、Ｓ５、及び傾斜面Ｓ２、Ｓ４を有する。平坦面Ｓ１と傾斜面Ｓ２との間、及び傾斜面Ｓ４と平坦面Ｓ５との間は、段差部Ｍ１、Ｍ２となっている。ここで、平坦面Ｓ１、Ｓ３、Ｓ５は、物品２００の基準面をなす。 [Surface shape including steps]
Fig. 6 is a diagram for explaining an estimated surface shape when the article 200 includes a step portion. Fig. 6(a) is a diagram for explaining the surface shape of the article 200. The surface shape of the article 200 has flat surfaces S1 , S3 , S5 and inclined surfaces S2 , S4 . Step portions M1 , M2 are formed between the flat surface S1 and the inclined surface S2 , and between the inclined surface S4 and the flat surface S5 . Here, the flat surfaces S1 , S3 , S5 form the reference surfaces of the article 200.
　このように、物品２００の表面形状が、物品２００の平坦面（基準面）Ｓ１、Ｓ３、Ｓ５に対して垂直な側面を有する段差部Ｍ１、Ｍ２を含む場合、段差部Ｍ１、Ｍ２の平坦面Ｓ１、Ｓ５からの高低差を把握することができない。そのため、画素毎の傾き及び画素の幅から算出した画素毎の高さを順次積算すると、平坦面Ｓ１と傾斜面Ｓ２との間、及び傾斜面Ｓ４と平坦面Ｓ５との間に、段差部Ｍ１、Ｍ２がないものとして積算されてしまう。その結果、傾斜面Ｓ２～Ｓ４は、図６の（ｂ）に示すように、平坦面Ｓ１、Ｓ５よりも下の位置にある傾斜面Ｓ’２～Ｓ’４として推定されてしまう。推定された傾斜面Ｓ’２～Ｓ’４の相対的な高さ関係は、正しく推定される。 In this way, when the surface shape of the article 200 includes step portions M 1 , M 2 having side surfaces perpendicular to the flat surfaces (reference surfaces) S 1 , S 3 , S 5 of the article 200, it is not possible to grasp the height difference of the step portions M 1 , M 2 from the flat surfaces S 1 , S 5. Therefore, when the heights of each pixel calculated from the inclination of each pixel and the width of the pixel are sequentially integrated, the integration is performed assuming that there are no step portions M 1 , M 2 between the flat surface S 1 and the inclined surface S 2 , and between the inclined surface S 4 and the flat surface S 5. As a result, the inclined surfaces S 2 to S 4 are estimated as inclined surfaces S' 2 to S' 4 located below the flat surfaces S 1 and S 5 , as shown in FIG. 6B. The relative height relationship of the estimated inclined surfaces S' 2 to S' 4 is correctly estimated.
　このように、物品２００の表面形状が、撮像装置２０で把握できない段差部を含む場合に、推定された物品２００の表面形状を補完する方法について、図７を参照しながら説明する。図７は、段差を含む場合の推定された物品２００の表面形状を補完する方法を説明する図である。以下に説明する物品２００の表面形状の補完は、補完部４０により実行される。 In this manner, a method for complementing the estimated surface shape of the article 200 when the surface shape of the article 200 includes a step portion that cannot be grasped by the imaging device 20 will be described with reference to FIG. 7. FIG. 7 is a diagram illustrating a method for complementing the estimated surface shape of the article 200 when the surface shape includes a step portion. The complementation of the surface shape of the article 200 described below is performed by the complementation unit 40.
　図７の（ａ）は、物品２００の表面形状を示す図である。物品２００の表面形状は、高さＨ１の段差部Ｍ１、Ｍ２、及び高さＨ２の段差部Ｍ３、Ｍ４を含んでいる。図７の（ｂ）は、図６の（ｂ）で説明した方法により、推定部３０で推定された表面形状を示す図である。 Fig. 7A is a diagram showing the surface shape of the article 200. The surface shape of the article 200 includes step portions M1 and M2 of height H1 and step portions M3 and M4 of height H2 . Fig. 7B is a diagram showing the surface shape estimated by the estimation unit 30 using the method described in Fig. 6B.
　上述したように、撮像装置２０では、段差部Ｍ１、Ｍ２及びＭ３、Ｍ４を把握できない。このため、傾斜面Ｓ２～Ｓ４及び傾斜面Ｓ６～Ｓ８は、図７の（ｂ）に示すように、平坦面Ｓ１、Ｓ５、Ｓ９よりも下の位置にあるそれぞれ傾斜面Ｓ’２～Ｓ’４及び傾斜面Ｓ’６～Ｓ’８として推定されてしまう。段差部Ｍ３、Ｍ４の高さＨ２は、段差部Ｍ１、Ｍ２の高さＨ１より低いので、平坦面Ｓ’７は、平坦面Ｓ’３よりも上の位置にあるように推定されてしまう。 As described above, the imaging device 20 cannot grasp the step portions M1 , M2 , M3 , and M4 . Therefore, the inclined surfaces S2 to S4 and the inclined surfaces S6 to S8 are estimated as the inclined surfaces S'2 to S'4 and the inclined surfaces S'6 to S'8 , respectively, which are located below the flat surfaces S1 , S5 , and S9, as shown in Fig. 7B. Since the height H2 of the step portions M3 and M4 is lower than the height H1 of the step portions M1 and M2 , the flat surface S'7 is estimated to be located above the flat surface S'3 .
　本実施形態における検査システムは、物品２００が、撮像装置２０において、物品２００の表面の傾きを検出できない領域（段差部）を含む場合、物品２００の表面の傾きを検出できる検出領域内の任意の基準位置における高さ情報を外部から取得し、当該取得した高さ情報に基づいて、推定部３０で推定された検出領域における表面形状を、高さ情報を含む表面形状に補完する。 In the inspection system of this embodiment, when the object 200 includes an area (step portion) where the inclination of the surface of the object 200 cannot be detected by the imaging device 20, height information at any reference position within the detection area where the inclination of the surface of the object 200 can be detected is acquired from the outside, and the surface shape in the detection area estimated by the estimation unit 30 is complemented to a surface shape including the height information based on the acquired height information.
　物品の表面形状が、設計情報に基づいて形成されている場合、検出領域内の任意の基準位置における高さ情報は、設計情報から取得することができる。高さ情報は、良品の物品２００の表面形状を測定して得られたデータから取得してもよい。物品２００の高さ情報を含む表面形状の情報は、記憶部５０に記憶されている。 If the surface shape of the object is formed based on design information, height information at any reference position within the detection area can be obtained from the design information. The height information may be obtained from data obtained by measuring the surface shape of a good object 200. Surface shape information including the height information of the object 200 is stored in the memory unit 50.
　図７の（ｃ）に示すように、記憶部５０に記憶された情報から、物品２００の表面の傾きを検出できる検出領域（ｘ１≦ｘ≦ｘ２）内の基準位置Ｖ１（ｘ＝ｘ１）における高さ情報Ｈ１を取得する。ここで、基準位置Ｖ１は、段差部Ｍ１のある位置である。高さ情報Ｈ１は、段差部Ｍ１の高さである。 7C, height information H1 at a reference position V1 (x= x1 ) in a detection area ( x1 ≦x≦ x2 ) where the inclination of the surface of the article 200 can be detected is obtained from the information stored in the memory unit 50. Here, the reference position V1 is the position where the step portion M1 is located. The height information H1 is the height of the step portion M1 .
　図７の（ｂ）に示す推定された表面形状において、基準位置Ｖ１における高さ（図７の（ｂ）ではゼロ）を、記憶部５０から取得された高さ情報Ｈ１に補完する。これにより、平坦面Ｓ１と傾斜面Ｓ２との間に、高さＨ１を有する段差部Ｍ１が補完される。 In the estimated surface shape shown in Fig. 7B, the height at the reference position V1 (zero in Fig. 7B) is supplemented to the height information H1 acquired from the storage unit 50. As a result, a step portion M1 having a height H1 is supplemented between the flat surface S1 and the inclined surface S2 .
　上述したように、傾斜面Ｓ’２～Ｓ’４の相対的な高さ関係は、正しく推定されている。このため、傾斜面Ｓ’２～Ｓ’４における高さを、段差部Ｍ１と同じ高さＨ１だけ補完することによって、推定された傾斜面Ｓ’２～Ｓ’４を、正しい高さ情報を含む傾斜面Ｓ２～Ｓ４に補完することができる。段差部Ｍ２は、段差部Ｍ１と同じ高さＨ１である。このため、傾斜面Ｓ２～Ｓ４の補完によって、自動的に、高さＨ１を有する段差部Ｍ２に補完される。 As described above, the relative height relationships of the inclined surfaces S' 2 to S' 4 are correctly estimated. Therefore, by complementing the heights of the inclined surfaces S' 2 to S' 4 by the same height H 1 as the step portion M 1 , the estimated inclined surfaces S' 2 to S' 4 can be complemented to inclined surfaces S 2 to S 4 containing correct height information. The step portion M 2 has the same height H 1 as the step portion M 1. Therefore, by complementing the inclined surfaces S 2 to S 4 , the step portion M 2 is automatically complemented to have the height H 1 .
　物品２００の表面の傾きを検出できる検出領域（ｘ３≦ｘ≦ｘ４）においても、同様の方法により、基準位置Ｖ２（ｘ＝ｘ３）における高さ情報Ｈ２を記憶部５０から取得することができる。取得した高さ情報Ｈ２に基づいて、推定された傾斜面Ｓ’６～Ｓ’８を、正しい高さ情報を含む傾斜面Ｓ６～Ｓ８に補完することができる。 In the detection region ( x3 ≦x≦ x4 ) where the inclination of the surface of the article 200 can be detected, height information H2 at the reference position V2 (x= x3 ) can be obtained from the memory unit 50 by a similar method. Based on the obtained height information H2 , the estimated inclined surfaces S'6 to S'8 can be complemented with inclined surfaces S6 to S8 including correct height information.
　図８は、段差Ｍ１、Ｍ２を含む場合の推定された物品２００の表面形状を補完する他の方法を説明する図である。上記実施形態では、補完するための高さ情報を、段差部Ｍ１、Ｍ２のある位置を基準位置Ｖ１、Ｖ２として取得する。他の方法として、例えば、図８に示すように、物品２００の表面の傾きを検出できる検出領域（ｘ１≦ｘ≦ｘ２）内の任意の位置Ｐ（ｘ＝ｘｊ）（ここでは、平坦面Ｓ’３内の位置）を基準位置として、当該基準位置Ｐにおける高さ情報（ここでは、ゼロ）を取得してもよい。この場合も、図７の（ｃ）で説明したのと同様の方法で、取得した高さ情報に基づいて、推定された傾斜面Ｓ’２～Ｓ’４を、正しい高さ情報を含む傾斜面Ｓ２～Ｓ４に補完することができる。 FIG. 8 is a diagram for explaining another method of complementing the estimated surface shape of the article 200 when the surface shape includes steps M 1 and M 2. In the above embodiment, the height information for complementation is acquired with the positions of the steps M 1 and M 2 as the reference positions V 1 and V 2. As another method, for example, as shown in FIG. 8, an arbitrary position P (x=x j ) (here, a position in the flat surface S' 3) in the detection area (x 1 ≦x≦x 2 ) in which the inclination of the surface of the article 200 can be detected may be set as the reference position, and height information (here, zero) at the reference position P may be acquired. In this case, too, the estimated inclined surfaces S' 2 to S' 4 can be complemented to inclined surfaces S 2 to S 4 including correct height information based on the acquired height information in a manner similar to that described in FIG. 7(c).
　上記実施形態では、検出領域として、傾斜面Ｓ２～Ｓ４及び傾斜面Ｓ６～Ｓ８の両方の領域に対して、高さ方向を含む表面形状に補完する例を説明した。しかし、どちらか一方の検出領域だけ補完しても構わない。 In the above embodiment, the detection areas are supplemented to the surface shape including the height direction for both the inclined surfaces S2 to S4 and the inclined surfaces S6 to S8 . However, it is also possible to supplement only one of the detection areas.
　［撮像装置で検出できない傾斜面を含む表面形状］
　上述したように、物品２００からの反射光を撮像する撮像装置２０は、物品２００の表面に対して対向した位置に固定して配置されている。このため、物品２００の平坦な基準面に対して一定以上の角度を有する傾斜面は、撮像装置２０で検出することができない。ここで、撮像装置２０で検出できない一定以上の角度は、照明装置１０からの検査光が、物品の基準面に入射する角度範囲によって、一義的に決まる。 [Surface shape including inclined surfaces that cannot be detected by imaging device]
As described above, the imaging device 20, which captures an image of the reflected light from the article 200, is fixedly disposed at a position facing the surface of the article 200. For this reason, an inclined surface having an angle of a certain degree or more with respect to the flat reference surface of the article 200 cannot be detected by the imaging device 20. Here, the angle of a certain degree or more that cannot be detected by the imaging device 20 is uniquely determined by the angle range in which the inspection light from the illumination device 10 is incident on the reference surface of the article.
　図９は、撮像装置２０で検出できない傾斜面を含む場合の推定された表面形状を補完する方法を説明する図である。図９の（ａ）は、物品２００の表面形状を示す図である。物品２００の表面形状は、平坦面Ｓ１、Ｓ４、Ｓ７に対して、角度の異なる傾斜面Ｓ２、Ｓ６、及びＳ３、Ｓ５を有している。ここで、平坦面Ｓ１、Ｓ４、Ｓ７は、物品２００の基準面をなす。平坦面Ｓ１、Ｓ７に対して、一定以上の角度αを有する傾斜面Ｓ２、Ｓ６は、撮像装置２０で検出することができない。 9 is a diagram for explaining a method for complementing an estimated surface shape when an inclined surface that cannot be detected by the imaging device 20 is included. FIG. 9(a) is a diagram showing the surface shape of an article 200. The surface shape of the article 200 has inclined surfaces S2 , S6 , and S3 , S5 having different angles with respect to flat surfaces S1 , S4 , S7 . Here, the flat surfaces S1 , S4 , S7 form the reference surfaces of the article 200. The inclined surfaces S2 , S6 having an angle α of a certain value or more with respect to the flat surfaces S1 , S7 cannot be detected by the imaging device 20.
　そのため、画素毎の傾き及び画素の幅から算出した画素毎の高さを順次積算する際、図９の（ｂ）に示すように、傾斜面Ｓ２の領域Ａ１で積算が途切れてしまう。このため、傾斜面Ｓ３では、平坦面Ｓ１の高さ（ここでは、ゼロ）から積算が開始される。従って、図９の（ａ）に示す面Ｓ３～Ｓ５は、平坦面Ｓ１よりも下の位置にある面Ｓ’３～Ｓ’５として推定されてしまう。面Ｓ’３～Ｓ’５の相対的な高さ関係は、正しく推定される。 Therefore, when the heights of each pixel calculated from the inclination and width of each pixel are sequentially accumulated, the accumulation stops at the area A1 of the inclined surface S2 , as shown in Fig. 9B. Therefore, for the inclined surface S3 , the accumulation starts from the height of the flat surface S1 (zero in this case). Therefore, the surfaces S3 to S5 shown in Fig. 9A are estimated as surfaces S'3 to S'5 located below the flat surface S1 . The relative height relationship of the surfaces S'3 to S'5 is correctly estimated.
　傾斜面Ｓ６の領域Ａ２でも積算が途切れてしまう。しかし、平坦面Ｓ７は基準面をなすため、平坦面Ｓ７の高さは正しく推定される。 The integration is also interrupted in the area A2 of the inclined surface S6 . However, since the flat surface S7 forms a reference surface, the height of the flat surface S7 is correctly estimated.
　図９の（ｃ）を参照しながら、物品２００の表面形状が、撮像装置２０で把握できない傾斜面Ｓ２、Ｓ６を含む場合に、推定された物品２００の表面形状を補完する方法について説明する。 With reference to FIG. 9C, a method for complementing the estimated surface shape of the article 200 when the surface shape of the article 200 includes inclined surfaces S 2 and S 6 that cannot be grasped by the imaging device 20 will be described.
　この場合でも、上述した段差部を含む場合と同様に、物品２００の表面の傾きを検出できる検出領域内の任意の基準位置における高さ情報を外部から取得することができる。取得された高さ情報に基づいて、推定部３０で推定された検出領域における表面形状を、高さ情報を含む表面形状に補完することができる。 In this case, as in the case where the above-mentioned step portion is included, height information at any reference position within the detection area where the inclination of the surface of the item 200 can be detected can be acquired from the outside. Based on the acquired height information, the surface shape in the detection area estimated by the estimation unit 30 can be complemented with a surface shape including the height information.
　具体的には、図９の（ｃ）に示すように、記憶部５０で記憶された情報から、物品２００の表面の傾きを検出できる検出領域（ｘ２≦ｘ≦ｘ３）内の任意の位置Ｑ（ｘ＝ｘｊ）（ここでは、平坦面Ｓ’４内の位置）を基準位置として、基準位置Ｑにおける高さ情報（ここでは、ゼロ）を取得する。 Specifically, as shown in (c) of Figure 9, from the information stored in memory unit 50, an arbitrary position Q (x = xj ) (here, a position on flat surface S'4 ) within a detection area ( x2 ≦ x ≦ x3 ) where the inclination of the surface of item 200 can be detected is set as a reference position, and height information (here, zero) at reference position Q is obtained.
　図９の（ｂ）に示す推定された検出領域の表面形状（面Ｓ’３～Ｓ’５）において、基準位置Ｑにおける高さ（ここでは、マイナスの高さ）を、取得された高さ情報（ここでは、ゼロ）に補完する。上述したように、面Ｓ’３～Ｓ’５の相対的な高さ関係は、正しく推定されている。このため、面Ｓ’３～Ｓ’５における高さを、基準位置Ｑと同じ高さだけ補完することによって、推定された面Ｓ’３～Ｓ’５を、正しい高さ情報を含むＳ３～Ｓ５に補完することができる。 In the estimated surface shape of the detection area (surfaces S'3 to S'5 ) shown in FIG. 9B, the height at reference position Q (here, a negative height) is complemented to the acquired height information (here, zero). As described above, the relative height relationship of surfaces S'3 to S'5 is correctly estimated. Therefore, by complementing the heights of surfaces S'3 to S'5 by the same height as reference position Q, the estimated surfaces S'3 to S'5 can be complemented to S3 to S5 including correct height information.
　上記実施形態では、物品２００の表面の傾きを検出できる検出領域（ｘ２≦ｘ≦ｘ３）における推定された表面形状（面Ｓ’３～Ｓ’５）を、正しい高さ情報を含むＳ３～Ｓ５に補完する例を説明した。しかし、物品２００の表面の傾きを検出できなかった非検出領域（ｘ１≦ｘ≦ｘ２；ｘ３≦ｘ≦ｘ４）についても、同様の方法により、推定部３０で推定できなかった表面形状（傾斜面Ｓ２、Ｓ６）を、高さ情報を含む表面形状に補完することができる。 In the above embodiment, an example has been described in which the estimated surface shape (surfaces S'3 to S'5 ) in the detection region ( x2 ≦x≦ x3 ) where the inclination of the surface of the article 200 can be detected is complemented to S3 to S5 including correct height information. However, for the non-detection region ( x1 ≦x≦ x2 ; x3 ≦x≦ x4 ) where the inclination of the surface of the article 200 cannot be detected, the surface shape (inclined surfaces S2 , S6 ) that could not be estimated by the estimation unit 30 can be complemented to a surface shape including height information by a similar method.
　図１０は、撮像装置２０で検出できない傾斜面を含む場合の推定された表面形状を補完する他の方法を説明する図である。具体的には、図１０に示すように、非検出領域（ｘ１≦ｘ≦ｘ２；ｘ３≦ｘ≦ｘ４）の高さを含む表面形状（傾斜面Ｓ２、Ｓ６）の情報を外部（例えば、設計情報）から取得する。取得した表面形状の情報に基づいて、非検出領域において推定できなかった表面形状を、高さ情報を含む表面形状に補完する。上述したように、面Ｓ’３～Ｓ’５の相対的な高さ関係は、正しく推定されている。したがって、非検出領域（ｘ１～ｘ２）で、傾斜面Ｓ２の表面形状を補完することによって、傾斜面Ｓ’３～Ｓ’５も、高さの正しい傾斜面Ｓ３～Ｓ５に補完することができる。非検出領域（領域ｘ３～ｘ４）ついても、同様の方法で補完することができる。 FIG. 10 is a diagram for explaining another method of complementing an estimated surface shape when an inclined surface that cannot be detected by the imaging device 20 is included. Specifically, as shown in FIG. 10, information on the surface shape (inclined surfaces S 2 , S 6 ) including the height of the non-detection region (x 1 ≦x≦x 2 ; x 3 ≦x≦x 4 ) is acquired from the outside (for example, design information). Based on the acquired surface shape information, the surface shape that could not be estimated in the non-detection region is complemented to a surface shape including height information. As described above, the relative height relationship of the surfaces S′ 3 to S′ 5 is correctly estimated. Therefore, by complementing the surface shape of the inclined surface S 2 in the non-detection region (x 1 to x 2 ), the inclined surfaces S′ 3 to S′ 5 can also be complemented to the inclined surfaces S 3 to S 5 with the correct height. The non-detection region (regions x 3 to x 4 ) can also be complemented in a similar manner.
　以上、説明したように、本実施形態では、画素毎の高さを順次積算することにより物品２００の表面形状を推定する検査システムにおいて、物品２００が、撮像装置２０において、物品２００の表面の傾きを検出できない非検出領域を含む場合であっても、物品２００の表面の傾きを検出できる検出領域内の任意の基準位置における高さ情報を外部から取得することによって、取得した高さ情報に基づいて、推定部３０で推定された検出領域における表面形状を、高さ情報を含む表面形状に補完することができる。 As described above, in this embodiment, in an inspection system that estimates the surface shape of an object 200 by sequentially accumulating the height of each pixel, even if the object 200 includes a non-detection area where the inclination of the surface of the object 200 cannot be detected by the imaging device 20, height information at any reference position within the detection area where the inclination of the surface of the object 200 can be detected can be externally acquired, and the surface shape in the detection area estimated by the estimation unit 30 can be complemented with a surface shape including height information based on the acquired height information.
　物品２００の表面の傾きを検出できない非検出領域は、物品２００の平坦な基準面に対して垂直な側面を有する段差部、または、一定以上の角度を有する傾斜面である。 The non-detection areas where the inclination of the surface of the article 200 cannot be detected are steps that have sides perpendicular to the flat reference surface of the article 200, or inclined surfaces that have an angle of a certain degree or more.
　非検出領域とは別に、意図的に検出しない領域を設けた場合でも、当該領域において、画素毎の高さを順次積算することが途切れてしまう。このため、検出領域において推定された表面形状は、高さ情報が失われてしまうという問題が生じる。この場合でも、検査システム１００を適用すれば、推定部３０で推定された検出領域における表面形状を、高さ情報を含む表面形状に補完することができる。従って、検査システム１００において、「非検出領域」は、意図的に検出しない領域も含む。 Even if an area that is not to be detected intentionally is provided in addition to the non-detection area, the sequential accumulation of heights for each pixel is interrupted in that area. This causes a problem in that the surface shape estimated in the detection area loses height information. Even in this case, by applying the inspection system 100, the surface shape in the detection area estimated by the estimation unit 30 can be complemented with a surface shape that includes height information. Therefore, in the inspection system 100, the "non-detection area" also includes an area that is not to be detected intentionally.
　物品２００の表面形状が、設計情報に基づいて形成されている場合、任意の基準位置での高さ情報は、設計情報から取得することができる。 If the surface shape of the item 200 is formed based on design information, height information at any reference position can be obtained from the design information.
　物品２００の表面の傾きを検出できない非検出領域において、非検出領域の高さを含む表面形状の情報を外部から取得してもよい。取得した表面形状の情報に基づいて、非検出領域において推定できなかった表面形状を、高さ情報を含む表面形状に補完してもよい。 In non-detection areas where the inclination of the surface of the article 200 cannot be detected, surface shape information including the height of the non-detection area may be acquired from the outside. Based on the acquired surface shape information, the surface shape that could not be estimated in the non-detection area may be complemented with a surface shape including height information.
　本実施形態における検査システムでは、推定部３０で推定した物品２００の表面形状を表示する表示部６０をさらに備えてもよい。表示部６０に、検出領域の範囲及び基準位置を表示することによって、検出領域における表面形状の補完を、より容易に実行することができる。このように、表示部６０は、検出領域の範囲、非検出領域の範囲、基準位置、及び補完された高さ情報を含む表面形状のうち少なくとも一つを表示することができる。 The inspection system in this embodiment may further include a display unit 60 that displays the surface shape of the item 200 estimated by the estimation unit 30. By displaying the range of the detection area and the reference position on the display unit 60, the surface shape in the detection area can be more easily complemented. In this way, the display unit 60 can display at least one of the range of the detection area, the range of the non-detection area, the reference position, and the surface shape including the complemented height information.
　上記実施形態では、照明装置として、図２Ａ、図３に示したように、検査光が、物品２００の表面の各点において同じ照射立体角ＩＳを有するように照射する光学系を備える照明装置を用いている。しかし、これに限定されず、例えば、分割発光照明を用いたフォトメトリックステレオ法よる光学系を備えた照明装置を用いてもよい。 In the above embodiment, as shown in Figures 2A and 3, a lighting device equipped with an optical system that irradiates the inspection light so that each point on the surface of the article 200 has the same illumination solid angle IS is used. However, this is not limited to this, and for example, a lighting device equipped with an optical system using a photometric stereo method with divided light emission lighting may be used.
　いずれの照明装置を採用しても、推定部では、撮像装置２０における撮像素子の撮像面に対する物品２００からの反射光の光軸の変化を、撮像装置で撮像された画像の各画素における輝度値または濃淡値の変化として検知する。これによって、画素毎に物品２００の表面の傾きを推定することができる。 No matter which lighting device is used, the estimation unit detects the change in the optical axis of the reflected light from the object 200 relative to the imaging surface of the imaging element in the imaging device 20 as a change in the brightness value or gray value of each pixel of the image captured by the imaging device. This makes it possible to estimate the inclination of the surface of the object 200 for each pixel.
　本開示の技術は、外観検査に用いられる照明装置として、有用である。 The technology disclosed herein is useful as a lighting device for visual inspection.
　　１０　　　照明装置
　　１１　　　面光源
　　１２　　　カラーフィルター
　　１３　　　レンズ
　　１４　　　ハーフミラー
　　２０　　　撮像装置
　　３０　　　推定部
　　４０　　　補完部
　　５０　　　記憶部
　　６０　　　表示部
　　７０　　　支持台
　　１００　　検査システム
　　２００　　物品 REFERENCE SIGNS LIST 10 Illumination device 11 Surface light source 12 Color filter 13 Lens 14 Half mirror 20 Imaging device 30 Estimation unit 40 Complement unit 50 Storage unit 60 Display unit 70 Support stand 100 Inspection system 200 Article

Claims (6)

  1. 　物品に検査光を照射する照明装置と、
    　前記検査光が照射された前記物品からの反射光を、複数の画素を有する撮像素子で受光して、前記物品の表面を撮像する撮像装置と、
    　前記撮像装置で撮像された画像の各画素における輝度値または濃淡値に基づいて、前記画素毎に前記物品の表面の傾きを検出するとともに、検出された前記画素毎の傾き及び前記画素の幅から算出した前記画素毎の高さを順次積算することにより、前記物品の表面形状を推定する推定部と、
    　前記推定部で推定した前記物品の表面形状を補完する補完部と
    を備え、
    　前記補完部は、前記物品が、前記撮像装置において、前記物品の表面の傾きを検出できない非検出領域を含む場合、前記物品の表面の傾きを検出できる検出領域内の任意の基準位置における高さ情報を外部から取得し、前記高さ情報に基づいて、前記検出領域における表面形状を、高さ情報を含む表面形状に補完する、検査システム。     An illumination device that irradiates an inspection light onto an article;
    an imaging device that receives reflected light from the object irradiated with the inspection light using an imaging element having a plurality of pixels to capture an image of a surface of the object;
    an estimation unit that detects a slope of the surface of the article for each pixel based on a luminance value or a gray value of each pixel of an image captured by the imaging device, and estimates a surface shape of the article by sequentially integrating the detected slope for each pixel and a height for each pixel calculated from a width of the pixel;
    a complementing unit that complements the surface shape of the article estimated by the estimation unit,
    The inspection system wherein, when the article includes a non-detection area where the inclination of the surface of the article cannot be detected by the imaging device, the complementation unit externally acquires height information at any reference position within a detection area where the inclination of the surface of the article can be detected, and complements the surface shape in the detection area to a surface shape including height information based on the height information.
  2. 　前記物品の表面の傾きを検出できない前記非検出領域は、前記物品の平坦な基準面に対して垂直な側面を有する段差部、または、一定以上の角度を有する傾斜面である、請求項１に記載の検査システム。 The inspection system according to claim 1, wherein the non-detection area in which the inclination of the surface of the object cannot be detected is a step having a side perpendicular to a flat reference surface of the object, or an inclined surface having an angle equal to or greater than a certain angle.
  3. 　前記物品の表面形状は、設計情報に基づいて形成されたものであり、
    　前記任意の基準位置における高さ情報は、前記設計情報から取得する、請求項１に記載の検査システム。     the surface shape of the article is formed based on design information;
    The inspection system according to claim 1 , wherein the height information at the arbitrary reference position is obtained from the design information.
  4. 　前記推定部で推定した前記物品の表面形状を表示する表示部をさらに備え、
    　前記表示部は、前記検出領域の範囲、前記非検出領域の範囲、前記基準位置、及び補完された高さ情報を含む表面形状のうち少なくとも一つを表示する、請求項１に記載の検査システム。     a display unit that displays the surface shape of the article estimated by the estimation unit,
    The inspection system according to claim 1 , wherein the display unit displays at least one of a range of the detection area, a range of the non-detection area, the reference position, and a surface shape including the interpolated height information.
  5. 　前記推定部は、前記撮像素子の撮像面に対する前記物品からの反射光の光軸の変化を、前記撮像装置で撮像された画像の各画素における輝度値または濃淡値の変化として検知することによって、前記画素毎に前記物品の表面の傾きを検出する、請求項１に記載の検査システム。 The inspection system of claim 1, wherein the estimation unit detects the inclination of the surface of the object for each pixel by detecting a change in the optical axis of the reflected light from the object relative to the imaging surface of the imaging element as a change in the luminance value or gray value of each pixel of the image captured by the imaging device.
  6. 　物品に検査光を照射する照明装置と、
    　前記検査光が照射された前記物品からの反射光を、複数の画素を有する撮像素子で受光して、前記物品の表面を撮像する撮像装置と、
    　前記撮像装置で撮像された画像の各画素における輝度値または濃淡値に基づいて、前記画素毎に前記物品の表面の傾きを検出するとともに、検出された前記画素毎の傾き及び前記画素の幅から算出した前記画素毎の高さを順次積算することにより、前記物品の表面形状を推定する推定部と、
    　前記推定部で推定した前記物品の表面形状を補完する補完部と
    を備え、
    　前記補完部は、前記物品が、前記撮像装置において、前記物品の表面の傾きを検出できない非検出領域を含む場合、前記非検出領域における高さを含む表面形状の情報を外部から取得し、前記表面形状の情報に基づいて、前記非検出領域において推定できなかった表面形状を、高さ情報を含む表面形状に補完する、検査システム。     An illumination device that irradiates an inspection light onto an article;
    an imaging device that receives reflected light from the object irradiated with the inspection light using an imaging element having a plurality of pixels to capture an image of a surface of the object;
    an estimation unit that detects a slope of the surface of the article for each pixel based on a luminance value or a gray value of each pixel of an image captured by the imaging device, and estimates a surface shape of the article by sequentially integrating the detected slope for each pixel and a height for each pixel calculated from a width of the pixel;
    a complementing unit that complements the surface shape of the article estimated by the estimation unit,
    The complementation unit, when the article includes a non-detection area where the imaging device cannot detect the inclination of the surface of the article, externally acquires information on the surface shape including the height in the non-detection area, and complements the surface shape that could not be estimated in the non-detection area to a surface shape including height information based on the surface shape information.
PCT/JP2023/038734 2022-11-29 2023-10-26 Inspection system WO2024116674A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022189788 2022-11-29
JP2022-189788 2022-11-29

Publications (1)

Publication Number Publication Date
WO2024116674A1 true WO2024116674A1 (en) 2024-06-06

Family

ID=91323689

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/038734 WO2024116674A1 (en) 2022-11-29 2023-10-26 Inspection system

Country Status (1)

Country Link
WO (1) WO2024116674A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015200539A (en) * 2014-04-07 2015-11-12 パナソニック株式会社 Liquid droplet measuring method and liquid droplet measuring system
JP2017013000A (en) * 2015-07-01 2017-01-19 パナソニックＩｐマネジメント株式会社 Droplet measuring method and droplet measuring system
JP2019100930A (en) * 2017-12-05 2019-06-24 マシンビジョンライティング株式会社 Inspection system and inspection method
JP2021196256A (en) * 2020-06-14 2021-12-27 マシンビジョンライティング株式会社 Inspection measurement system and inspection measurement method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015200539A (en) * 2014-04-07 2015-11-12 パナソニック株式会社 Liquid droplet measuring method and liquid droplet measuring system
JP2017013000A (en) * 2015-07-01 2017-01-19 パナソニックＩｐマネジメント株式会社 Droplet measuring method and droplet measuring system
JP2019100930A (en) * 2017-12-05 2019-06-24 マシンビジョンライティング株式会社 Inspection system and inspection method
JP2021196256A (en) * 2020-06-14 2021-12-27 マシンビジョンライティング株式会社 Inspection measurement system and inspection measurement method

Similar Documents

Publication Publication Date Title
US6876775B2 (en) Technique for removing blurring from a captured image
CN101971072B (en) Image sensor and focus detection apparatus
JP5808502B2 (en) Image generation device
US7105848B2 (en) Dual level out-of-focus light source for amplification of defects on a surface
US6556706B1 (en) Three-dimensional surface profile imaging method and apparatus using single spectral light condition
JP2010112941A (en) Surface inspection apparatus
KR100582317B1 (en) Monitor system for projection apparatus, projection apparatus, monitor program for projection apparatus, and monitor method for projection apparatus
CA2521857A1 (en) Method and apparatus for quantifying visual showthrough of printed images on the reverse of planar objects
JP6581293B2 (en) Measurement of rotational position of lenticular lens sheet
JPH07218451A (en) Device for optically inspecting steel plate for surface flaw
JPH05223746A (en) Method and device for detecting defect of transparent object
JP5890953B2 (en) Inspection device
WO2024116674A1 (en) Inspection system
KR102014171B1 (en) Apparatus and method for inspecting color mix defect of OLED
KR100633798B1 (en) Apparatus for testing installation condition and outer shape of a semiconductor
WO2024116634A1 (en) Inspection system
US7717574B1 (en) Method for simplifying the imaging of objects with non-Lambertian surfaces
JP2013044635A (en) Defect detecting device
JP2006105926A (en) Inspection apparatus
JP2007003332A (en) Method and detector for detecting defect on planar body side face
KR102551372B1 (en) Vision scanning device equipped with 3D shape recognition function
JP2007198762A (en) Flaw detection method and detector
WO2021131508A1 (en) Surface defect detection device, surface defect detection method and program, and surface defect detection system
JP2720360B2 (en) Painted surface inspection equipment for vehicles
JP2021004762A (en) Measurement device, imaging device, measurement system, control method, program and recording medium