JP2019200215A - Inspection device, inspection method and program - Google Patents

Abstract

To allow a flatness level (reference level) to be adjusted in a shape image indicating the surface shape of an inspection object.SOLUTION: A reference level adjustment unit 3101 adjusts a reference level being a pixel value indicating that the surface shape is flat. An image generation unit 3100 images an inspection object by a photometric stereo method, obtains a normal vector of the front surface of the inspection object, obtains a shape value indicating the shape of the front surface of the inspection object on the basis of the normal vector, allocates a pixel value larger than the reference level for the shape value indicating a protrusion out of the obtained shape values, and allocates the pixel value smaller than the reference level for the shape value indicating a recess out of the obtained shape values to generate a contour image in which the contour of the inspection object is emphasized. The generated contour image is available to determine the quality of the inspection object.SELECTED DRAWING: Figure 31

Description

  The present invention relates to an inspection apparatus, an inspection method, and a program.

  If a contour image in which the contour of the inspection object is accurately extracted can be generated, it will be useful for dimensional inspection of the inspection target, inspection of a lack of contour, and dynamic setting of the measurement region.

  Patent Document 1 describes a depth edge detection method as a technique for imaging an inspection object and extracting the contour of an object without using the principle of photometric stereo. In the depth edge detection method, a plurality of images are acquired by illuminating the inspection object from different directions, and normalization is performed with the pixel value of the maximum luminance for each image, and the brightness is changed from light to dark in each image. It has been proposed that a changing portion is detected as an edge, and information on edges detected in each image is merged to form a depth edge image.

US Patent Application Publication No. 2011/0123122

  In the depth edge detection method described in Patent Document 1, if there is a portion having a different reflectance on the surface of the inspection object, erroneous detection of an edge is likely to occur. Further, in the depth edge detection method, if there is a part that causes specular reflection on the surface of the inspection object, a double line may be generated in the part. In addition, since the edge detection process is required for each image in the depth edge detection method, the calculation load increases in proportion to the number of images. In addition, the depth edge detection method may require affine transformation.

  By the way, according to the principle of photometric stereo, it is possible to obtain a surface normal vector from a plurality of luminance images having different illumination directions, and to generate a shape image indicating the surface shape of the workpiece based on the normal vector. The photometric stereo method is a method of imaging the shape of an object, but according to the research of the inventor, it has been found that a contour image can be generated by applying this method. In addition, the photometric stereo method is advantageous for the above-described problem as compared with the depth edge detection method.

  As described above, conventionally, since the flat level (reference level) in the shape image indicating the surface shape of the inspection object cannot be adjusted, the contour image cannot be efficiently generated from the shape image. Therefore, an object of the present invention is to make it possible to adjust a flat level (reference level) in a shape image showing a surface shape of an inspection object.

According to the present invention, for example,
The inspection object is imaged by the photometric stereo method, a normal vector of the surface of the inspection object is obtained, a shape value indicating the shape of the surface of the inspection object is obtained based on the normal vector, and the obtained Of the shape values, a pixel value larger than a reference level indicating flatness is assigned to the shape value indicating a convex portion, and among the obtained shape values, a pixel value smaller than the reference level is assigned to a shape value indicating a concave portion. By assigning, image generation means for generating a shape image,
An inspection apparatus having a determination unit that determines the quality of the inspection object using the shape image,
A reference level adjusting means for adjusting the reference level;
The inspection apparatus is characterized in that the image generation means determines each pixel value constituting the shape image according to the reference level adjusted by the reference level adjustment means.

Further, according to the present invention, the reference level adjusting means for adjusting the reference level, which is a pixel value indicating that the shape of the surface of the inspection object is flat,
The inspection object is imaged by a photometric stereo method, a normal vector of the surface of the inspection object is obtained, a shape value indicating the shape of the surface of the inspection object is obtained based on the normal vector, and the obtained value is obtained. Among the obtained shape values, a pixel value larger than the reference level adjusted by the reference level adjusting unit is assigned to the shape value indicating the convex portion, and the reference level adjusting unit is assigned to the shape value indicating the concave portion among the obtained shape values. Image generation means for generating a contour image in which the contour of the inspection object is emphasized by assigning a pixel value smaller than the reference level adjusted by
There is provided an inspection apparatus comprising: a determination unit that determines the quality of the inspection object using the contour image.

  According to the present invention, it is possible to adjust the flat level (reference level) in the shape image indicating the surface shape of the inspection object. As a result, the contour image can be efficiently generated by applying the photometric stereo method. That is, it is possible to realize a contour image generation method that is not easily affected by a change in reflectance or specular reflection on the surface of the inspection object and that is light in calculation load. Further, according to the present invention, an accurate contour image can be obtained, so that the accuracy of product inspection using the contour image will be improved.

Diagram showing the outline of the inspection device Diagram for explaining the principle of photometric stereo Diagram for explaining stacking operation The figure which shows the determination method of the weight based on feature size A figure showing an example of inspection images with different feature sizes The figure explaining the image involved in the generation of the surface shape image Diagram explaining how to create a texture image Functional block diagram of inspection equipment Flow chart showing setting mode The figure which shows an example of the user interface The figure which shows an example of the user interface The figure which shows an example of the user interface The figure which shows an example of the user interface The figure which shows an example of the user interface The figure which shows an example of the user interface The figure which shows an example of the user interface The figure which shows an example of the user interface The figure which shows an example of the user interface The figure which shows an example of the user interface Flow chart showing inspection mode The figure which shows an example of the user interface The figure which shows an example of the user interface The figure which shows an example of the user interface The figure which shows an example of the user interface The figure which shows an example of the user interface The figure which shows an example of the user interface Diagram explaining the adjustment of the reference level The figure explaining the relation between the generation of shadows by illumination light and the contour The figure which shows an example of the outline image produced | generated by adjustment of a reference level The figure which shows an example of the outline image produced | generated by adjustment of a reference level The figure which shows the function of the photometric processing section

  An embodiment of the present invention is shown below. The individual embodiments described below will help to understand various concepts, such as the superordinate concept, intermediate concept and subordinate concept of the present invention. Further, the technical scope of the present invention is determined by the scope of the claims, and is not limited by the following individual embodiments.

  FIG. 1 is a diagram illustrating an example of an appearance inspection system. A line 1 is a conveyance belt that conveys a workpiece 2 that is an inspection object. The illuminating device 3 is an example of an illuminating unit that illuminates an inspection object according to a photometric stereo method. The camera 4 is an example of an imaging unit that receives reflected light from the illuminated inspection object according to a photometric stereo method and generates a luminance image. The image processing device 5 calculates a normal vector of the surface of the workpiece 2 from a plurality of luminance images acquired by the camera 4, and an inclination image composed of pixel values based on the normal vectors calculated from the plurality of luminance images. And the reduced image of the tilt image, the pixel value of the pixel of interest is calculated using the normal vector of the adjacent pixel adjacent to the pixel of interest, an inspection image having the pixel value is generated, and the inspection image is used. It is an appearance inspection apparatus that determines the quality of an inspection object. The tilt image is sometimes called a normal vector image. The image processing device 5 may create a reflectance image (albedo image) from the luminance image. The display unit 7 displays a user interface for setting control parameters related to the inspection, an inclination image, a reflectance image, an inspection image, and the like. The input unit 6 is a console, a pointing device, a keyboard, or the like, and is used for setting control parameters. The image processing device 5 and the illumination device 3 are connected by a signal line 8. The image processing device 5 and the camera 4 are connected by a signal line 9.

  In particular, according to FIG. 1, the camera 4 and the illumination device 3 are supported by different frames so that they can move independently. Thus, since the illuminating device 3 can move independently from the camera 4, the distance from the workpiece 2 to the illuminating device 3 can be freely adjusted. That is, by arranging the illumination device 3 away from the work 2 in accordance with the type and placement of the work 2, the camera 4 can strongly receive regular reflection light. Further, by arranging the illumination device 3 close to the workpiece 2, the camera 4 can strongly receive diffuse reflection light. Note that the camera 4 and the illumination device 3 may be instructed by the same frame. In this case, what is necessary is just to fix the illuminating device 3 to a flame | frame with the clamp mechanism etc. for adjusting the attachment position of the illuminating device 3. FIG.

<Principle of photometric stereo>
In a general photometric stereo method, as shown in FIG. 2, illumination light L1 to L4 is irradiated onto a work 2 while sequentially switching from four directions to generate four luminance images. The direction of the illumination light used when photographing each luminance image is only one direction. The luminance image is composed of a plurality of pixels, and four pixels having the same coordinates in the four luminance images correspond to the same work surface. 2 is established between the pixel values (luminance values) I1, I2, I3, and I4 of the four pixels and the normal vector n. Here, ρ is the reflectance. L is the amount of illumination light from each direction and is known. Here, the light quantity is the same in all four directions. S is an illumination direction matrix and is known. By solving this mathematical formula, the reflectance ρ and the normal vector n for each coordinate (work surface) are obtained. As a result, a reflectance image and a tilt image are obtained.

  In the present embodiment, the height component is further extracted from the tilt image, and a shape image indicating the shape of the work is created as an inspection image. The inspection image is obtained by a stacking arithmetic expression which is Expression 2 shown in FIG. Here, zn is the result of the n-th stacking and indicates the shape of the workpiece surface. x and y indicate pixel coordinates. n indicates the number of repeated calculations. p indicates a horizontal inclination component, and q indicates a vertical inclination component. p and q are obtained from the normal vector n. w is a weight. In addition, a 1/1 tilt image is used for the first stacking operation, a 1/2 reduced tilt image is used for the second stacking operation, and a 1/4 reduced tilt image is used for the third stacking operation. When creating a reduced image, reduction processing may be performed after performing Gaussian processing.

  In this embodiment, a parameter called feature size is employed in the stacking calculation. The feature size is a parameter that gives a weight to the components of the reduced image used in the accumulation operation. The feature size is a parameter indicating the size of the surface shape of the workpiece 2. For example, if the feature size is 1, the weights for the four pixels adjacent to the pixel of interest in the xy direction are maximized and the calculation is performed. If the feature size is 2, the weights for the eight pixels adjacent to the pixel of interest in the xy direction are set to be the largest, and the accumulation operation is executed. However, since calculation using eight pixels causes an increase in the amount of calculation, the above-described reduced image is created and used for the calculation. That is, instead of using eight adjacent pixels, the tilt image is reduced to ½ and the calculation is executed. Thus, four pixels in the reduced image may be taken into consideration for a certain target pixel. Even when the feature size increases to 4, 8, 16, 32, a reduced image corresponding to the feature size is created and the weight is set to the maximum for the reduced image corresponding to the feature size. Is obtained.

  FIG. 3 shows an example of a stacking operation. In this example, two tilt images obtained from the normal vector n (a horizontal tilt component p image and a vertical tilt component q image) are input. First, the entire shape is accumulated with an inclination image having a large reduction degree, and the detailed shape is accumulated with an image having a smaller reduction degree. As a result, the entire shape can be restored in a short time. According to FIG. 3, for example, the shape z of the workpiece surface is calculated for the target pixel using Equation 2 for a 1/32 reduced image. The weight w is determined according to the feature size. The accumulation operation is iterated (repeatedly processed) with each pixel constituting the reduced image as a target pixel. The initial value of z is zero. Next, z is calculated for a 1/16 reduced image according to Equation 2. Here, the inclination components of 1/16 reduced images are accumulated with respect to the 1/32 calculation result. Similarly, a stacking operation is performed from a 1/8 reduced image to a 1/1 image.

  FIG. 4 shows an example of the weight for each feature size. The horizontal axis indicates the resolution level (reduction degree), and the vertical axis indicates the weight. As can be seen from FIG. 4, in the feature size 1, the weight of level 0 (1/1 image) having the smallest reduction degree is the largest. Thereby, it becomes possible to accumulate fine shapes. In feature size 2, the weight of level 1 (1/2 image) is the maximum. Thereby, it becomes possible to pile up the shape of a bigger size. Thus, each weight is determined so that a peak occurs at a level corresponding to the feature size.

  As a method for restoring a shape image, a known Fourier transform integration method can be adopted in addition to the above-described accumulation operation (A Method for Enforcing Integrability in Shape from Shading Algorithms, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 10, No.4 July 1988). Also in this method, it is possible to change the feature size to be extracted by generating a reduced image in the calculation process and adjusting the weighting component.

  FIG. 5 shows an example of an inspection image corresponding to a difference in feature size. It can be seen that the detailed shape is extracted in the feature size 4, the entire shape is extracted in the feature size 64, and the intermediate size is extracted in the feature size 16. The small feature size is useful for inspecting fine scratches, the large feature size is suitable for determining the presence or absence of an object, and the intermediate feature size is suitable for OCR of uneven characters. That is, it is possible to improve inspection accuracy by selecting an appropriate feature size according to the inspection tool.

  FIG. 6 is a diagram showing a process of creating an inspection image by the photometric stereo method. The luminance images 601 to 604 are luminance images obtained by illuminating the work 2 with illumination light having different illumination directions. Note that the luminance image 600 is a luminance image obtained by simultaneously illuminating from four directions. A normal vector of the workpiece surface is obtained by calculation from a plurality of luminance images obtained by illuminating the workpiece 2 with illumination light having different illumination directions. The tilt image 611 is a tilt image in which the X direction tilt component of the normal vector obtained from the luminance images 601 to 604 is used as a pixel value. The tilt image 612 is a tilt image in which the Y direction tilt component of the normal vector obtained from the luminance images 601 to 604 is a pixel value. The reflectance image 610 is a reflectance image obtained by removing the variation of the luminance value due to the tilt of the workpiece surface from the normal vector obtained from the luminance images 601 to 604 and using the reflectance of the workpiece surface as an image. The inspection images 621 to 623 are images (surface shape images) having different feature sizes obtained from the tilt images 611 and 612, respectively. The inspection images 621 to 623 are also a kind of tilt image because they are composed of pixels based on the tilt component. An inspection image of the work 2 is generated by such a procedure. Depending on the inspection tool, the luminance image 600 and the reflectance image 610 that are omnidirectional illumination images may be adopted as the inspection image. An omnidirectional illumination image is a luminance image acquired by turning on all of a plurality of light sources included in the illumination device 3.

<Texture information>
The texture information is information based on the reflectance ρ of the surface of the work 2. The reflectance ρ is obtained by Equation 1, that is, one reflectance image is obtained from four luminance images. The reflectance image is an image having a pixel value proportional to the reflectance ρ of the workpiece surface. As shown in FIG. 7, a normal vector is calculated from four luminance images 701 to 704, and the reflectance of each pixel is calculated based on the calculated normal vector and the luminance value of each corresponding pixel in the plurality of luminance images. The texture images 711 and 712, which are reflectance images, are obtained by calculating pixel values proportional to. As this synthesis method, there are a method for obtaining a texture image by pixel average of four luminance images, a method for obtaining a texture image by pixel average after removing halation from the four luminance images, and the like. The texture image 711 is obtained by image averaging, and the texture image 712 is an example obtained by removing halation. There are four pixels with the same coordinates in the four luminance images. Eliminate halation by excluding the pixel with the largest pixel value from the four pixels, or excluding the first to Nth pixels (N is a natural number of 3 or less) in the descending order of the pixel value. Is possible. This is because halation appears in the image as high luminance. Since texture images 711 and 712 are both composed of pixels based on reflectance, they are a kind of reflectance image.

<Functional block>
FIG. 8 is a block diagram of the inspection apparatus. In this example, the illumination device 3, the camera 4, and the image processing device 5 are each housed in separate housings, but this is only an example and may be appropriately integrated. The illumination device 3 is an example of illumination means for illuminating an inspection object according to a photometric stereo method, and includes a light source group 801 and an illumination controller 802 that controls the light source group 801. One segment may be composed of a plurality of light emitting elements, and the light source group 801 may be composed of a plurality of segments. The number of segments is generally four, but may be three or more. This is because if the workpiece 2 can be illuminated from three or more illumination directions, an inspection image can be generated by the photometric stereo method. As shown in FIG. 1, the outer shape of the lighting device 3 may be ring-shaped. Moreover, the illuminating device 3 may be comprised by the some illumination unit each isolate | separated. For example, there are lighting units in the market that are used to photograph the workpiece 2, but these are not developed for photometric stereo. However, you may comprise the illuminating device 3 by preparing several such lighting units and connecting the lighting controller which controls these. The illumination controller 802 controls the lighting timing and lighting pattern (lighting pattern) of the light source group 801 in accordance with a control command from the image processing apparatus 5. Although the illumination controller 802 is described as being incorporated in the illumination device 3, it may be incorporated in the camera 4, the image processing device 5, or a housing independent of these. It may be accommodated.

  The camera 4 is an example of an imaging unit that receives reflected light from an inspection object illuminated according to a photometric stereo method and generates a luminance image, and executes imaging processing according to a control command from the image processing device 5 To do. The camera 4 may create a luminance image of the work 2 and transfer it to the image processing device 5, or transfer a luminance signal obtained from the image sensor to the image processing device 5, and the image processing device 5 generates a luminance image. May be. Since the luminance signal is a signal from which the luminance image is based, the luminance signal is also a luminance image in a broad sense.

  The image processing apparatus 5 is a kind of computer, and includes a processor 810 such as a CPU and an ASIC, a storage device 820 such as a RAM, a ROM, and a portable storage medium, an image processing unit 830 such as an ASIC, and a communication such as a network interface. Part 850. The processor 810 is in charge of setting an inspection tool, adjusting control parameters, and generating / regenerating / updating an inspection image. The photometric processing unit 811 calculates a normal vector n of the surface of the work 2 from a plurality of luminance images acquired by the camera 4, and has a pixel value based on the normal vector n calculated from the plurality of luminance images. An arithmetic unit that accumulates the pixel value of the target pixel using the normal vector n of the adjacent pixel adjacent to the target pixel and generates an inspection image having the pixel value for the image and the reduced image of the tilt image ( Function as inspection image generation means). Specifically, the inspection image is generated using the above-described mathematical formula and the like. The illumination control unit 812 controls a lighting pattern, illumination switching timing, and the like by transmitting a control command to the illumination controller 802. The imaging control unit 813 controls the camera 4. The UI management unit 814 displays a user interface (UI) for setting an inspection tool, a UI for setting parameters necessary for generating an inspection image, and the like on the display unit 7 and inputs from the input unit 6. Set information and thus inspection tools and parameters. In particular, the feature size setting unit 815 functions as a setting unit that sets a feature size that is a parameter that gives a weight w to a component of a reduced image used in a stacking operation. The image selection unit 816 selects an image to be displayed from among a plurality of luminance images, a plurality of inspection images, a plurality of tilt images, and a plurality of reflectance images. The image selection unit 816 may select an image to be stored or output from among a plurality of luminance images and inspection images acquired by the camera 4. The inspection tool setting unit 817 sets an inspection tool for the inspection image selected by the image selection unit 816. For example, the inspection tool setting unit 817 sets a flaw inspection area or a character recognition area for the reference image. The reference image setting unit 818 sets a reference image that is an inspection image acquired from a non-defective product. The display control unit 851 switches the luminance image and the inspection image to be displayed on the display unit 7 or displays the luminance image and the inspection image at the same time. Further, when the control parameter is adjusted, the display control unit 851 updates the image displayed on the display unit 7 to an image reflecting the control parameter. The inspection tool setting unit 817 may include a display control unit 851, a feature size setting unit 815, an image selection unit 816, a reference image setting unit 818, and a condition setting unit 819. The image processing unit 830 functions as an inspection area setting unit that performs a pattern search on the inspection image using the reference image and sets an inspection area (eg, a flaw inspection area or a character recognition area) in the inspection image. The inspection area is, for example, a character recognition area. The condition setting unit 819 sets conditions for outputting an image to an external device connected to the display unit 7 and the communication unit 850, and conditions for saving the image in a portable storage medium. The determination unit 840 functions as a determination unit that determines the quality of the workpiece 2 using the inspection image. For example, the determination unit 840 receives the result of the inspection performed using the inspection image in the image processing unit 830 and determines whether the inspection result satisfies a non-defective product condition (tolerance or the like).

  The storage device 820 stores luminance image data 821 that is luminance image data acquired by the camera 4, tilt image data 822 and reflectance image data 823 generated by the photometric processing unit 811. The storage device 820 also stores various setting data, a program code for generating a user interface, and the like. The storage device 820 may store and hold inspection images having different feature sizes. In addition to the inspection image, the storage device 820 may store tilt image data and reflectance image data that are the basis of the inspection image. These will help to identify whether there is a problem with the inspection image, the tilt image or the reflectance image and to correct the control parameters when an erroneous determination of the workpiece 2 is found.

  The image processing unit 830 performs an appearance inspection using the inspection image (tilt image data 822 and reflectance image data 823) generated by the photometric processing unit 811. The flaw inspection unit 831 performs a flaw inspection in a flaw inspection region of a plurality of inspection images generated using different feature sizes. The OCR unit 832 functions as a character recognition processing unit that executes character recognition processing on a plurality of inspection images generated using different feature sizes. The wound inspection unit 831 and the OCR unit 832 read the inspection image (tilt image data 822 and reflectance image data 823) stored in the storage device 820, perform inspection in the character recognition area, and store the inspection result in the storage device 820. It may be written or passed to the determination unit 840. The determination unit 840 determines the quality of the workpiece 2 based on the inspection result.

<Setting mode>
The inspection system has a setting mode for setting an inspection tool and an inspection mode (operation mode) for executing an appearance inspection of the workpiece 2 in accordance with the set inspection tool. Here, an example of the setting mode will be described.

  FIG. 9 is a flowchart regarding the setting mode. When the start of the setting mode is instructed through the input unit 6, the UI management unit 814 of the processor 810 displays a UI for setting the inspection tool on the display unit 7.

  FIG. 10 shows an example of a UI. The UI 1000 displayed on the display unit 7 by the UI management unit 814 is provided with a pull-down menu 1001 for designating a storage destination of the inspection result and a text box 1002 for inputting the name of the inspection tool. When the UI management unit 814 detects that the execution button is pressed, the UI management unit 814 displays the next UI.

  A UI 1100 illustrated in FIG. 11 includes a guidance 1101 for setting an inspection tool, a measurement execution button 1102 for instructing the camera 4 to perform imaging, a display area 1103 for displaying an image captured by the camera 4, and camera settings. A camera setting button 1104 for instructing start is provided. Note that the image selection unit 1105 is a button for selecting an image to be displayed in the display area 1103 or an image to be used for inspection. In this example, the image selection unit 1105 selectively selects one of the images of shape 1, shape 2, texture, and normal. When the measurement execution button 1102 is operated, the imaging control unit instructs the camera 4 to perform imaging. The UI management unit 814 renders the luminance image acquired by the camera 4 in the display area 1103. When another image is selected by the image selection unit 1105, the UI management unit 814 renders the image selected by the image selection unit 1105 in the display area 1103. As described above, the user can switch the image displayed in the display area 1103 by operating the image selection unit 1105 or instructing the switching of the image through the input unit 6. When the camera setting button 1104 is operated, the UI management unit 814 switches to the next UI.

  In step S <b> 901, the UI management unit 814 displays a UI on the display unit 7 for setting the camera 4, and executes camera setting. FIG. 12 shows an example of the camera setting UI 1200. The camera setting tab 1201 includes a pull-down menu 1202 for setting the camera model, a pull-down menu 1203 for setting the image size, a pull-down menu 1204 for setting the shutter speed, and a slider 1205 for setting the sensitivity of the camera. When the measurement execution button 1102 is operated, the UI management unit 814 displays the luminance image acquired by the camera 4 in the display area 1103 according to the imaging parameter set at that time. Thereby, the user can determine whether the set parameter is appropriate.

  In step S902, the UI management unit 814 displays a UI on the display unit 7 for setting photometric processing, and executes the setting. For example, when detecting that the photometric stereo setting tab 1210 provided in the camera setting UI 1200 is operated, the UI management unit 814 switches the photometric stereo setting tab 1210 to be valid as shown in FIG. To switch to enable means to switch the display state of the photometric stereo setting tab 1210 to a state in which the user can operate. The photometric stereo setting tab 1210 includes a pull-down menu 1301 for selecting an image and a feature size setting unit 1302. In this example, it is assumed that one of three inspection images (shape 1, shape 2, and shape 3) having different feature sizes can be selected. A feature size is set by the feature size setting unit 1302 for each image selected by the pull-down menu 1301.

  A selection unit for selecting a lighting pattern may be arranged in the photometric stereo setting tab 1210. In addition, a designation unit that designates a light emission amount per one illumination may be provided.

  In step S903, the UI management unit 814 displays a UI for setting the inspection tool on the display unit 7, and executes the setting. FIG. 14 shows an example of a UI 1400 for setting an inspection tool. The image selection button 1401 is a button for selecting an inspection image to be used for inspection from among a plurality of inspection images. The inspection category selection button 1402 is a button for selecting a category of a tool to be added as an inspection tool from among a plurality of inspection categories. The recognition target setting button 1403 is a button for selecting one of a plurality of recognition targets. In this example, “shape 1” is selected as the inspection image, “recognition” is selected as the category, and “character recognition” is selected as the recognition process. When the add button 1404 is operated, the UI management unit 814 switches to the next UI. FIG. 15 shows a reference image registration UI 1500. The reference image registration UI 1500 includes a registration button 1501 in addition to the above-described measurement execution button 1102 and display area 1103. When the registration button 1501 is operated, the UI management unit 814 registers the image acquired by the measurement execution button 1102 and displayed in the display area 1103 as a reference image. When registration is completed, the UI management unit 814 switches to the next UI.

  FIG. 16 shows a measurement area setting UI 1600. A reference image 1601 and a frame 1602 indicating the measurement area are arranged in the display area 1103 of the measurement area setting UI 1600. The UI management unit 814 changes the position and size of the frame 1602 according to an instruction from the input unit 6. The user adjusts the position and size of the frame 1602 in accordance with the position and size of the portion to be measured in the reference image 1601. Further, the UI management unit 814 may perform character cutout settings, dictionary settings for registering specific examples (character images) of characters to be recognized and character characters corresponding to the character images, and the like.

  Next, the flaw inspection tool will be described. As shown in FIG. 17, when a wound inspection is selected by the inspection category selection button 1402, the UI management unit 814 displays an inspection content selection button 1701. In this example, a tool for measuring the total area of the wound is selected by the inspection content selection button 1701. When the add button 1404 is operated, the UI management unit 814 switches the UI.

  FIG. 18 shows a measurement area setting UI 1800. In the measurement area setting UI 1800, a frame 1802 indicating a measurement area (scratch inspection area) is arranged. The shape of the frame 1802 can be changed. For example, one of a plurality of shapes is selected by a pull-down menu 1801 for selecting a shape. The UI management unit 814 renders the frame 1802 having the shape selected by the pull-down menu 1801 in the display area 1103. The UI management unit 814 changes the position and size of the frame 1802 in accordance with an instruction from the input unit 6.

  FIG. 19 shows a setting UI 1900 for setting a flaw detection condition. The setting UI 1900 includes a pull-down menu 1901 for selecting a flaw detection direction, a box 1902 for designating a flaw segment size, and a slider 1903 for designating a flaw level. When the flaw inspection unit 831 detects a flaw within the flaw inspection area (within the frame 1802) based on the flaw detection condition set by the setting UI 1900, the UI management unit 814 displays a flaw detection mark 1910 at the flaw position. May be. Thereby, the user will be able to judge whether the flaw detection condition is appropriate.

<Inspection mode>
FIG. 20 is a flowchart showing the inspection mode. When the start of the inspection mode is instructed through the input unit 6, the processor 810 shifts the operation mode to the inspection mode.

  In step S2001, the processor 810 captures and acquires an image of the work 2 while switching the illumination direction according to the set lighting pattern. Specifically, the illumination control unit 812 identifies a lighting pattern with reference to setting data held in the storage device 820, and sends a command for designating the lighting pattern to the lighting controller 802. The imaging control unit 813 specifies control parameters (such as shutter speed and sensitivity) related to the camera 4 with reference to the setting data held in the storage device 820, and transmits a command for specifying the control parameter to the camera 4. The photometric processing unit 811 transmits a trigger signal for instructing the start of illumination to the illumination controller 802 and transmits a trigger signal for instructing the start of imaging to the camera 4 in conjunction with the trigger signal. The illumination controller 802 switches the illumination direction in synchronization with the trigger signal. For example, the lighting controller 802 turns on the light emitting elements corresponding to the four lighting directions one by one in accordance with the lighting pattern specified by the command. The lighting controller 802 may hold the correspondence between the command and the lighting pattern in a memory or the like. Only one trigger signal may be issued at the start of illumination, or may be issued at the switching timing. The camera 4 captures the workpiece 2 according to the control parameters and transfers the luminance image to the image processing device 5. In this way, for example, one luminance image is generated for one illumination direction.

  In S2002, the processor 810 obtains a normal vector n and a reflectance ρ from a plurality of luminance images. As described above, the photometric processing unit 811 applies Equation 1 to the pixel values of a plurality of luminance images to obtain the normal vector n and the reflectance ρ.

  In step S2003, the processor 810 generates an inspection image according to the set feature size. As described above, the photometric processing unit 811 determines the weight W corresponding to the feature size from the weight table or the like, and executes a stacking operation using Expression 2 to generate an inspection image (tilt image). As described above, the photometric processing unit 811 may generate an inclination image having a pixel value based on the normal vector n on the surface of the workpiece 2 from a plurality of luminance images. When a plurality of feature sizes having different values are set, the photometric processing unit 811 may generate an inspection image for each of the set feature sizes. The photometric processing unit 811 may generate a reflectance image or a texture image by the above-described method. For example, the photometric processing unit 811 calculates the reflectance ρ of the surface of the workpiece 2 together with the normal vector n of the surface of the workpiece 2 from a plurality of luminance images, and generates a reflectance image having a pixel value based on the reflectance ρ. May be. Here, an image to be inspected is generated, and generation of an image not to be inspected may be omitted.

  In step S2004, the processor 810 displays the inspection image on the display unit 7. The UI management unit 814 may display the luminance image, the tilt image, and the reflectance image together with the inspection image on the display unit 7 simultaneously or selectively. In the case of selective display, the UI management unit 814 may switch and display, for example, four luminance images in accordance with a switching instruction from the input unit 6. For example, a specific key provided on the console of the input unit 6 may be assigned as an image switching button.

  In step S2005, the processor 810 instructs the image processing unit 830 to execute inspection. When the inspection is instructed, the image processing unit 830 activates a preset inspection tool and executes the inspection on the inspection image. For example, the wound inspection unit 831 determines the level of the scratch according to the set measurement region and detection condition, and transfers the inspection result (scratch level) to the determination unit 840. Note that the flaw inspection unit 831 may perform a pattern search using the above-described reference image to set an inspection region, and perform inspection in the inspection region. In addition, the OCR unit 832 performs character recognition processing on the inspection image in accordance with preset character recognition settings, and transfers the character recognition result to the determination unit 840. The OCR unit 832 may perform a pattern search using the above-described reference image to set an inspection area (character recognition area), and execute an inspection in the inspection area.

  In S2006, the determination unit 840 of the processor 810 compares the inspection result with the determination threshold value to determine whether the workpiece 2 is a non-defective product. For example, in a case where both the wound inspection and the OCR are set to be executed, the determination unit 840 determines whether the inspection result of the wound inspection unit 831 and the character recognition result of the OCR unit 832 are at a pass level. 2 is judged as a good product.

<Image save settings>
FIG. 21 shows an example of a UI 2100 for setting an inspection flow. The UI management unit 814 displays the UI 2100 on the display unit 7, and sets a plurality of processes to be executed from the start to the end of the inspection flow according to instructions input from the input unit 6. In this example, an imaging process, a pattern search process, a position correction process, and a flaw inspection process are added to the inspection flow. For example, when the end of the inspection flow is designated through the input unit 6, the UI management unit 814 may be set to accumulate the inspection history at the end. The inspection history is an inspection result or an image used for the inspection.

  When adding each process, the UI management unit 814 may accept selection of an image used in each process through the input unit 6. For example, the user designates four luminance images, tilt images, reflectance images, and the like that are different from the illumination direction for the imaging process through the input unit 6 as acquisition targets, and any luminance image (for the pattern search process). An omnidirectional illumination image or the like) may be designated as a search target, and an inspection image or the like generated from an inclination image may be designated as an inspection target for a scratch inspection process.

  FIG. 22 shows an example of a UI 2200 for setting conditions for accumulating history. The setting unit 2201 for setting identification information for identifying the accumulation condition is configured by a pull-down menu for selecting identification information to be set from a plurality of identification information. In this example, an accumulation condition of identification information “0:” is selected in the setting unit 2201. Examples of the accumulation condition include a condition that an image is accumulated only when the inspection result is not a non-defective product, and a condition that an image is always accumulated for each workpiece without depending on the inspection result. Here, the processor 810 activates the condition setting unit 819 when detecting that the detailed setting button or the like is pressed. For example, the condition setting unit 819 sets one of a mode in which an image is always stored or output and a mode in which an image is stored or output when the determination unit 840 determines that the inspection target is not a good product. May be. The image selection unit 2202 selects an image to be saved when the accumulation condition is satisfied. Here, “all” and “designation” can be selected by the image selection unit 2202. The storage destination selection unit 2203 includes a pull-down menu for selecting an image storage destination (for example, portable media such as a built-in memory or a memory card, or a network storage such as an FTP server).

  FIG. 23 shows an example of a UI 2300 that the UI management unit 814 displays on the display unit 7 when “designation” is selected by the image selection unit 2202. In this example, a check box 2301 for selecting an image to be actually saved among all types of images handled in the inspection flow is provided. Shapes 1 and 2 are inspection images (tilt images) having different feature sizes. The texture is a reflectance image. Normal is an image acquired by omnidirectional illumination. Four arrows are icons indicating the illumination direction. That is, four luminance images having different illumination directions are distinguished by the arrow marks. An image whose check box is checked is set as a save target.

  By the way, the processor 810 may include a determination unit that determines whether a condition for saving or outputting an image is satisfied after the determination unit 840 completes the determination. That is, at the end part of the inspection flow, the processor 810 may determine whether the storage condition and the output condition set by the condition setting unit 819 are satisfied.

  FIG. 24 shows an example in which an image output process 2401 is added to the inspection flow. In the above-described embodiment, an image is set to be output at the end of the inspection flow. However, in this example, the UI management unit 814 performs an image output process 2401 at an arbitrary position in the inspection flow according to a user instruction input from the input unit 6. Set. As described above, the processor 810 may determine whether or not a condition for saving or outputting an image is satisfied in the image output step 2401 positioned before the determination unit 840 ends the determination. The accumulation settings and the like related to the image output process 2401 may be the same as or different from those described with reference to FIGS.

  FIG. 25 shows another example of a UI related to accumulation setting (output setting). When the image output process 2401 is selected by the input unit 6 and an instruction to start setting is further input by the input unit 6, the UI management unit 814 displays the UI 2501. The image variable 2502 functions as an image selection unit that selects an image to be output. In this example, an image to be output is designated by an image variable assigned to each process in the inspection flow. That is, an image to be output can be selected for each process. In the UI 2501, the number of output images and the image format may be set. An output destination selection unit 2503 is a pull-down menu for selecting an image output destination (for example, a portable storage medium such as a built-in memory or a memory card or a network storage such as an FTP server).

  FIG. 26 shows an example of a UI 2600 for selecting an image. When the detailed setting button is pressed on the UI 2501, the UI management unit 814 displays the UI 2600. In the UI 2600, radio buttons for selecting whether all images are stored or individually designated, check boxes for individually selecting images, and the like are arranged. In this example, since individual designation is selected by a radio button, the check box is valid, and several images are selected through the check box. As described above, an image to be stored or output may be selected from a combined luminance image obtained by combining a plurality of luminance images, an inspection image, an omnidirectional illumination image, and a plurality of luminance images. Further, the UI 2600 may be configured such that an image to be saved or output can be selected from a plurality of inspection images each having a different feature size. Further, the UI 2600 may be configured so that an image to be stored or output can be selected from a plurality of luminance images, an inspection image, and a reflectance image using the reflectance of the surface of the inspection object as a pixel value.

<Generation of contour-enhanced image>
If a contour image obtained by accurately extracting the contour of the inspection object can be generated, it will be useful for dimensional inspection, chip inspection, and dynamic setting of the measurement region.

  FIG. 27A to FIG. 27D are diagrams for explaining a reference level in a shape image. The reference level is a pixel value indicating that the surface shape of the inspection object is flat, and may be called a flat level. FIG. 27A is a diagram for explaining a general reference level. Here, the recorded image is an 8-bit image as an example, but the number of bits is arbitrary, such as a 16-bit image. In an 8-bit image, each pixel value can take a value from 0 to 255. Therefore, by setting the reference level to 128, the convex shape and the concave shape can be expressed with a good balance. That is, the concave shape is represented by the gray values of 0 to 127, and the convex shape is represented by the gray values of 129 to 255. Thus, bright pixels indicate that the surface shape is convex, and dark pixels indicate that the surface shape is concave. In this way, the shape image can indicate the surface shape by shading.

  As shown in FIG. 27B, in this embodiment, the user can freely adjust the reference level so that a contour image can be formed efficiently. FIG. 27C shows an example in which the reference level is set to 0, which is the minimum value of the shade expression range. By setting the reference level to 0, the concave shape can be ignored (the shape value corresponding to flatness or the shape value of the concave portion among the calculated shape values is assigned to 0 as a pixel value (shading value)), Only the convex shape can be extracted (the convex shape value is assigned to the gray value of 1 to 255). FIG. 27D shows an example in which the reference level is set to 255, which is the maximum value of the shade expression range. By setting the reference level to 255, the convex shape can be ignored, and only the concave shape can be extracted.

  In the photometric stereo method, the normal vector of the surface of the workpiece is obtained from the difference in brightness between a plurality of luminance images having different illumination directions, the surface inclination (X component, Y component) is estimated, and the X component Although the shape value is calculated by weighting and combining the inclination and the inclination of the Y component, the shadow of the work may be reflected in the luminance image depending on the illumination direction. Such a shadow has been considered undesirable because it causes an estimation error in the tilt, but it has been found that the occurrence of this shadow in the vicinity of the contour of the workpiece can be used for the extraction of the contour.

  FIG. 28A is a side view showing a shadow generated by applying illumination light from the right side of the workpiece. FIG. 28B is a plan view showing a shadow generated by applying illumination light from the right side of the workpiece. FIG. 28C is a side view showing a shadow generated by applying illumination light from the left side of the workpiece. FIG. 28D is a plan view showing a shadow generated by applying illumination light from the left side of the workpiece. FIG. 28E shows an estimation result of the inclination in the X direction obtained based on the luminance image acquired by the right illumination light and the luminance image acquired by the left illumination light. As can be seen from FIG. 28E, the inclination is erroneously estimated in the shadow portion. FIG. 28F shows a shape image when the reference level is set to an intermediate value of 128. The shape image is formed from four luminance images corresponding to the four illumination directions. Of the surface of the workpiece, the portion where the surface shape is flat is gray because the gray value is 128. Since the outline of the workpiece is convex, the gray value thereof is greater than 128 and is white. Since the shadow portion has a concave shape, the gray value is smaller than 128 and is black. If the dimension of the workpiece is measured using the shape image as it is as the contour image, a dimensional error occurs due to the influence of the shadow. FIG. 28G shows a shape image as a contour image obtained by setting the reference level to 0 by the reference level adjusting means of this embodiment. It can be seen that the concave shape can be eliminated by setting the reference level to 0, and only the convex shape, that is, only the correct contour can be accurately extracted. In this way, when a convex contour is necessary, the contour-enhanced image can be obtained by lowering the reference level below the intermediate value. On the other hand, when a concave contour is required, a contour-enhanced image can be obtained by increasing the reference level from the intermediate value.

  FIG. 29A is an example of a shape image obtained by setting the reference level to 128 for convex characters formed on the surface of a credit card or the like. As this example shows, black shadows appear around the characters, which reduces the accuracy of character recognition. FIG. 29B is an example of a shape image obtained by setting the reference level to 0 for convex characters formed on the surface of a credit card or the like. As this example shows, the influence of the shadow is reduced and the characters can be clearly recognized.

  FIG. 29C is an example of a shape image obtained by setting the reference level to 128 for concave characters formed on the surface of the workpiece. As shown in this example, it can be seen that characters are difficult to recognize. FIG. 29D is an example of a shape image obtained by setting the reference level to 255 for concave characters formed on the surface of the workpiece. If the reference level is adjusted as shown in this example, even a concave character can be clearly recognized.

  FIG. 30A is an example of a shape image of the cooling fan and is obtained by setting the reference level to 128. FIG. 30B is an example of a shape image of the cooling fan and is obtained by setting the reference level to 0. By reducing the reference level, it becomes possible to highlight only the workpiece outline, and the contour image can be used to efficiently and accurately perform dimension inspection, chipping inspection, and dynamic measurement area setting by pattern search. It becomes like this.

  FIG. 31 is a block diagram illustrating functions of the photometric processing unit 811. The reference level adjustment unit 3101 adjusts a reference level that is a pixel value indicating that the surface shape is flat. The UI management unit 814 displays a user interface for adjusting the reference level on the display unit 7, receives an operation on the user interface by the input unit 6, and inputs a numerical value or level input or designated through the user interface to the reference level adjustment unit 3101 may be passed. Examples of the user interface include a slider bar and a numerical value input unit that can adjust the reference level between a minimum value and a maximum value. The reference level adjustment unit 3101 adjusts the reference level according to a numerical value or level input or designated through the user interface. The adjusted reference level is passed to the image generation unit 3100.

  The image generation unit 3100 captures the workpiece 2 by the photometric stereo method with the camera 4, obtains a normal vector of the surface of the workpiece 2, obtains an inclination based on the normal vector, and indicates a shape value indicating the surface shape according to the inclination The shape value is converted into a gray value by assigning the shape value to the pixel value according to the adjusted reference level, and a contour image in which the contour of the workpiece 2 is emphasized is generated. As described above, the image generation unit 3100 expresses the concave shape by the gray value from the minimum value to the reference level, expresses flatness by the gray value of the reference level, and expresses the convex shape by the gray value from the reference level to the maximum value. To do. The image generation unit 3100 may include a tilt image generation unit 3110 and a contour image generation unit 3120. The inclination image generation unit 3110 obtains a normal vector of the surface of the work 2 from a plurality of luminance images obtained by imaging the work 2 by the photometric stereo method, and further obtains an inclination from the normal vector. That is, the tilt image generation unit 3110 obtains a normal vector for each pixel based on the pixel values of a plurality of pixels whose coordinates match in the luminance image for each illumination direction, and obtains the tilt from the normal vector. Also good. Thereby, an inclination image in which each pixel value represents an inclination may be generated. However, since it is sufficient to know the inclination here, it is not essential to generate an inclination image as image data. That is, the inclination calculation unit 3111 calculates the inclination and passes the calculated inclination to the contour image generation unit 3120.

  The contour image generation unit 3120 functions as a shape image generation unit and a contour image generation unit. The shape value calculation unit 3121 of the contour image generation unit 3120 may calculate the shape value of each surface using the x component inclination image and the y component inclination image to generate the shape image. The method for obtaining the shape value is as described with reference to FIGS. The correction unit 3122 of the contour image generation unit 3120 enhances the contour of the work 2 included in the shape image by converting the shape value into a pixel value (shading value) according to the reference level. Generate an image. For example, the correction unit 3122 assigns a pixel value larger than the reference level indicating flatness to the shape value indicating the convex portion among the shape values obtained by the shape value calculation unit 3121. Further, the correction unit 3122 assigns a pixel value smaller than the reference level for the shape value indicating the recess. As described above, the contour image generation unit 3120 may obtain the shape value based on the inclination, and may convert the shape value into a gray value according to the reference level to generate the contour image data 3150. Data may be received and shape image data converted to contour image data 3150 using the adjusted reference level afterwards. In any case, the correction unit 3122 corrects the shape value (pixel value) indicating the surface shape according to the reference level and converts it into a gray value.

  The contrast adjustment unit 3102 is an option, and increases the contrast of the contour image when the reference level adjustment unit 3101 decreases the reference level. When the reference level is lowered, the contrast of the image may be lowered. Therefore, the contrast may be recovered by image processing.

  The contour image data 3150 is stored in the storage device 820, and the image processing unit 830 performs a dimensional inspection of the workpiece, the scratch inspection unit 831 performs a scratch inspection (chip inspection), and the OCR unit 832 performs character recognition. It is read and used when executing. The image processing unit 830 sets an inspection region for the non-defective contour image, executes a search process using the non-defective contour image, determines the position of the workpiece, and determines the position of the workpiece according to the workpiece position. The position of the inspection area (measurement area) may be corrected. As described above, the contour image data 3150 is also useful when the measurement region is dynamically set. Although the workpiece 2 is conveyed by the line 1, it is not always conveyed at the same position as viewed from the camera 4. Therefore, in order to improve the accuracy of the appearance inspection, the position correction used for the position determination result is necessary for the luminance image of the work 2 and the image generated from the luminance image. The contour image is also effective when performing such position correction. The determination unit 840 refers to various inspection results (eg, dimension measurement results, scratch inspection results, character recognition results, etc.) obtained by using the contour image in the image processing unit 830, and compares the inspection results with tolerances. The quality of the workpiece may be determined.

  The correcting unit 3122 may generate a contour image by emphasizing the contour of the convex portion of the contour of the work 2. As described with reference to FIG. 27C, the correction unit 3122 sets the minimum value (eg, 0) as the reference level within the possible range of the pixel value (shading value) in the contour image. The outline may be emphasized. The correcting unit 3122 may generate an outline image by emphasizing the outline of the recess in the outline of the work 2. For example, the correction unit 3122 emphasizes the contour of the recess by setting the maximum value (eg, 255 for 8-bit image, 65,535 for 16-bit image) to the reference level in the possible range of pixel values in the contour image. May be.

<Summary>
According to this embodiment, since the reference level adjustment unit 3101 is provided, the flat level (reference level) in the shape image indicating the surface shape of the inspection object can be adjusted. Further, according to the present embodiment, a contour image can be generated by applying the photometric stereo method. That is, it is possible to realize a contour image generation method that is not easily affected by a change in reflectance or specular reflection on the surface of the workpiece and that is light in computational load. In addition, according to the present embodiment, an accurate and accurate contour image can be obtained, so that the accuracy of product inspection using the contour image will be improved. As described above, by enabling the user to adjust the reference level for converting the shape value of the workpiece surface obtained according to the photometric stereo method into a gray value, the dynamic range can be effectively used. . For example, if the reference level is set to an intermediate value (for example, 128), a part of the gray value range is not used, and the dynamic range may not be used effectively. For example, in a shape image in which there are no gray values from 0 to 87, the range is wasted. In such a case, by setting the reference level to 88 which is the minimum value in the shape image, the shape values from 0 to 87 are deleted (the slopes from 0 to 87 are all expressed by the minimum gray value). It is possible to associate a range from 0 to 255 with a shape value that actually exists. Therefore, the dynamic range is effectively used.

  A user interface for adjusting the reference level may be displayed on the display unit 7. This will allow the user to easily display the reference level. Note that the image generation unit 3100 may generate a contour image in real time in conjunction with the adjustment of the reference level and display the preview image on the display unit 7. As a result, the user can easily visually grasp how the adjustment amount of the reference level is reflected in the contour image. Therefore, it will be easier to find an appropriate reference level more accurately.

  As described with reference to FIGS. 29A and 29B, for convex characters, character recognition of convex characters is performed by adjusting the reference level so that the contours of the convex portions are emphasized. The rate will improve. For example, the contour of the convex portion can be easily emphasized by setting the minimum value to the reference level in the range of pixel values that can be taken in the contour image. As described with reference to FIGS. 29C and 29D, the character recognition rate of the concave character is adjusted by adjusting the reference level so that the contour of the concave portion is emphasized for the concave character. Will improve. For example, the contour of the concave portion can be easily emphasized by setting the maximum value to the reference level within the possible range of the pixel value in the contour image.

  The correction unit 3122 may input already completed shape image data and adjust the reference level for output. As a result, data stored in the past can be used effectively.

  The contour image can be used not only for the character recognition inspection described above but also for various appearance inspections such as dynamic setting of an inspection area, flaw inspection, and dimension measurement. In particular, according to the present embodiment, the contour image can be generated efficiently and accurately, so that the inspection efficiency and accuracy will be improved.

  In the present embodiment, the appearance inspection apparatus for inspecting the appearance of the workpiece has been described as an example. However, the image processing apparatus connected to the camera 4 may be equipped with the reference level adjustment function and the contour image generation function of the present embodiment.

  In the present embodiment, it is assumed that the photometric stereo method is used. However, the image generation unit 3100 adjusts the reference level afterwards for the image data generated by the non-photometric stereo method to obtain the contour image. It may be generated. Although the effect of the photometric stereo method cannot be obtained, it is possible to easily adjust the reference level and form a contour image.

  DESCRIPTION OF SYMBOLS 1 ... Line, 2 ... Work, 3 ... Illuminating device, 4 ... Camera, 5 ... Image processing apparatus, 6 ... Input part, 7 ... Display part

According to the present invention, for example,
A plurality of luminance images obtained by irradiating light from different directions are synthesized by a photometric stereo method to obtain an inclination image of an x component and an inclination image of a y component on the surface of the inspection object, and the x component and the y component A reference level is assigned to a pixel corresponding to a portion where the surface shape of the inspection object is flat based on the inclination of the inspection object, and a shape image having a gray value corresponding to the uneven shape of the inspection object surface is generated as an inspection image Image generating means for
UI management means for displaying an inspection image generated by the image generation means on a display unit and for displaying a user interface for adjusting the reference level on the display unit;
Input means for receiving an adjustment operation of the reference level for the user interface,
The image generation means adjusts each pixel value constituting the shape image according to a reference level adjusted via the input means,
The UI management unit causes the display unit to display a shape image in which each pixel value is adjusted.
An inspection apparatus characterized by the above is provided.

Moreover, according to the present invention,
A plurality of luminance images obtained by irradiating light from different directions are synthesized by a photometric stereo method to obtain an inclination image of an x component and an inclination image of a y component on the surface of the inspection object, and the x component and the y component A reference level is assigned to a pixel corresponding to a portion where the surface shape of the inspection object is flat based on the inclination of the inspection object, and a shape image having a gray value corresponding to the uneven shape of the inspection object surface is generated as an inspection image An image generation process to
Displaying the inspection image generated in the image generating step on the display unit, and displaying the user interface for adjusting the reference level on the display unit;
Receiving a reference level adjustment operation for the user interface,
The image generation step includes a step of adjusting each pixel value constituting the shape image according to a reference level adjusted based on the adjustment operation received in the reception step,
The display step includes a step of causing the display unit to display a shape image in which each pixel value is adjusted . A method for controlling an inspection apparatus is provided.

Claims (19)

The inspection object is imaged by the photometric stereo method, a normal vector of the surface of the inspection object is obtained, a shape value indicating the shape of the surface of the inspection object is obtained based on the normal vector, and the obtained Of the shape values, a pixel value larger than a reference level indicating flatness is assigned to the shape value indicating a convex portion, and among the obtained shape values, a pixel value smaller than the reference level is assigned to a shape value indicating a concave portion. Assigning the image generation means for generating the shape image,
An inspection apparatus having a determination unit that determines the quality of the inspection object using the shape image,
A reference level adjusting means for adjusting the reference level;
The inspection apparatus, wherein the image generation means determines each pixel value constituting the shape image according to a reference level adjusted by the reference level adjustment means.   The inspection apparatus according to claim 1, wherein the reference level adjustment unit adjusts the reference level so that an outline of the inspection object in the shape image is emphasized. A reference level adjusting means for adjusting a reference level which is a pixel value indicating that the shape of the surface of the inspection object is flat;
The inspection object is imaged by a photometric stereo method, a normal vector of the surface of the inspection object is obtained, a shape value indicating the shape of the surface of the inspection object is obtained based on the normal vector, and the obtained value is obtained. Among the obtained shape values, a pixel value larger than the reference level adjusted by the reference level adjusting unit is assigned to the shape value indicating the convex portion, and the reference level adjusting unit is assigned to the shape value indicating the concave portion among the obtained shape values. Image generation means for generating a contour image in which the contour of the inspection object is emphasized by assigning a pixel value smaller than the reference level adjusted by
An inspection apparatus comprising: a determination unit that determines the quality of the inspection object using the contour image.   The inspection apparatus according to claim 3, further comprising a contrast adjusting unit that increases a contrast of the contour image when the reference level adjusting unit decreases the reference level. Display means for displaying the contour image and displaying a user interface for adjusting the reference level;
The inspection apparatus according to claim 3, wherein the reference level adjustment unit adjusts the reference level according to a numerical value or level input or designated through the user interface.   The inspection apparatus according to claim 3, wherein the image generation unit generates the contour image by emphasizing a contour of a convex portion of the contour of the inspection object.   The inspection apparatus according to claim 6, wherein the image generation unit emphasizes the contour of the convex portion by setting a minimum value as a reference level within a range of pixel values in the contour image.   6. The inspection apparatus according to claim 3, wherein the image generation unit generates the contour image by emphasizing a contour of a concave portion of the contour of the inspection object.   The inspection apparatus according to claim 8, wherein the image generation unit emphasizes the contour of the concave portion by setting a maximum value as a reference level within a possible range of pixel values in the contour image. The image generating means includes
A plurality of inclination image generation means for imaging the inspection object by a photometric stereo method and generating an inclination image having a pixel value corresponding to the inclination of the surface of the inspection object;
A shape image generating means for generating a shape image having a pixel value corresponding to the shape of the surface of the inspection object based on the plurality of tilt images;
By correcting each pixel value of the shape image in accordance with the reference level adjusted by the reference level adjusting means, the contour of the inspection object included in the shape image is emphasized and the inspection object 10. The inspection apparatus according to claim 3, further comprising contour image generation means for generating a contour image. Setting means for setting an inspection region for a non-defective contour image;
Position determining means for determining a position of the inspection object by performing a search process using the contour image of the non-defective product;
11. The inspection according to claim 3, further comprising: a position correcting unit that corrects a position of the inspection area with respect to a contour image of the inspection object according to a position of the inspection object. apparatus. Character recognition means for executing character recognition processing on the contour image;
12. The inspection apparatus according to claim 3, wherein the determination unit determines whether the inspection object is good based on a character recognition result obtained by the character recognition unit. Further comprising a wound inspection means for inspecting the presence or absence of a scratch on the contour of the inspection object using the contour image;
The inspection apparatus according to any one of claims 3 to 12, wherein the determination unit determines the quality of the inspection object based on the inspection result of the scratch obtained by the scratch inspection unit. Further comprising dimension measuring means for measuring the dimension of the inspection object using the contour image;
14. The inspection apparatus according to claim 3, wherein the determination unit determines whether the inspection object is good based on a measurement result obtained by the dimension measurement unit. Illuminating means for illuminating the inspection object alternatively from at least three illumination directions;
Imaging means for imaging the inspection object in synchronization with illumination by the illumination means and generating a luminance image for each illumination direction;
The image generation means obtains a normal vector for the pixel based on pixel values of a plurality of pixels whose coordinates match in the luminance image for each illumination direction, obtains an inclination from the normal vector, and further The inspection apparatus according to claim 3, wherein a shape value is obtained from an inclination. The inspection object is imaged by the photometric stereo method, a normal vector of the surface of the inspection object is obtained, a shape value indicating the shape of the surface of the inspection object is obtained based on the normal vector, and the obtained Of the shape values, a pixel value larger than a reference level indicating flatness is assigned to the shape value indicating a convex portion, and among the obtained shape values, a pixel value smaller than the reference level is assigned to a shape value indicating a concave portion. Assigning the image generation means for generating the shape image,
Reference level adjusting means for adjusting the reference level;
The image processing apparatus, wherein the image generation means determines each pixel value constituting the shape image according to a reference level adjusted by the reference level adjustment means. Illuminating means for illuminating the inspection object alternatively from at least three illumination directions;
Imaging means for imaging the inspection object in synchronization with illumination by the illumination means and generating a luminance image for each illumination direction;
A reference level adjusting means for adjusting a reference level which is a pixel value indicating that the shape is flat;
Based on the pixel values of a plurality of pixels whose coordinates match in the luminance image for each illumination direction, a normal vector for the pixel is obtained, and a plurality of in accordance with the inclination of the surface of the inspection object from the normal vector Image generation means for correcting the pixel value according to the reference level adjusted by the reference level adjustment means, and generating an outline image in which the outline of the inspection object is emphasized;
An inspection apparatus comprising: a determination unit that determines the quality of the inspection object using the contour image. A reference level adjustment step of adjusting a reference level that is a pixel value indicating that the shape of the surface of the inspection object is flat;
The inspection object is imaged by a photometric stereo method, a normal vector of the surface of the inspection object is obtained, a shape value indicating the shape of the surface of the inspection object is obtained based on the normal vector, and the obtained value is obtained. Of the obtained shape values, a pixel value larger than the reference level adjusted in the reference level adjustment step is assigned to the shape value indicating the convex portion, and among the obtained shape values, the reference level adjustment step is applied to the shape value indicating the concave portion. An image generation step of generating a contour image in which the contour of the inspection object is emphasized by assigning a pixel value smaller than the reference level adjusted in
And a determination step of determining the quality of the inspection object using the contour image. A reference level adjustment step of adjusting a reference level that is a pixel value indicating that the shape of the surface of the inspection object is flat;
The inspection object is imaged by a photometric stereo method, a normal vector of the surface of the inspection object is obtained, a shape value indicating the shape of the surface of the inspection object is obtained based on the normal vector, and the obtained value is obtained. Of the obtained shape values, a pixel value larger than the reference level adjusted in the reference level adjustment step is assigned to the shape value indicating the convex portion, and among the obtained shape values, the reference level adjustment step is applied to the shape value indicating the concave portion. An image generation step of generating a contour image in which the contour of the inspection object is emphasized by assigning a pixel value smaller than the reference level adjusted in
A program for causing a computer to execute a determination step of determining pass / fail of the inspection object using the contour image.