JP2021092439A - Illumination optimization method, control device, and program - Google Patents

Abstract

To provide an illumination optimization method that can improve accuracy of defect detection.SOLUTION: An illumination parameter of an inspection device inspecting an object is determined according to photographic images of the object photographed by utilizing a plurality of lighting groups. An illumination optimization method includes: steps S101-S103 for acquiring a first image of an object photographed by an inspection device by applying a plurality of illumination conditions respectively for a plurality of lighting groups; step S105 for generating a plurality of second images by applying an image filter utilized in the inspection device when inspecting the object, with respect to the plurality of first images corresponding to the plurality of illumination conditions; and steps S106-S108 for calculating, as an illumination parameter for controlling the plurality of lighting groups, a weight value that minimizes an error between a weighted linear sum and a target value of corresponding pixel values in the plurality of second images, corresponding to an attention region of the object.SELECTED DRAWING: Figure 8

Description

The present disclosure relates to lighting optimization methods, controls, and programs.

Inspection techniques that analyze external images and automatically detect defects on the surface of an object are widely used in various fields. Inspection techniques that utilize external images can be applied to various objects of different sizes, materials, and shapes, such as tablets, food containers, and semiconductor chips. An inspection device including a camera that captures an external image of the object, a lighting device that illuminates the object, and a computer that determines the presence or absence of defects from the external image is used for the inspection of the object.

In the above inspection technique, the appearance image is analyzed to determine the presence or absence of defects. Therefore, in order to improve the accuracy of defect detection, it is important to obtain an appearance image suitable for defect detection. For example, when handling an object having an uneven surface, the shadow generated on the uneven portion may be erroneously determined as a defect depending on how the light is applied. Further, in the case of an object having a glossy surface, the contrast in the appearance image increases depending on the type of light source and how the light is applied, while the contrast of the defective portion relatively decreases, so that the surface of the object has a glossy surface. Defects such as small scratches can be overlooked.

Regarding the above circumstances, in Patent Document 1 below, on the premise of using an inspection device provided with a plurality of lights, a plurality of lighting patterns for evaluation are prepared, and an external image taken by turning on the lights in each lighting pattern is evaluated. A method of determining a lighting pattern for measurement has been proposed. In this proposed method, a lighting pattern that maximizes the difference in feature amount between the area of the object (feature area) where the scratch is provided and the other area is obtained by the method of solving the mathematical optimization problem. ..

Japanese Unexamined Patent Publication No. 2019-148438

In the above proposed method, the feature amount is extracted from the feature area and other areas by using the appearance image as it is, and the lighting pattern is determined based on the extracted feature amount. That is, this proposed method does not assume the application of an image filter to an external image, and when an image filter suitable for defect detection is applied to an external image, a lighting pattern that does not match the characteristics of the image filter may occur. .. In this case, the non-scratched portion is emphasized by the image filter, and there is a risk that the portion is erroneously determined to be a scratch.

According to one aspect of the present disclosure, an object of the present disclosure is to provide a lighting optimization method, a control device, and a program capable of improving the accuracy of defect detection.

According to one aspect of the present disclosure, it is a lighting optimization method for determining a lighting parameter of an inspection device that inspects an object based on a photographed image of the object taken by using illumination by a plurality of lighting groups. A step of acquiring a first image of an object taken by an inspection device by applying each of a plurality of lighting conditions for a plurality of lighting groups, and a target for a plurality of first images corresponding to a plurality of lighting conditions. A step of generating a plurality of second images by applying an image filter used in an inspection device when inspecting an object, and weighting of corresponding pixel values in the plurality of second images corresponding to a region of interest of the object. A lighting optimization method is provided in which a computer performs a process that includes a step of calculating a weight value that minimizes the error between the linear sum and the target value as a lighting parameter for controlling multiple lighting groups. ..

Further, according to another aspect of the present disclosure, a control device that executes each of the above steps, an inspection device that includes the control device, a program that causes a computer to execute each of the above steps, and a non-stored program. A non-transitory computer-readable storage medium may be provided.

According to the present disclosure, the accuracy of defect detection can be improved.

It is a figure which showed typically the structure of the inspection apparatus which concerns on this embodiment. It is a figure which showed typically the arrangement example of the camera, the illumination, and the polarizing plate mounted on the inspection apparatus which concerns on this embodiment. It is a figure which showed the example of the polarizing plate mounted on the inspection apparatus which concerns on this embodiment, and the positional relationship between a polarizing plate and illumination schematically. It is a figure which shows typically the arrangement example of the camera, the illumination, the polarizing plate, and the prism mounted on the inspection apparatus which concerns on this embodiment, and the optical path of the reflected light. It is a figure which showed typically the example of the photographed image photographed by the inspection apparatus which concerns on this embodiment. It is a block diagram which showed the example of the hardware which can realize the function of the control device which concerns on this embodiment. It is a block diagram for demonstrating the function of the control apparatus which concerns on this embodiment. It is a flow chart for demonstrating the lighting optimization method which concerns on this Embodiment.

An embodiment of the present disclosure (hereinafter referred to as the present embodiment) will be described below with reference to the accompanying drawings. In the present specification and the drawings, elements having substantially the same function may be designated by the same reference numerals to omit duplicate description.

The technique according to the present embodiment can be applied to an inspection device that inspects an object based on a photographed image of the object to be inspected (hereinafter referred to as the object). In particular, the technique according to the present embodiment relates to a lighting optimization method for optimizing the lighting at the time of photographing to the lighting suitable for inspection. In the following, the inspection device to which the technique according to the present embodiment can be applied will be described, and the lighting optimization method according to the present embodiment will be described in detail.

[1. Inspection device]
A configuration example of the inspection device according to the present embodiment will be described with reference to FIG. FIG. 1 is a diagram schematically showing a configuration of an inspection device according to the present embodiment. The inspection device 10 shown in FIG. 1 is an example of the inspection device according to the present embodiment. In the following description, in order to facilitate the description, the description will proceed assuming a tablet inspection device for inspecting various tablets. However, the technique according to the present embodiment can be applied to various inspection devices for inspecting other objects having a shape and material different from those of tablets, and such modifications will be described later.

As shown in FIG. 1, the inspection device 10 includes a display device 11, a control device 12, a camera 13, and a lighting unit 14. Further, the inspection device 10 includes a transport belt 15 for transporting the object P and rollers 16a and 16b for driving the transport belt 15.

The display device 11 may be a display device such as an LCD (Liquid Crystal Display) or an ELD (Electro-Luminescence Display). The control device 12 may be a computer such as a PC (Personal Computer), a server device, or a workstation, or a control computer having substantially the same functions as these. The camera 13 is an imaging device including an optical system such as a lens and an image sensor such as a CCD (Charged-coupled devices) sensor or a CMOS (Complementary metal-oxide-semiconductor) sensor.

The lighting unit 14 includes a plurality of lights for irradiating the object P transported by the transport belt 15 to the photographing region of the camera 13 (intersection of the optical axis of the optical system and the transport belt 15 or its vicinity). .. Further, the illumination unit 14 guides the light emitted from each illumination and the polarizing plate for controlling the polarization state of the light reflected by the object P and the light reflected by the object P to the optical system of the camera 13. It may include an optical member (prism or the like) for the purpose.

As suggested by the arrow a in FIG. 1, the object P is sequentially placed on the transport belt 15. The rotation of the rollers 16a and 16b causes the mounting surface of the transport belt 15 to move in the direction of the arrow b, and the object P on the transport belt 15 is transported to the inside of the lighting unit 14. In the example of FIG. 1, the object P located inside the lighting unit 14 is represented by a broken line. The object P conveyed inside the lighting unit 14 is further carried to the photographing area of the camera 13 and illuminated by each illumination of the lighting unit 14. Then, the appearance is photographed by the camera 13.

Although not shown, a mechanism may be provided for picking up the object P transported to the outside of the lighting unit 14 and photographing the object P from the back side. For example, another transport belt that the mounting surface abuts and attracts to the upper surface of the object P, and a photographing unit that captures an external image of the object P transported by the other transport belt may be provided. In this case, the photographing unit provides substantially the same functions as the camera 13 and the lighting unit 14. By providing such a mechanism, it becomes possible to efficiently inspect the front surface, the back surface, and each side surface of the object P.

(Internal structure)
Here, the camera, the illumination, and the polarizing plate mounted on the inspection device according to the present embodiment will be further described with reference to FIG. FIG. 2 is a diagram schematically showing an arrangement example of a camera, lighting, and a polarizing plate mounted on the inspection device according to the present embodiment. The arrangement example shown in FIG. 2 is only an example, and the positional relationship between the camera, the illumination, and the polarizing plate, the number of illuminations, the type of illumination, and the presence or absence of the polarizing plate may be changed.

FIG. 2 schematically shows a cross section of the camera 13 and the lighting unit 14 cut along a surface including the optical axis of the camera 13. In the example of FIG. 2, the object P is located directly below the camera 13, and the center of the upper surface of the object P coincides with the optical axis. The position of the object P at the time of shooting is not limited to the example of FIG. 2, and a reference position may be set according to the type of the object P and the like. In the following, for the sake of simplicity, it is assumed that the object P is photographed by the camera 13 at the timing when the object P is located directly under the camera 13.

As already mentioned, the lighting unit 14 has a plurality of lights. Each lighting is composed of a light emitting element such as an LED (Light Emitting Diode) and a substrate on which the light emitting element is placed. A plurality of light emitting elements may be placed on the substrate of each illumination, and one illumination may be formed by the plurality of light emitting elements. Further, each illumination may be composed of only a light emitting element that emits white light, or may include a light emitting element that emits colored light (for example, red, green, or blue light). In addition, each illumination may include a light emitting element that emits white light and a color filter. Although FIG. 2 illustrates LED lighting having a flat light emitting surface, dome-shaped lighting, ring lights, and the like may be used.

FIG. 2 schematically shows the illuminations 21a, 21b, 22a, 22b, 23a, and 23b among the illuminations included in the illumination unit 14. In the example of FIG. 2, the illuminations 21a and 21b are arranged so that the upper surface of the object P and the light emitting surface are substantially parallel to each other, and can irradiate light in a direction parallel to the optical axis of the camera 13 and its periphery. The illuminations 22a and 22b are arranged diagonally above the object P and irradiate a region including the upper surface and the side surface of the object P with light. The illuminations 23a and 23b are arranged so that the side surface of the object P and the light emitting surface are substantially parallel to each other, and irradiate the region including the side surface and the upper surface of the object P with light.

Further, FIG. 2 schematically shows the polarizing plates 31, 32, 33a, and 33b included in the lighting unit 14. In the following, for the sake of simplicity, the surface of the polarizing plate 32 that faces the light emitting surface of the illumination 22a is referred to as the polarizing plate surface 32a, and the surface that mainly faces the light emitting surface of the illumination 22b is referred to as the polarizing plate surface 32b. May be done. Further, among the polarizing plates 32, the surface arranged substantially perpendicular to the optical axis may be referred to as the upper surface of the polarizing plate 32.

In the example of FIG. 2, the polarizing plate 31 is arranged so as to be substantially perpendicular to the optical axis of the camera 13. The upper surface of the polarizing plate 32 is arranged so as to face the illuminations 21a and 21b. Further, the polarizing plate 33a is arranged at a position facing the light emitting surface of the illumination 23a. Further, the polarizing plate 33b is arranged at a position facing the light emitting surface of the illumination 23b.

The polarizing plates 31, 32, 33a, 33b have an optical property of cutting a monopolarizing component (hereinafter, first polarizing component) of passing light. Therefore, the polarizing plate 31 can cut the first polarization component of the light reflected by the object P that is incident on the optical system through the vicinity of the optical axis of the camera 13. Further, the polarizing plate surface 32a can cut the first polarization component of the light emitted from the illumination 22a. Further, the polarizing plate surface 32b can cut the first polarization component of the light emitted from the illumination 22b. Further, the upper surface of the polarizing plate 32 can mainly cut the first polarization component of the light emitted from the illuminations 21a and 21b. Further, the polarizing plate 33a can cut the first polarization component of the light emitted from the illumination 23a. Further, the polarizing plate 33b can cut the first polarization component of the light emitted from the illumination 23b.

As described above, the light emitted from the illuminations 21a, 21b, 22a, 22b, 23a, 23b has the first polarization component cut by the polarizing plates 31, 32, 33a, 33b. Therefore, the light composed of the second polarization component whose polarization direction is orthogonal to the first polarization component hits the object P. The light reflected by the object P passes through the polarizing plate 31 and enters the optical system of the camera 13.

Of the light reflected by the object P, the specular reflection component is mainly composed of the first polarization component. Therefore, the specular reflection component can be cut by the polarizing plate 31. On the other hand, among the light reflected by the object P, the diffuse reflection component includes the second polarization component. Therefore, the second polarized light component of the diffuse reflection component passes through the polarizing plate 31 and enters the optical system of the camera 13.

By cutting the specular reflection component, even when the glossy object P is photographed with strong lighting, the reflection of the light source and the occurrence of flare and ghost that occur when strong light enters the optical system are suppressed. be able to. The reflection of the light source and the generation of ghosts cause a contrast irrelevant to the appearance of the object P in the captured image, which may cause erroneous detection. Further, since the occurrence of flare lowers the contrast as a whole, it can be a factor of lowering the detection accuracy. However, by arranging the polarizing plate as described above, the cause of erroneous detection can be eliminated and the decrease in detection accuracy can be suppressed.

The shape of each polarizing plate can be arbitrarily deformed as long as the specular reflection component can be removed by the mechanism described above. For example, the polarizing plate 32 may have a shape as shown in FIG.

Here, with reference to FIG. 3, an example of the polarizing plate mounted on the inspection apparatus according to the present embodiment and the positional relationship between the polarizing plate and the illumination will be further described. FIG. 3 is a diagram schematically showing an example of a polarizing plate mounted on the inspection device according to the present embodiment and the positional relationship between the polarizing plate and the illumination. The structure of the polarizing plate and the positional relationship between the polarizing plate and the illumination shown in FIG. 3 are merely examples, and the structure and the positional relationship may be changed.

FIG. 3 schematically shows the upper surface of the polarizing plate 32. As shown in FIG. 3, the polarizing plate 32 is composed of an octagonal upper surface having a circular hole in the center and a trapezoidal polarizing plate surface 32a, ..., 32h having one side of the octagon as the upper side. .. The optical characteristics of the polarizing plate 32 can be realized, for example, by attaching a polarizing film to the surface of the transparent plate that is the base material of the polarizing plate 32. The transparent plate may be a material that transmits light, and may be a glass plate, an acrylic plate, or the like. Further, the surface to which the polarizing film is attached may be the front surface of the transparent plate or the back surface.

In FIG. 3, the rectangles drawn with broken lines on the upper surface of the polarizing plate 32 and the portions of the polarizing plate surfaces 32a, 32b, 32c, and 32d indicate the area where the illumination is arranged. For example, the illumination 22a is arranged so that the polarizing plate surface 32a and the light emitting surface face each other. The light emitting surface of the illumination 22a and the polarizing plate surface 32a do not have to be in contact with each other. As described with reference to FIG. 2, the polarizing plate surface 32a is arranged between the illumination 22a and the imaging position of the object P. The same applies to the positional relationship between the polarizing plate surfaces 32b, 32c, 32d and the illuminations 22b, 22c, 22d.

Illuminations 21a, ..., 21h are arranged at positions facing the upper surface of the polarizing plate 32. In the example of FIG. 3, eight lights (lights 21a, ..., 21h) are arranged in a ring shape, but one dome-shaped light or a ring light may be provided instead. Further, a part of the illuminations 21a, ..., 21h may be omitted. For example, the lights 21a, 21b, 21c, 21d may be omitted, or the lights 21e, 21f, 21g, 21h may be omitted.

In the example of FIG. 3, the illumination is not arranged at the positions facing the polarizing plate surfaces 32e, 32f, 32g, and 32h. This is because the camera 13 captures an image of the object P that can be seen through the polarizing plate surfaces 32e, 32f, 32g, and 32h, as will be described later with reference to FIG. As a modification, in the case of adopting a mechanism for capturing only the light entering the optical system of the camera 13 in the optical path shown by the broken line in FIG. 2, the positions facing the polarizing plate surfaces 32e, 32f, 32g, and 32h are illuminated, respectively. May be placed.

Here, with reference to FIG. 4, an arrangement example of a camera, an illumination, a polarizing plate, and a prism mounted on the inspection device according to the present embodiment and an optical path of reflected light will be described. FIG. 4 is a diagram schematically showing an arrangement example of a camera, lighting, a polarizing plate, and a prism mounted on the inspection device according to the present embodiment and an optical path of reflected light. The arrangement example of FIG. 4 is only an example, and the positional relationship between the camera, the illumination, the polarizing plate, and the prism, the number of illuminations, and the presence or absence of the polarizing plate may be changed.

The example of FIG. 2 described above schematically shows a cross-sectional view of the camera 13 and the lighting unit 14 cut along a plane including the optical axis of the camera 13 and passing near the center of the polarizing plate surfaces 32a and 32b. On the other hand, the example of FIG. 4 schematically shows a cross-sectional view of the camera 13 and the lighting unit 14 cut along a plane including the optical axis of the camera 13 and passing near the center of the polarizing plate surfaces 32e and 32f. Further, FIG. 4 shows a prism 41e for guiding the light transmitted through the polarizing plate surfaces 32e and 32f to the optical system of the camera 13 in order to capture an image of the object P viewed from diagonally above with the camera 13. 41f is schematically shown.

The arrow shown by the broken line in FIG. 4 indicates an optical path of light that passes through the polarizing plate 31 and enters the optical system of the camera 13, as in the example of FIG. The arrow shown by a thick solid line in FIG. 4 indicates an optical path of light that passes through either of the polarizing plate surfaces 32e and 32f of the polarizing plate 32 and enters the optical system of the camera 13. In the example of FIG. 4, the optical path of light transmitted through the polarizing plate surface 32e is referred to as L1, and the optical path of light transmitted through the polarizing plate surface 32f is referred to as L2.

As already described with reference to FIG. 2, the light incident on the object P is the light whose first deflection component is cut by the polarizing plates 32, 33a, 33b, .... Therefore, of the light reflected by the object P, the specular reflection component mainly composed of the first polarization component is cut by the polarizing plate 32 when traveling along the optical paths L1 and L2. Therefore, the light entering the prisms 41e and 41f is a diffuse reflection component including a second polarization component. As shown in FIG. 4, the optical paths L1 and L2 are bent by the prisms 41e and 41f, and the light is guided to the optical system of the camera 13.

By using the prisms 41e and 41f, it is possible to simultaneously capture an image of the object P from a plurality of viewpoints with one camera (camera 13). In other words, an image of the object P viewed from a plurality of viewpoints can be contained in one captured image. Although not shown, prisms having the same structure as the prisms 41e and 41f may be arranged on the polarizing plate surfaces 32g and 32h shown in FIG. 3, respectively. In this case, an image (see FIG. 5) in which the object P is captured from five viewpoints (a viewpoint viewed through the polarizing plate 31 and a viewpoint viewed through each prism) can be obtained at the same time.

Here, a photographed image taken by the inspection apparatus according to the present embodiment will be described with reference to FIG. FIG. 5 is a diagram schematically showing an example of a photographed image taken by the inspection device according to the present embodiment. The photographed image shown in FIG. 5 schematically shows the photographed image obtained when the structures of FIGS. 2 to 4 are applied.

FIG. 5 schematically shows the photographed image Img. The captured image Img includes five image regions R1, ..., R5 corresponding to the five viewpoints. The image region R1 includes an image from a viewpoint viewed through the polarizing plate 31. The image areas R2, ..., R5 include images from the viewpoint viewed through the corresponding prisms. As can be seen from the example of FIG. 5, the upper surface and each side surface of the object P are contained in one captured image Img. The optical path lengths corresponding to each viewpoint are designed to be the same. Therefore, images from a plurality of viewpoints can be obtained under the same illumination conditions and the same exposure conditions.

When a photographed image of the object P such as the photographed image Img illustrated in FIG. 5 is obtained, the control device 12 analyzes the photographed image, executes a defect extraction process in the photographed image, and the object P is a non-defective product. Determine if it is present or defective. Hereinafter, the control device 12 will be further described.

(Hardware of control device)
First, with reference to FIG. 6, the hardware capable of realizing the function of the control device according to the present embodiment will be described. FIG. 6 is a block diagram showing an example of hardware capable of realizing the function of the control device according to the present embodiment.

The function of the control device 12 can be realized by using, for example, the hardware resources shown in FIG. That is, the function of the control device 12 can be realized by controlling the hardware shown in FIG. 6 using a computer program.

The hardware illustrated in FIG. 6 includes a processor 12a, a memory 12b, a display IF (Interface) 12c, a communication IF 12d, and a connection IF 12e.

The processor 12a may be a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), a GPU (Graphic Processing Unit), or the like. The memory 12b may be a ROM (Read Only Memory), a RAM (Random Access Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like.

The display IF12c is an interface for connecting a display device such as an LCD or ELD (display device 11 in the example of FIG. 1). For example, the display IF 12c controls the display by the processor 12a or the GPU mounted on the display IF 12c. The communication IF12d is an interface for connecting to a wired and / or wireless network. The communication IF12d can be connected to a wired LAN (Local Area Network), a wireless LAN, an optical communication network, or the like.

The connection IF12e is an interface for connecting various devices. The connection IF12e may be a USB (Universal Serial Bus) port, an IEEE1394 port, a SCSI (Small Computer System Interface), or the like. An input interface such as a keyboard, mouse, touch panel, and touch pad, an audio device such as a speaker, a printer, or the like can be connected to the connection IF12e. Further, a portable recording medium 12f such as a magnetic recording medium, an optical disk, a magneto-optical disk, or a semiconductor memory can be connected to the connection IF12e.

The processor 12a described above can read the program stored in the recording medium 12f and store it in the memory 12b, and control the operation of the control device 12 according to the program read from the memory 12b. The program that controls the operation of the control device 12 may be stored in the memory 12b in advance, or may be downloaded from the network via the communication IF 12d. Note that some of the elements illustrated in FIG. 6 may be omitted depending on the embodiment. Further, a plurality of computers including the same elements as the hardware illustrated in FIG. 6 may be used.

(Function of control device)
Next, the function of the control device according to the present embodiment will be described with reference to FIG. 7. FIG. 7 is a block diagram for explaining the function of the control device according to the present embodiment.

As shown in FIG. 7, the control device 12 includes a storage unit 121, a lighting control unit 122, an imaging control unit 123, a defect detection unit 124, and a lighting optimization unit 125. The function of the storage unit 121 can be realized by using the memory 12b described above. The functions of the illumination control unit 122, the image pickup control unit 123, the defect detection unit 124, and the illumination optimization unit 125 can be realized mainly by using the processor 12a described above.

Information and data such as the defect extraction filter 121a, the illumination parameter 121b, and the image data 121c can be stored in the storage unit 121.

The defect extraction filter 121a is an image filter applied to a captured image of the object P and for generating an image suitable for defect detection. For example, when the object P is a tablet, the defect extraction filter 121a is for making it difficult for the original surface shape of the tablet (secant line, engraving, etc.) to be emphasized and for scratches and stains to be emphasized in the image after application. It may be an image filter. As a result, scratches and stains can be emphasized relative to the surface shape.

The defect extraction filter 121a is designed in consideration of various factors such as the type, shape, material, and size of the object P. For example, the defect extraction filter 121a may be configured by combining an edge-related filter (edge enhancement filter / edge extraction filter), an averaging filter, and the like. The edge enhancement filter is a filter designed so that the total of the filter coefficients is 1, and the edge extraction filter is a filter designed so that the total of the filter coefficients is 0. The defect extraction filter used in the actual inspection may be different from the defect extraction filter applied to the lighting optimization method described later. For example, when a non-linear filter including either a shrink filter, a median filter, or a binarization filter or a combination thereof is applied to the defect extraction filter used in the actual inspection, the non-linearity is applied when the illumination optimization method is performed. A convolution filter that extracts features similar to the filter may be used.

The edge-related filter may consist of a differential filter. The differential filter includes a first-order differential filter such as a Previt filter and a Sobel filter, and a second-order differential filter such as a Laplacian filter. The Previt filter and the Sobel filter execute edge processing (edge enhancement / edge extraction) in the row direction and the column direction, respectively, combine the results of both, and output the result of the edge processing. When it is desired to execute edge processing only in the row direction or the column direction, a direction-designated edge-related filter that executes edge processing in a desired direction may be used. For example, in the case of an object P having a vertical striped pattern or linear unevenness extending in the vertical direction, a defect extraction filter 121a more suitable for defect detection may be obtained by combining with a direction-designated edge enhancement filter.

The illumination parameter 121b is a parameter for controlling the illumination unit 14. For example, the illumination parameter 121b may include an intensity parameter that specifies the intensity (brightness) of each illumination and a color parameter that specifies the color of the illumination. The intensity parameter may be a value such as voltage, current, pulse width, pulse interval, or a predefined index value of brightness (eg, a value between 0 and 255 corresponding to the brightness value). Good. The color parameter may be an index representing white, red (R), green (G), and blue (B), or may be a parameter set representing a mixing ratio of R, G, and B.

The image data 121c may include data of a captured image captured by the camera 13 and data of an image (hereinafter, convoluted image) obtained by applying the defect extraction filter 121a to the captured image. For example, the storage unit 121 may store data of a plurality of captured images obtained by photographing the same object P while changing the combination of lightings to be lit (hereinafter, lighting pattern). Further, the storage unit 121 may store the data of the convolution image obtained by applying the defect extraction filter 121a to the captured image.

The lighting control unit 122 controls the operation of each lighting included in the lighting unit 14 according to instructions by the defect detection unit 124 and the lighting optimization unit 125, which will be described later. The image pickup control unit 123 controls the operation of the camera 13 according to instructions from the defect detection unit 124 and the illumination optimization unit 125, which will be described later.

For example, the lighting control unit 122 and the imaging control unit 123 cooperate with sensors of an inspection device 10 (not shown) so that the object P is located at a reference position in the lighting unit 14 (for example, the optical axis of the camera 13 and the center of the object P). The operation of the lighting unit 14 and the camera 13 is controlled so that the object P is irradiated with light and the object P is photographed at the timing when the object P is reached. Further, the image pickup control unit 123 acquires a photographed image from the camera 13 and stores the photographed image data in the storage unit 121.

The defect detection unit 124 operates the illumination control unit 122 and the image pickup control unit 123 to photograph the object P. At this time, the defect detection unit 124 sets the lighting parameters obtained by the lighting optimization method described later in the lighting control unit 122, and operates the lighting control unit 122 and the imaging control unit 123 under the lighting conditions based on the lighting parameters. .. In addition, the defect detection unit 124 analyzes the captured image of the object P stored in the storage unit 121 by the imaging control unit 123 to determine whether the object P is non-defective or defective.

As the functions related to the above determination, the defect detection unit 124 has a preprocessing function 124a, a filter processing function 124b, and a determination processing function 124c.

The pre-processing function 124a is a function for performing pre-processing of a captured image. For example, when the captured image is corrected on the camera 13 side at the time of shooting, the defect detection unit 124 performs the reverse processing of the correction so that the black level gain is 0 and the gamma value is 1 by the preprocessing function 124a. You can do it. If the influence of the ambient light cannot be ignored, the defect detection unit 124 may execute a process for removing the ambient light component from the captured image at the time of lighting by the preprocessing function 124a. For example, when the object can be stopped at the time of inspection, when the posture of the object is constant and a photographed image containing an ambient light component can be obtained in advance, or when the ambient light component has simple characteristics and can be modeled. In certain cases, such as, it is possible to remove the ambient light component at the time of inspection. However, if these conditions are not satisfied and it is difficult to remove the ambient light component at the time of inspection, the function related to the removal of the ambient light component may be omitted.

The filter processing function 124b is a function of applying the defect extraction filter 121a to the captured image after the preprocessing to generate a convolution image. The determination processing function 124c is a function for determining a non-defective product / defective product of the object P based on the folded image. The defect detection unit 124 extracts a defective portion from the convoluted image by the determination processing function 124c, determines that the extracted defective portion does not satisfy a predetermined condition, determines that the product is defective, and otherwise determines that the product is non-defective. Is determined to be.

As an example, when an edge enhancement filter is used as the defect extraction filter 121a, the defect portion such as a scratch is emphasized in the convolution image, and the pixel value of the defect portion becomes large, while the pixel value of the other portion becomes relatively small. Become. In this case, by using a threshold value (first threshold value) for determining the pixel value of the defective portion and comparing the first threshold value with each pixel value, the defect detection unit 124 determines the pixel corresponding to the defective portion. Can be judged. Further, by using a threshold (second threshold) that determines the area of the defective portion to be determined as a defective product and comparing the second threshold with the number of pixels (area) of the defective portion, the defect detection unit 124 can be used. Good / defective products can be judged. In addition, other determination method may be applied.

The illumination optimization unit 125 calculates an illumination parameter (hereinafter referred to as an optimum parameter) suitable for determination by the defect detection unit 124 by using an optimization calculation algorithm described later. The optimum parameters may differ depending on the shape, material, and size of the object P even if the same lighting unit 14 is used. Therefore, the features of the object P transported by the transport belt 15 may be automatically detected, and when different features are detected, the illumination optimization unit 125 may automatically recalculate the optimum parameters. The optimum parameters may be recalculated according to the user operation.

As a function related to the calculation of the optimum parameter, the illumination optimization unit 125 has an image acquisition function 125a, a pre-processing function 125b, a filter processing function 125c, an attention area setting function 125d, and an optimization calculation function 125e.

The image acquisition function 125a is a function of operating the illumination control unit 122 and the image pickup control unit 123 to acquire a captured image used for calculating the optimum parameters. The pre-processing function 125b is a function of performing pre-processing on a captured image in the same manner as the pre-processing function 124a of the defect detection unit 124. The filter processing function 125c is a function of applying the defect extraction filter 121a to the captured image after the preprocessing to generate a convolution image, similarly to the filter processing function 124b of the defect detection unit 124.

The attention area setting function 125d is a function of setting a region of interest P (attention region: ROI (Region Of Interest)) when calculating the optimum parameter. The shape, size, setting position, and number of ROIs to be set can be arbitrarily set. Further, the ROI may be automatically set by the lighting optimization unit 125, or may be manually set by a user operation. When manually set, the attention area setting function 125d may provide a UI (User Interface) for assisting the ROI setting operation. Further, the ROI may be set in advance for each type of the object P.

The optimization calculation function 125e is a function of setting an objective function used for the optimization calculation, executing the optimization calculation based on the set objective function, and determining the optimum lighting conditions. Hereinafter, the configuration of the objective function and the algorithm of the optimization calculation will be described in detail.

For the optimization calculation, K (K> 1) captured images taken under different lighting conditions are used. The lighting conditions include a lighting pattern of lighting, a color of each lighting, a brightness of each lighting, and a difference in specific polarization. In the following, in order to simplify the explanation, a method of preparing K different lighting patterns and performing optimization calculation using K captured images having different lighting patterns will be described. Further, for the sake of simplicity, the description will proceed with an example of a lighting pattern in which the kth (1 ≦ k ≦ K) lighting set is turned on and all other lights are turned off.

The illumination optimization unit 125 acquires K shot images of the object P while changing the lighting pattern by the image acquisition function 125a. Hereinafter, the captured image corresponding to the k-th lighting pattern is referred to as an image I _k , and the pixel value of the pixel located in the x-th row and y-th column of the _{image I k is referred to as I k} (x, y). The pixel value I (x, y) of the pixel located in the x-row and y-th column of the photographed image I in which the object P is photographed with all K lighting sets lit is the condition of the following equation (1) (1). Hereinafter, linearity) can be satisfied.

The above linearity needs to be satisfied in each ROI of the object P. Therefore, in each lighting pattern, it is preferable that the entire region of the object P in the field of view of the camera 13 (hereinafter, the ROI settable region) is evenly illuminated. For example, if a plurality of lights arranged symmetrically with respect to the optical axis of the camera 13 are regarded as one lighting set, the ROI settable area can be evenly illuminated by the lighting pattern for lighting the lighting set.

In the examples of FIGS. 2 to 4, a lighting pattern for lighting a lighting set composed of lighting 21a, ..., 21h, a lighting pattern for lighting a lighting set composed of lighting 22a, ..., 22d, lighting 23a, ..., A lighting pattern for lighting a lighting set composed of 23f can be set. According to this setting, regardless of which lighting pattern is applied, the ROI settable area can be evenly illuminated. Then, the above linearity can be satisfied. Similarly, the above linearity can be satisfied when a dome-shaped illumination or a ring light is used.

When the above linearity is satisfied, the captured image I taken by adjusting the brightness of each illumination set so that the brightness of the _kth illumination set becomes wk times the reference value for k = 1 to K. The pixel value I'(x, y) of the pixel located in the x-th row and y-th column of'can be given by the following equation (2). The reference value of the brightness in the k-th lighting set is the brightness in the k-th lighting set when the _{image I k is taken.}

If the influence of ambient light cannot be ignored, or if the captured image is corrected on the camera 13 side at the time of shooting, in order to satisfy the above equations (1) and (2), the photographed image must be before. Processing is required. Therefore, when the captured image is corrected, the illumination optimization unit 125 performs the reverse processing of the correction so that the black level gain is 0 and the gamma value is 1 by the preprocessing function 125b. When the influence of the ambient light cannot be ignored, the illumination optimization unit 125 uses the preprocessing function 125b to obtain the disturbance light from the captured image at the time of lighting based on the difference between the captured image at the time of lighting and the image captured at the time of all extinguishing. Remove the ingredients.

The condition of the above equation (2) may not be satisfied when the _{weight value w k is large enough to cause overexposure.} Further, the condition of the above equation (2) may not be satisfied when the _{weight value w k is so small that the random noise is conspicuous.} Therefore, it is preferable that the weight value w _k is set within a range that sufficiently satisfies the condition of the above equation (2). In the following description, it is assumed that each captured image is an image after preprocessing.

The illumination optimization unit 125 applies the defect extraction filter 121a to each captured image after preprocessing by the filter processing function 125c, and generates a convolution image from each captured image. Hereinafter, the convolution image corresponding to the _{image I k} is referred to as an image F _k , and the pixel value of the pixel located in the x-th row and y-th column of the _{image F k is referred to as F k} (x, y).

Illumination optimization unit 125, the optimization calculation function 125e, acquires the pixel values F _k corresponding to each pixel in the ROI (x, y), the pixel values F _k (x, y) corresponding to the ROI of The set is used to generate the _{vector a k.} For example, when the region composed of the pixels of the 2nd row, 2nd column, 2nd row, 3rd column, 3rd row, 2nd column, and 3rd row, 3rd column of the _{image F k is set to ROI, the vector a k} is F _{It becomes a vector having k} (2,2), F _k (2,3), F _k (3,2), and F _k (3,3) as elements.

In addition, a method of converting a set of pixel values into a vector is illustrated for ease of explanation, but in terms of implementation, the set of pixel values is not converted into a vector, and the set of pixel values is used as it is. It may be transformed into a mechanism for performing a calculation equivalent to the calculation described in. Naturally, such a modification also belongs to the technical scope of the present embodiment.

Here, if the convoluted image obtained by applying the defect extraction filter 121a to the image I'is expressed as the image F', the pixel value F'(x) corresponding to the ROI of the image F'is similar to the _{above vector a k. The vector a opt} obtained by using the set of, y) can be expressed by the following equation (3) using the vector a _k and the weight value w _{k (scalar).} Further, if a matrix A in which K _{vectors a k are arranged and a vector w having K weight values w k} as elements is defined as the following equation (4a), the vector a _opt is the following equation (4b). ) Can be expressed as.

The lighting optimization unit 125 sets the objective function used for the optimization calculation by the optimization calculation function 125e. The lighting conditions corresponding to the optimum parameters shall satisfy at least the following three conditions. The first condition is that the absolute value of the element of the _{vector a opt} becomes a sufficiently small value when the defect extraction filter 121a is applied to the region of the object P which can be regarded as a non-defective product. The second condition is that the average pixel value in ROI is a constant value c (c> 0). The third condition is that the weight value w _k becomes a positive value. That is, the third condition guarantees that the illumination parameter has a non-negative value. The third condition can be omitted when a plurality of captured images are acquired while the object is stationary and the lighting conditions are optimized by using the difference between the plurality of captured images (modification example). .. However, the case where the third condition is imposed will be described below.

First, the objective function f (w) that satisfies the first condition is given by the following equation (5a). In the following equation (5a), | ... | represents the L2 norm, and the vector b represents the target value of the _{vector a opt.} For example, when the defect extraction filter 121a is an edge enhancement filter, the vector b is set to 0 vector. Further, when the defect extraction filter 121a is an identity function, the vector b is a vector in which all the elements are c in order to obtain an illumination condition in which the error between the element of _{the vector a opt and the average luminance value c is minimized.} Is set to.

The vector w that minimizes the above objective function f (w) can be a 0 vector. Therefore, the second condition is imposed as a constraint condition (equal constraint h according to the following equation (6a)). Further, the third condition (inequality constraint g _k according to the following equation (6b)) is also imposed.

Combining the above equations (5a), (5b), equations (6a) and equations (6b) results in a constrained quadratic programming problem with the vector w as a variable. The lighting optimization unit 125 solves this quadratic programming problem by the optimization calculation function 125e, and obtains the vector w as the optimum parameter.

If the brightness of each illumination has an adjustable upper limit, the constraint condition given by the following equation (6c) may be further imposed. In the following equation (6c), the upper limit value w _UL represents the value of _{w k} corresponding to the above-mentioned adjustable upper limit value. Instead of imposing the constraint condition given by the following equation (6c), the vector w obtained by the optimization calculation method may be scale-adjusted. For example, a method of scaling each element of the vector w so that the maximum value of the vector w _{is smaller than the upper limit w UL can be applied.}

By the way, the above quadratic programming problem can be solved by using an optimization calculation method such as the Lagrange undetermined multiplier method. For example, when the Lagrange undetermined multiplier method is applied to the above equations (5a), (5b), equations (6a) and (6b), the following equation (7a) is obtained. μ and λ _i are undetermined multipliers. Further, ∇f (w) and ∇h (w) included in the following formula (7a) are given by the following formulas (7b) and (7c), respectively. The optimum vector w can be obtained by solving the above equations (6a), (6b), and equations (7a) to (7c). An optimization calculation method other than the Lagrange undetermined multiplier method may be applied.

The lighting optimization unit 125 stores the vector w obtained by the above optimization calculation as an optimum parameter in the storage unit 121. The vector w may be stored in the storage unit 121 as an optimum parameter, or the intensity parameters such as the voltage, current, pulse width, and pulse interval applied to each illumination calculated based on the vector w are optimal. It may be stored as a parameter.

Further, the illumination optimization unit 125 may display a composite image based on the above equation (2) on the display device 11 by using the calculated optimum vector w. That is, the illumination optimization unit 125 may _{generate a composite image obtained} by synthesizing the image F k using the result of the optimization calculation and display it on the display device 11. The lighting optimization unit 125 may display an estimated image of the convolution image obtained when the object P is photographed under the optimum lighting conditions for confirmation.

The inspection device 10 has been described above.

[2. Lighting optimization method]
Next, the lighting optimization method according to the present embodiment will be described by describing the flow of the optimization calculation by the control device 12 described above with reference to FIG. FIG. 8 is a flow chart for explaining the lighting optimization method according to the present embodiment. Hereinafter, a method of executing each process of the flow chart shown in FIG. 8 by the control device 12 described above will be described, but some processes may be executed by using a computer outside the inspection device 10.

(S101, S103) The illumination optimization unit 125 of the control device 12 repeatedly executes the process of S102 while changing the index k indicating the illumination set from 1 to K. If the process reaches S103 when the index k is K, the process proceeds to S104.

(S102) The illumination optimization unit 125 operates the illumination control unit 122 and the image pickup control unit 123 to photograph the object P under the k-th illumination condition. For example, the k-th lighting condition may be that the lighting pattern is such that the k-th lighting set is turned on and the other lighting sets are turned off. It is assumed that the object P is a non-defective product or the ROI described later is set in the region of the object P that can be regarded as a non-defective product.

(S104) The illumination optimization unit 125 executes preprocessing for each captured image. When the captured image is corrected on the camera 13 side, the illumination optimization unit 125 performs the reverse processing of the correction so that the black level gain is 0 and the gamma value is 1. When the influence of the ambient light cannot be ignored, the illumination optimization unit 125 removes the ambient light component from the captured image at the time of lighting based on the difference between the captured image at the time of lighting and the photographed image at the time of all extinguishing.

(S105) The illumination optimization unit 125 applies the defect extraction filter 121a to each captured image to generate a convolution image. For example, the defect extraction filter 121a is an image filter for making the surface shape (secant line, engraving, etc.) of the object P inconspicuous and emphasizing scratches and stains in the image after application.

(S106) The illumination optimization unit 125 sets a region of interest (ROI). The shape, size, setting position, and number of ROIs to be set can be arbitrarily set. The ROI may be set automatically by the lighting optimization unit 125, or may be set manually by a user operation. When set manually, the lighting optimizer 125 may provide a UI to assist in the ROI setting operation.

(S107) The illumination optimization unit 125 sets an objective function used in the optimization calculation by using a set of pixel values obtained from the ROI of each convolution image. For example, the illumination optimization unit 125 _{generates a vector a k using a set of pixel values F k} (x, y) corresponding to the ROI, and uses the vector a _k and a predetermined target value (vector b) to generate a vector. An objective function f (w) with w as a variable (see the above equation (5a)) is set.

(S108) The illumination optimization unit 125 applies a predetermined optimization calculation method to calculate a set of weight values that minimizes the set objective function. For example, the lighting optimization unit 125 is given by the above equations (5a) and (5b) by applying the Lagrange undetermined multiplier method under the constraints given by the above equations (6a) to (6c). The quadratic programming problem to be solved is solved, and the optimum vector w is calculated.

(S109) The illumination optimization unit 125 generates a composite image using the calculated set of weight values, and displays the composite image for confirmation. For example, using the vector a _k and the vector w, the vector a _opt can be calculated based on the above equation (3). The vector a _opt gives each pixel value constituting the ROI of the convolution image obtained when the optimum vector w is applied as the optimum parameter. Therefore, the illumination optimization unit 125 can display an image based on the _{vector a opt on the display device 11 as a composite image for confirmation.}

When the process of S109 is completed, the series of processes shown in FIG. 8 is completed.

The lighting optimization calculation shown in FIG. 8 may be performed not only before the inspection device 10 is shipped but also by a user who actually uses the inspection device 10. For example, when the user performs the operation of executing the lighting optimization calculation, the object P prepared by the user may be transported to the reference position in the lighting unit 14 and the above-described processes S101 to S109 may be executed. .. By providing such a mechanism, the lighting conditions suitable for the inspection object actually inspected by the user are optimized, so that the inspection accuracy by the inspection device 10 is improved. In addition, since the lighting conditions can be optimized on the user side, it also contributes to the improvement of convenience.

The lighting optimization method has been described above.

[3. Modification example]
Hereinafter, a modified example according to the present embodiment will be described.

(First modification: setting of multiple ROIs)
As a first modification, a modification of the lighting optimization method that enables a plurality of ROIs to be set at various positions of the object P will be described. As illustrated in FIGS. 2 to 5, the inspection device 10 according to the present embodiment can simultaneously photograph the object P from a plurality of viewpoints. Therefore, images (corresponding to image areas R1 to R5) obtained by photographing the object P from a plurality of viewpoints at the same time and under the same lighting conditions with the object P at exactly the same position can be obtained.

In the illumination optimization method described above, for example, an ROI can be set in any of the image areas R1 to R5, and an optimum parameter that gives an optimum illumination condition to the ROI can be obtained. However, the lighting conditions optimized for the ROI of the image area R1 are often the lighting conditions that can obtain the inspection result with sufficient accuracy also in the image areas R2 to R4, but the optimum lighting conditions in the image areas R2 to R4. It may not be.

Therefore, in the first modification, for example, ROIs are set in two or more regions of the image regions R1 to R5, and the optimum parameters that give the optimum lighting conditions can be calculated in the set plurality of ROIs. To propose. Also in the first modification, a process of acquiring K shot images, performing preprocessing, and generating a convolution image using the defect extraction filter 121a (process corresponding to S101 to S105 in FIG. 8). Is the same as the lighting optimization method described above.

As described above, in the first modification, a plurality of ROIs are set. For example, consider a case where the first ROI is set in the image area R1 and the second ROI is set in the image area R2. The matrix A given by the above equation (4a) and the objective function f given by the above equation (5a) regarding the first ROI are expressed as A ₁ and f ₁ , respectively. The matrix A given by the above equation (4a) and the objective function f given by the above equation (5a) regarding the second ROI are expressed as A ₂ and f ₂ , respectively.

The illumination optimization unit 125 extracts pixel values from the first ROI and the second ROI in each convolution image to generate _{matrices A 1} and A _2. Further, the illumination optimization unit 125 uses the matrices A ₁ , A ₂ , the vector w, and the vector b of the target value, and uses the objective function f ₁ (w) corresponding to the objective function f of the above equation (5). Set f ₂ (w). Further, the illumination optimization unit 125 calculates a vector w that minimizes the _{objective function f 12} (w) given by the following equation (8a).

The constraints imposed when solving the quadratic programming problem given by the above equations (8a) and (8b) are the equality constraints h'and h given by the following equations (9) and (10). _{"And the inequality constraint g k} given by the above equation (6b). The equation constraint h', h" corresponds to the above-mentioned second condition. In the following equation (9), the first ROI is expressed as ROI1, the number of pixels of ROI1 is expressed as n1, and the average brightness value of ROI1 is expressed as c1. Similarly, in the following equation (10), the second ROI is expressed as ROI2, the number of pixels of ROI2 is expressed as n2, and the average brightness value of ROI2 is expressed as c2. The number of pixels n1 and n2 may be the same or different.

When the first ROI and the second ROI are set, the lighting optimization unit 125 solves the constrained quadratic programming problem that combines the above equations (8a) to (10b) to obtain the optimum parameters. The vector w is obtained as. In addition, a constraint condition corresponding to the above equation (6c) may be added. Further, the above quadratic programming problem can be solved by using an optimization calculation method such as the Lagrange undetermined multiplier method.

In the above example, the case where two ROIs are set is shown, but the optimum parameters can be obtained in the same manner when three or more ROIs are set. When setting N ROIs (N ≧ 2), set N objective functions in the same way as in the above example, impose the corresponding constraints, and solve the constrained quadratic programming problem. N vectors that give lighting conditions can be obtained as optimum parameters. As in this example, the method of finding the optimum parameters by combining a plurality of objective functions is sometimes called Pareto optimization.

(Second modification: variation of lighting pattern)
Next, as a second modification, variations of the lighting pattern will be described. In the above description, a lighting pattern in which one lighting set is turned on and all the other lighting sets are turned off is illustrated. This lighting pattern is an example, and the scope of application of the technique according to the present embodiment is not limited to this example. For example, an L-way (L ≧ K) lighting pattern for lighting each lighting set with an appropriate weight value is prepared, and the lighting conditions are set by using L shot images taken under the lighting of each lighting pattern. It may be optimized.

In this case, the illumination optimization unit 125 generates the _{vectors u 1} , ..., U _K corresponding to the K captured images, as in the illumination optimization method described above. Further, the illumination optimization unit 125 acquires a _{vector v k} corresponding to the illumination conditions used when the k-th captured image is captured. Each element of the vector v _k is a weight value corresponding to the illumination intensity (brightness) of each illumination set. _{The vectors u 1} , ..., U _K in the second modification are given by the following equation (11a) using the matrix A defined by the above equation (4a). However, the matrix V is given by the following equation (11b). If the matrix V is full rank, the above matrix A can be obtained by inverse matrix calculation as in the following equation (11c).

The above matrix A may be obtained by using K or more images for the purpose of noise reduction or the like and using a pseudo inverse matrix. When the above matrix A is obtained, the lighting optimization unit 125 sets the objective function as in the above equation (5a), and obtains the optimum parameters by solving the constrained quadratic programming problem by the optimization calculation method. be able to.

Incidentally, less than the rank of A ^T A is K, if the above matrix A is rank deficient, vector w to give the optimum parameter satisfies linear dependence relationship of Equation (12) below. Note that the matrix B included in the following formula (12) is a matrix of (K-Ranka ^T A) rows and K columns.

When the vector w is selected so as to satisfy the above equation (12), the value of the objective function f given by the above equation (5a) does not change. In this case, the vector w can be freely determined by satisfying the above equations (7a) and (12). For example, the dimension reduction is performed on the above equation (7a) to obtain the value of the vector w, and the vector w is selected from the obtained vector w so as to follow the above equation (12) to obtain the optimum lighting conditions. Can be sought.

(Third modification example: Countermeasures against minute misalignment)
As a third modification, a countermeasure for a case where a minute positional deviation of the object P occurs will be described. The position of the object P may be fixed when acquiring K shot images used for lighting optimization. In this case, the K shot images can be images of the objects P at the same position. However, depending on the configuration of the camera (for example, a line camera, etc.) and the environment at the time of shooting, a minute positional deviation in units of 1 to several pixels may not occur. When the positional deviation occurs, the linearity represented by the above equation (1) does not hold, and the lighting conditions may not always be optimized.

When the above-mentioned Lagrange undetermined multiplier method is applied, each element of the above equation (7b) is listed as the following equations (13a), (13b), and (14). For the sake of simplicity, it is assumed that all the elements of the vector b are constants b. Also, A ^T A (p, q) represents a p-q-th column element of A ^T A. Further, (i) is attached to the i-th element of the vector.

Here, the vector a _k a _k of no positional deviation captured image, the vector a _k of the captured image with a positional deviation is denoted as a _'k, the relationship of the following equation (15a) and formula (15b) Is assumed to hold. Using this relationship, the above equations (13a) and (13b) can be calculated based on the captured images having a positional deviation. To solve the above equation (7b) is, as long calculations A ^T A included in equation (14) above, also according to the following equation (15a) and formula (15b), the A ^T A Of the included elements, the element satisfying the condition of p = q is not affected by the positional deviation.

_{Here, the vector a pq} corresponding to the photographed image taken under the illumination condition in which the p-th illumination set and the q-th illumination set are turned on at the same time is given by the following equation (16). The sum of squares of the elements of the vector a _pq is as shown in Eq. (17) below. Applying the relationships given by the above equations (15a) and (15b) to the following equation (17), the following equation (18) can be obtained. _{That is, by using the vector a pq} of the captured image taken by turning on the two lighting sets at the same time, the above equation (14) is calculated based on the following equation (18) even if there is a slight positional deviation. By applying the calculation result to the above equation (7b), the above optimization calculation becomes possible even if a slight positional deviation occurs.

(Fourth modification: How to consider the effect on the defect area)
Next, as a fourth modification, a lighting optimization method that takes into consideration the condition for maximizing the objective function for the defect region will be described. In the lighting optimization method described so far, the ROI is set in the region (non-defective region) of the object P that can be regarded as a non-defective product, and the optimization function f given by the above equation (5a) is minimized for the set ROI. I was looking for parameters. In the fourth modification, in addition to this, a second objective function for the region (defect region) of the object P that can be regarded as a defect is set, the first objective function corresponding to the non-defective region is minimized, and the defect region. Find the optimal parameter that maximizes the second objective function corresponding to.

Also in the fourth modification, the illumination optimization unit 125 obtains the matrix A given by the above equation (4a) for the non-defective product region, and sets the objective function f (w) given by the above equation (5a). To do. Similarly, the illumination optimization unit 125 sets the above matrix A and the objective function f (w) for the defect region. Here, the matrix A for the non-defective region is expressed as A ₁ , the objective function f _{is expressed as f 1} , the matrix A for the defective region is expressed as A ₂ , and the objective function f is expressed _{as f 2.} Further, the illumination optimization unit 125 calculates a vector w that minimizes the objective function f'(w) given by the following equation (19a) (the following equation (19b)).

_{Α 1} and α ₂ included in the following equation (19a) are weighting coefficients (positive real numbers). Further, the objective functions f ₁ and f ₂ are given by the following equations (19c) and (19d). Note that the vector b ₁ of the target values in the objective function f _1, may be different from the vector b ₂ of targeted values included in the objective function f _2. Further, the weighting coefficient α ₂ is preferably set to a sufficiently small value so that the objective function f'(w) becomes a downwardly convex function.

A method for solving a problem equivalent to the above equations (19a) to (19d) will be described below.

(Step 1) _{Set ε such that the objective function f 1} (w) = ε (ε ≧ 0) has a solution. In the following, in order to simplify the notation, the objective functions f ₁ and f ₂ may be collectively referred to as the objective function f. Further, since the value of the objective function f increases as the area of the ROI increases, the problem is redefined in the form of the following equations (20) to (22). Further, the elements of the following equations (20) to (22) are developed as the following equations (23a) to (23c).

By redefining the objective function f as in the above equation (20), the value of ε is considered to represent the variance of the error, and it becomes possible to determine ε regardless of the magnitude of the ROI. When the objective function f defined in the above equation (20) is used as _{the objective function f 1} for the non-defective product region _{, the value of ε is when the objective function f 1} is minimized as shown in the following equation (24). It is set to be larger than the value of f _{1 of.}

(Step 2) Find the vector w that maximizes the _{objective function f 2 (w) with f 1} (w) ≦ ε as a constraint condition. _{When the vector w that maximizes the objective function f 2} (w) is obtained so as to satisfy the constraint condition f ₁ (w) ≤ ε, the vector w does not diverge. Further, this constraint condition indicates _{that the error in the non-defective product region expressed by the objective function f 1} is within the permissible range (ε), and it is easy to detect defects and it is difficult for false detection to occur. The optimum vector w that satisfies the condition can be derived. That is, by applying the fourth modification, there is a possibility that lighting conditions more convenient for defect detection can be obtained.

(Fifth modification: processing of print area)
Next, as a fifth modification, a printing process on the surface of the object P will be described. The technique according to the fifth modification can be applied not only to printing but also to treatment of stains and the like.

When the ROI and the print area overlap, the edge part such as printed characters may be emphasized by applying the defect extraction filter 121a, and as a result, it should not be the optimum parameter that gives the optimum lighting conditions for defect detection. There can be. For example, when the object P is a tablet, lighting conditions such that the area around the tablet becomes extremely bright may be obtained as a calculation result. As a method of avoiding the influence of such a print area, for example, a method of setting the ROI while avoiding the print area and a lighting optimization using an objective function normalized based on the brightness value of the image. There is a way to carry out.

When setting the ROI manually, the user should carefully avoid the print area and set the ROI to obtain the optimum lighting conditions. In this case, the lighting optimization unit 125 displays the captured image of the object P in the UI for setting the ROI, and automatically detects the print area of the object P and displays a frame indicating the print area. Or, the part of the print area may be displayed in a highlighted color, or the ROI may not be set in the print area.

When the ROI is automatically set on the control device 12 side, the illumination optimization unit 125 detects a print area on the surface of the object P and masks the detected print area. Then, the illumination optimization unit 125 sets the ROI in the remaining area excluding the print area and calculates the optimum illumination conditions. As a method for detecting the print area, a method of detecting an area having a high correlation between the square error of the edge and the square of the brightness value as the print area when the lighting conditions are changed can be considered. Further, a nonlinear optimization algorithm (for example, a QE (Quantifier Elimination) algorithm) may be applied to the normalized objective function to obtain an optimum solution.

Although preferred embodiments of the present disclosure have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modifications or modifications within the scope described in the scope of patent claims, which naturally belong to the technical scope of the present disclosure. For example, in the above description, a tablet is taken as an example of the object P, but a sheet-shaped inspection object or an inspection object such as a container having a complicated three-dimensional shape can also be handled by the above-mentioned inspection device 10.

10 Inspection device 11 Display device 12 Control device 13 Camera 14 Lighting unit 15 Conveyance belt 16a, 16b Rollers 21a to 21h, 22a to 22d, 23a, 23b, 23e, 23f Lighting 31, 32, 33a, 33b, 33e, 33f Plate plate 32a to 32h Plate plate surface 41f, 41e Prism 121 Storage unit 121a Defect extraction filter 121b Lighting parameter 121c Image data 122 Lighting control unit 123 Imaging control unit 124 Defect detection unit 125 Lighting optimization unit A1 to A5 Image area Img Captured image L1, L2 optical path P object

Claims (12)

It is a lighting optimization method that determines the lighting parameters of an inspection device that inspects an object based on a photographed image of the object taken by using illumination by a plurality of lighting groups.
A step of acquiring a first image of the object taken by the inspection device by applying each of the plurality of lighting conditions for the plurality of lighting groups, and
A step of applying an image filter used by the inspection device at the time of inspecting the object to generate a plurality of second images on the plurality of first images corresponding to the plurality of lighting conditions.
To control the plurality of illumination groups with a weight value that minimizes an error between the weighted linear sum of the corresponding pixel values in the plurality of second images and the target value corresponding to the region of interest of the object. Steps to calculate as lighting parameters of
A lighting optimization method in which a computer performs processing including. Further including the step of setting a plurality of areas of interest for the object.
The calculation step is to set an objective function representing the error between the weighted linear sum and the target value for each of the plurality of areas of interest, and to minimize the sum of the set objective functions. Including calculating the value,
The lighting optimization method according to claim 1. The inspection device can simultaneously capture images of the object viewed from different viewpoints.
Of the plurality of areas of interest, the first area of interest is set to a portion of the object visible from the first viewpoint, and the second area of interest is visible from a second viewpoint different from the first viewpoint. Set on the part of the object,
The lighting optimization method according to claim 2. The first area of interest is set on the upper surface of the object.
The second area of interest is set on the side surface of the object.
The lighting optimization method according to claim 3. The inspection device includes a first optical means for guiding the light reflected in the region of the object seen from the first viewpoint to the image sensor, and the light reflected in the region of the object seen from the second viewpoint. A second optical means for guiding the light generated to the image sensor is provided.
In the acquisition step, the first image including the image region corresponding to the first attention region and the image region corresponding to the second attention region is acquired.
The lighting optimization method according to claim 3 or 4. A control device that determines the lighting parameters of an inspection device that inspects an object based on captured images of the object taken by using illumination by a plurality of lighting groups.
A storage unit in which a first image of the object taken by the inspection device by applying each of the plurality of lighting conditions for the plurality of lighting groups is stored, and a storage unit.
A plurality of second images are generated by applying an image filter used by the inspection apparatus at the time of inspection of the object to the plurality of first images corresponding to the plurality of lighting conditions.
To control the plurality of illumination groups with a weight value that minimizes an error between the weighted linear sum of the corresponding pixel values in the plurality of second images and the target value corresponding to the region of interest of the object. A control device that has a control unit that calculates as the lighting parameters of. The control unit sets a plurality of areas of interest for the object, sets an objective function representing an error between the weighted linear sum and the target value for each of the plurality of areas of interest, and sets a plurality of objectives. The weight value that minimizes the sum of the functions is calculated as the lighting parameter.
The control device according to claim 6. The inspection device can simultaneously capture images of the object viewed from different viewpoints.
The control unit sets a first attention region among the plurality of attention regions to a portion of the object that can be seen from the first viewpoint, and sets the second attention region to a portion different from the first viewpoint. Set to the part of the object that can be seen from the viewpoint of 2.
The control device according to claim 7. The first area of interest is set on the upper surface of the object.
The second area of interest is set on the side surface of the object.
The control device according to claim 8. The inspection device includes a first optical means for guiding the light reflected in the region of the object seen from the first viewpoint to the image sensor, and the light reflected in the region of the object seen from the second viewpoint. A second optical means for guiding the light generated to the image sensor is provided.
The storage unit stores the first image including an image area corresponding to the first attention area and an image area corresponding to the second attention area.
The control device according to claim 8 or 9. The control device is included in the inspection device, and the control device is included in the inspection device.
The control unit controls the plurality of lighting groups based on the calculated lighting parameters and the plurality of lighting conditions to execute the inspection of the object.
The control device according to any one of claims 6 to 10. A computer is made to execute the lighting optimization method according to any one of claims 1 to 5.
program.

Images