JP2018112470A - Inspection system and inspection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To identify reference being used when an abnormality candidate area is identified from an intensity image, without depending on a paint color.SOLUTION: An inspection system of an embodiment comprises: a planar illumination unit that gives periodical time change and space change of light intensity to an object to be inspected; an image generation unit that generates an amplitude image, a phase image, and an intensity image from image data output from a time correlation camera or an imaging system performing an equivalent operation thereof; an area identification unit that identifies an abnormality candidate area where abnormality may exist in the object to be inspected, from each of the amplitude image, the phase image, and the intensity image; a feature calculation unit that calculates a plurality of features for determining the abnormality from the abnormality candidate area; and an abnormality determination unit that determines the abnormality by executing multivariate analysis using the plurality of features, and the area identification unit performs normalization processing on the intensity image and identifies the abnormality candidate area using the intensity image after the normalization processing is performed when the abnormality candidate area is identified from the intensity image.SELECTED DRAWING: Figure 1

Description

  Embodiments described herein relate generally to an inspection system and an inspection method.

  Conventionally, a time that includes not only the intensity of light but also information about temporal transition of light by irradiating the object to be inspected periodically and imaging the reflected light from the surface of the object to be inspected A technique for acquiring a correlation image has been proposed. Such a time correlation image is used, for example, to detect an abnormality of the object to be inspected.

Japanese Patent No. 5666971

  In anomaly detection using temporal correlation images, after identifying an anomaly candidate area where an anomaly may be present, calculate features from the anomaly candidate area and perform multivariate analysis using the computed features. An abnormality determination may be performed.

  Here, when the object to be inspected is a coated product, it is desired to accurately identify an area where an inner layer defect that does not appear as a change in the shape of the painted surface exists as an abnormal candidate area. However, since the light reflection performance of the paint surface changes depending on the paint color, even if it is the same inner layer defect, the ease of appearing as a change on the image (for example, a change in pixel value in the intensity image) Depending on the paint color. For this reason, conventionally, in order to accurately identify an abnormal candidate region as a region in which an inner layer defect exists from an intensity image, it is necessary to make a determination criterion as to whether it corresponds to an abnormal candidate region depending on the paint color. And complicated processing was required.

  Therefore, it is desired to make the reference used when specifying the abnormal candidate region from the intensity image the same regardless of the paint color.

  The inspection system of the embodiment is output from a planar illumination unit that gives a periodic temporal change and a spatial change of light intensity to an object to be inspected, and a time correlation camera or an imaging system that performs an equivalent operation. An image generation unit that generates an amplitude image, a phase image, and an intensity image from image data, and an abnormality candidate region that may have an abnormality in the inspected object from each of the amplitude image, the phase image, and the intensity image An abnormality determination that determines an abnormality by executing a multivariate analysis using a plurality of features, a feature calculation unit that calculates a plurality of features for determining abnormality from the region specifying unit to be identified, and a candidate region for abnormality When the abnormality candidate area is specified from the intensity image, the area specifying unit performs a normalization process on the intensity image, and uses the intensity image after the normalization process to perform the abnormality candidate area. To identify.

Drawing 1 is a figure showing the example of composition of the inspection system of an embodiment. FIG. 2 is a block diagram illustrating a configuration of the time correlation camera of the embodiment. FIG. 3 is a conceptual diagram showing frames accumulated in time series in the time correlation camera of the embodiment. FIG. 4 is a diagram illustrating an example of a fringe pattern irradiated by the illumination device of the embodiment. FIG. 5 is a diagram illustrating a first detection example of abnormality of the inspection object by the time correlation camera of the embodiment. FIG. 6 is a diagram illustrating an example of the amplitude of light that changes in accordance with the abnormality when the abnormality illustrated in FIG. FIG. 7 is a diagram illustrating a second detection example of an abnormality of an object to be inspected by the time correlation camera of the embodiment. FIG. 8 is a diagram illustrating a third example of detection of abnormality of an object to be inspected by the time correlation camera of the embodiment. FIG. 9 is a diagram illustrating an example of a fringe pattern output from the illumination control unit of the embodiment to the illumination device. FIG. 10 is a diagram illustrating an example of a wave shape representing a stripe pattern after passing through the screen of the embodiment. FIG. 11 is a diagram illustrating a schematic configuration of a coating surface in a case where an object to be inspected according to the embodiment is a product to which metallic coating or pearl coating is applied. FIG. 12 is a diagram for explaining an abnormal candidate region extracted from each of the amplitude image, the phase image, and the intensity image, and a feature calculated based on the abnormal candidate region. FIG. 13 is a flowchart showing a series of processes executed when the inspection system of the embodiment inspects an object to be inspected. FIG. 14 is a flowchart showing details of the abnormality candidate area specifying process executed by the inspection system of the embodiment. FIG. 15 is a diagram illustrating a specific example of a Laplacian filter that can be used in the embodiment. FIG. 16 is a diagram illustrating a specific example of the intensity image in the embodiment. FIG. 17 is a diagram illustrating a specific example of the intensity image after the Laplacian process is performed in the embodiment. FIG. 18 is a flowchart showing details of the feature calculation processing executed by the inspection system of the embodiment. FIG. 19 is a diagram illustrating a specific example of an amplitude image in the embodiment. FIG. 20 is a diagram illustrating a second region in a specific example of the intensity image after the Laplacian process is performed in the embodiment. FIG. 21 is a diagram illustrating a specific example of a region in which a feature relating to strength is calculated in the embodiment. FIG. 22 is a diagram illustrating a switching example of a fringe pattern output by the illumination control unit of the second modification. FIG. 23 is a diagram illustrating an example in which the illumination control unit according to the second modification irradiates the surface including the abnormality (defect) with a stripe pattern. FIG. 24 is a diagram illustrating a relationship between an abnormality (defect) and a stripe pattern on the screen when the stripe pattern is changed in the y direction in the second modification. FIG. 25 is a diagram illustrating an example of a fringe pattern output from the illumination control unit of the third modified example to the illumination device.

(Embodiment)
Hereinafter, the inspection system of the embodiment will be described. The inspection system of the embodiment has various configurations for inspecting an object to be inspected. Drawing 1 is a figure showing the example of composition of the inspection system of an embodiment. As shown in FIG. 1, the inspection system according to the embodiment includes a PC 100, a time correlation camera 110, a lighting device 120, a screen 130, and an arm 140.

  The arm 140 is used to fix the inspection object 150 and changes the position and orientation of the surface of the inspection object 150 that can be imaged by the time correlation camera 110 according to control from the PC 100.

  The illuminating device 120 is a device that irradiates light to the object 150 to be inspected, and can control the intensity of irradiated light in units of regions in accordance with a stripe pattern from the PC 100. Furthermore, the illuminating device 120 can control the intensity | strength of the light of the said area unit according to periodic time transition. In other words, the lighting device 120 can give a periodic temporal change and a spatial change of the light intensity. A specific light intensity control method will be described later.

  The screen 130 diffuses the light output from the illuminating device 120 and then irradiates the test object 150 with light in a plane. The screen 130 according to the embodiment irradiates the object 150 in a surface with the light input from the lighting device 120 and subjected to periodic time change and space change. An optical system component (not shown) such as a condensing Fresnel lens may be provided between the illumination device 120 and the screen 130.

  In addition, although embodiment demonstrates the example which comprises the planar illumination part which combines the illumination device 120 and the screen 130 and gives the periodic time change and spatial change of light intensity, the illumination part of embodiment is described. It is not limited to such a combination. In the embodiment, for example, the illumination unit may be configured by arranging LEDs on a surface or arranging a large monitor.

  FIG. 2 is a block diagram illustrating a configuration of the time correlation camera 110 according to the embodiment. The time correlation camera 110 includes an optical system 210, an image sensor 220, a data buffer 230, a control unit 240, and a reference signal output unit 250.

  The optical system 210 includes an imaging lens and the like, transmits a light beam from a subject (including the inspection object 150) outside the time correlation camera 110, and forms an optical image of the subject formed by the light beam.

  The image sensor 220 is a sensor that can output the intensity of light incident through the optical system 210 as a light intensity signal at high speed for each pixel.

  The light intensity signal of the embodiment is a signal obtained by the illumination device 120 of the inspection system irradiating a subject (including the inspected object 150) with light and the image sensor 220 receiving reflected light from the subject.

  The image sensor 220 is, for example, a sensor that can be read out at a higher speed than a conventional sensor, and has a two-dimensional planar shape in which pixels are arranged in two kinds of directions: a row direction (x direction) and a column direction (y direction). It shall be configured. Each pixel of the image sensor 220 is defined as a pixel P (1,1),..., P (i, j),..., P (X, Y) (Note that the image size of the embodiment is XX). Y.) Note that the reading speed of the image sensor 220 is not limited and may be the same as the conventional one.

  The image sensor 220 receives a light beam from the subject (including the inspected object 150) transmitted by the optical system 210 and photoelectrically converts the light intensity signal (intensity of light reflected from the subject) ( A two-dimensional planar frame composed of (imaging signals) is generated and output to the control unit 240. The image sensor 220 according to the embodiment outputs the frame for each readable unit time.

  The control unit 240 according to the embodiment includes, for example, a CPU, a ROM, a RAM, and the like, and executes a test program stored in the ROM, thereby transferring a transfer unit 241, a reading unit 242, and an intensity image superimposing unit 243. The first multiplier 244, the first correlation image superimposing unit 245, the second multiplier 246, the second correlation image superimposing unit 247, and the image output unit 248 are realized. Note that the present invention is not limited to implementation with a CPU or the like, and may be implemented with an FPGA or an ASIC.

  The transfer unit 241 stores the frames composed of the light intensity signals output from the image sensor 220 in the data buffer 230 in time series order.

  The data buffer 230 accumulates frames composed of light intensity signals output from the image sensor 220 in time series.

  FIG. 3 is a conceptual diagram illustrating frames accumulated in time series in the time correlation camera 110 according to the embodiment. As shown in FIG. 3, the data buffer 230 of the embodiment stores a plurality of light intensity signals G (1, 1, t),... At each time t (t = t0, t1, t2,..., Tn). .., G (i, j, t),..., G (X, Y, t), a plurality of frames Fk (k = 1, 2,..., N) are accumulated in chronological order. Is done. Note that one frame generated at time t is a light intensity signal G (1, 1, t),..., G (i, j, t), ..., G (X, Y, t). Composed.

  The light intensity signal (imaging signal) G (1,1, t),..., G (i, j, t),..., G (X, Y, t) of the embodiment has a frame Fk (k = Each pixel P (1, 1),..., P (i, j),.

  The frame output from the image sensor 220 includes only a light intensity signal, in other words, it can be considered as monochrome image data. In the embodiment, an example in which the image sensor 220 generates monochrome image data in consideration of resolution, sensitivity, cost, and the like will be described. However, the image sensor 220 is not limited to a monochrome image sensor. Alternatively, a color image sensor may be used.

  Returning to FIG. 2, the reading unit 242 of the embodiment receives the light intensity signals G (1,1, t),..., G (i, j, t),. , T) are read out in frame-by-frame order and output to the first multiplier 244, the second multiplier 246, and the intensity image superimposing unit 243.

  The time correlation camera 110 according to the embodiment generates image data for each output destination of the reading unit 242. In other words, the temporal phase camera 110 generates three types of image data.

  The time correlation camera 110 according to the embodiment generates intensity image data and two types of time correlation image data as three types of image data. The embodiment is not limited to generating the intensity image data and the two types of time correlation image data. When the intensity image data is not generated, one type or three or more types of time correlations are used. A case where image data is generated is also conceivable.

  As described above, the image sensor 220 according to the embodiment outputs a frame composed of a light intensity signal for each unit time. However, in order to generate normal image data, a light intensity signal corresponding to the exposure time necessary for imaging is required. Therefore, in the embodiment, the intensity image superimposing unit 243 generates intensity image data by superimposing a plurality of frames for an exposure time necessary for imaging. In addition, each pixel value (value representing the intensity of light) G (x, y) of the intensity image data can be derived from the following equation (1). The exposure time is the time difference between t0 and tn.

  Thereby, the intensity image data in which the subject (including the inspected object 150) is imaged is generated in the same manner as the conventional camera imaging. Then, the intensity image superimposing unit 243 outputs the generated intensity image data to the image output unit 248.

  The time correlation image data is image data indicating changes in light intensity according to time transition. That is, in the embodiment, for each frame in time series order, the light intensity signal included in the frame is multiplied by the reference signal indicating the time transition, and the time that is the result of multiplying the reference signal and the light intensity signal is obtained. Temporal correlation image data is generated by generating a temporal correlation value frame composed of correlation values and superimposing a plurality of temporal correlation value frames.

  By the way, in order to detect an abnormality of the inspection object 150 using the time correlation image data, it is necessary to change the light intensity signal input to the image sensor 220 in synchronization with the reference signal. For this reason, as described above, the illumination device 120 according to the embodiment is configured to irradiate the surface light through the screen 130 so as to periodically change the time and spatially move the stripes. The

In the embodiment, two types of time correlation image data are generated. The reference signal may be a signal representing a time transition, but in the embodiment, a complex sine wave e ^−jωt is used. It is assumed that the angular frequency is ω and the time is t. The angular frequency so that the complex sine wave e ^−jωt representing the reference signal correlates with one period of the above-described exposure time (in other words, the time required to generate the intensity image data and the time correlation image data). Assume that ω is set. In other words, the planar and dynamic light formed by the illumination unit 120 and the illumination unit such as the screen 130 has a first period (time period) at each position on the surface (reflection surface) of the object 150 to be inspected. And a distribution of increase or decrease in spatial irradiation intensity in a second period (spatial period) along at least one direction along the surface. When this planar light is reflected by the surface, it is complex-modulated according to the specifications of the surface (normal vector distribution, etc.). The time correlation camera 110 receives the light complex-modulated on the surface and performs quadrature detection (orthogonal demodulation) using the reference signal of the first period, thereby obtaining time correlation image data as a complex signal. By modulation / demodulation based on such complex time correlation image data, it is possible to detect features corresponding to the surface normal vector distribution.

The complex sine wave e ^−jωt can also be expressed as e ^−jωt = cos ωt−j · sin ωt. Accordingly, each pixel value C (x, y) of the time correlation image data can be derived from the following equation (2).

  In the embodiment, in the formula (2), two types of time correlation image data are divided into a pixel value C1 (x, y) representing the real part and a pixel value C2 (x, y) representing the imaginary part. Generate.

For this reason, the reference signal output unit 250 generates and outputs different reference signals for the first multiplier 244 and the second multiplier 246, respectively. Embodiment of the reference signal output section 250 outputs the first reference signal cosωt corresponding to the real part of the complex sine wave e ^-Jeiomegati the first multiplier 244, corresponding to the imaginary part of the complex sine wave e ^-Jeiomegati The second reference signal sin ωt is output to the second multiplier 246. As described above, the reference signal output unit 250 according to the embodiment outputs two types of reference signals expressed as time functions of a sine wave and a cosine wave that form a Hilbert transform pair as an example. However, the reference signal is not limited to the example described here, and may be a reference signal that changes according to a time transition, such as a time function.

Then, the first multiplier 244 ^calculates the real part cosωt of the complex sine wave e ^−jωt input from the reference signal output unit 250 for each light intensity signal of the frame in units of frames input from the reading unit 242. Multiply.

  The first correlation image superimposing unit 245 performs a process of superimposing the multiplication result of the first multiplier 244 on a pixel-by-pixel basis for a plurality of frames for an exposure time necessary for imaging. Thereby, each pixel value C1 (x, y) of the first time correlation image data is derived from the following equation (3).

Then, the second multiplier 246 multiplies the light intensity signal of the frame input from the reading unit 242 by the imaginary part sinωt of the complex sine wave e ^−jωt input from the reference signal output unit 250.

  The second correlation image superimposing unit 247 performs a process of superimposing the multiplication result of the second multiplier 246 on a pixel-by-pixel basis for a plurality of frames corresponding to the exposure time necessary for imaging. Thereby, each pixel value C2 (x, y) of the second time correlation image data is derived from the following equation (4).

  By performing the processing described above, two types of time correlation image data, in other words, time correlation image data having two degrees of freedom can be generated.

Further, the embodiment does not limit the type of reference signal. For example, in the embodiment, two types of time correlation image data of the real part and the imaginary part of the complex sine wave e ^−jωt are generated, but two types of image data based on the light amplitude and the light phase are generated. May be.

  Note that the time correlation camera 110 of the embodiment can generate a plurality of systems as time correlation image data. Thereby, for example, when light in which stripes having a plurality of widths are combined is irradiated, two types of time correlation image data of the real part and the imaginary part described above can be generated for each stripe width. For this purpose, the time correlation camera 110 includes a combination of two multipliers and two correlation image superimposing units for a plurality of systems, and the reference signal output unit 250 has an angular frequency suitable for each system. The reference signal by ω can be output.

  Then, the image output unit 248 outputs two types of time correlation image data and intensity image data to the PC 100. As a result, the PC 100 detects an abnormality of the inspected object 150 using the two types of time correlation image data and the intensity image data. For that purpose, it is necessary to irradiate the test object 150 with light.

  The illuminating device 120 of embodiment irradiates the fringe pattern which moves at high speed. FIG. 4 is a diagram illustrating an example of a fringe pattern irradiated by the illumination device 120 of the embodiment. In the example shown in FIG. 4, the stripe pattern is scrolled (moved) in the x direction. A white area is a bright area corresponding to the stripe, and a black area is an interval area (dark area) corresponding to the stripe.

  In the embodiment, the fringe pattern irradiated by the illuminating device 120 moves by one period with the exposure time when the time correlation camera 110 captures the intensity image data and the time correlation image data. Thereby, the illuminating device 120 gives the time change of the light intensity periodically by the spatial movement of the stripe pattern of the light intensity. In the embodiment, by associating the time during which the fringe pattern of FIG. 4 moves by one period with the exposure time, each pixel of the time-correlated image data has information on the light intensity signal for at least one period of the fringe pattern. Is embedded.

  As shown in FIG. 4, in the embodiment, an example in which the illumination device 120 irradiates a stripe pattern based on a rectangular wave will be described, but other than a rectangular wave may be used. In the embodiment, the illumination device 120 is irradiated through the screen 130, so that the boundary area between the light and dark of the rectangular wave can be blurred.

  In the embodiment, the fringe pattern irradiated by the illumination device 120 is represented as A (1 + cos (ωt + kx)). That is, the stripe pattern includes a plurality of stripes repeatedly (periodically). Note that the intensity of light applied to the inspection object 150 can be adjusted between 0 and 2A, and is a light phase kx. k is the wave number of the stripe. x is the direction in which the phase changes.

  The fundamental frequency component of the light intensity signal f (x, y, t) of each pixel in the frame can be expressed as the following equation (5). As shown in Expression (5), the brightness of the stripe changes in the x direction.

f (x, y, t) = A (1 + cos (ωt + kx))
= A + A / 2 {e ^{j (ωt + kx)} + e ^{−j (ωt + kx)} } (5)

  As shown in Expression (5), the intensity signal of the fringe pattern irradiated by the illumination device 120 can be considered as a complex number.

  Then, the light from the illumination device 120 is reflected and input to the image sensor 220 from the subject (including the inspection object 150).

  Therefore, the light intensity signal G (x, y, t) input to the image sensor 220 is used as the light intensity signal f (x, y, t) of each pixel of the frame when the illumination device 120 is irradiated. Can do. Therefore, when Expression (5) is substituted into Expression (1) for deriving intensity image data, Expression (6) can be derived. Note that the phase is kx.

  From Expression (6), it can be confirmed that each pixel of the intensity image data is input with a value obtained by multiplying the exposure time T by the intermediate value A of the intensity of the light output from the illumination device 120. Further, when Expression (5) is substituted into Expression (2) for deriving time correlation image data, Expression (7) can be derived. Note that AT / 2 is the amplitude and kx is the phase.

  Thereby, the time correlation image data shown by the complex number shown by Formula (7) is replaceable with the two types of time correlation image data mentioned above. That is, the time correlation image data composed of the real part and the imaginary part described above includes a phase change and an amplitude change in the light intensity change irradiated on the object 150. In other words, the PC 100 according to the embodiment can detect the phase change of the light emitted from the illumination device 120 and the light amplitude change based on the two types of time correlation image data.

  Therefore, the PC 100 according to the embodiment includes a phase image representing a phase change of light entering each pixel and an amplitude image representing the amplitude of light entering each pixel based on the time correlation image data and the intensity image data. Generate. In addition, the PC 100 generates an intensity image representing the intensity of light entering each pixel based on the intensity image data. Then, the PC 100 detects an abnormality of the inspected object 150 based on at least one of the phase image, the amplitude image, and the intensity image.

  By the way, when an abnormality based on the unevenness occurs in the surface shape of the inspection object 150, the distribution of the normal vectors on the surface of the inspection object 150 changes corresponding to the abnormality. In addition, when an abnormality that absorbs light occurs on the surface of the inspection object 150, the intensity of the reflected light changes. A change in the normal vector distribution is detected as at least one of a phase change and an amplitude change of light. Therefore, in the embodiment, using the time correlation image data, at least one of the light phase change and the amplitude change corresponding to the change in the normal vector distribution is detected. As a result, it is possible to specify a region where there is a possibility that a surface shape abnormality exists. Hereinafter, the relationship between the abnormality of the inspected object 150, the normal vector, and the light phase change or amplitude change will be described by way of example.

  FIG. 5 is a diagram illustrating a first detection example of abnormality of an object to be inspected by the time correlation camera 110 according to the embodiment. In the example shown in FIG. 5, it is assumed that the inspected object 500 has a projecting shape abnormality 501. In this situation, it can be confirmed that the normal vectors 521, 522, and 523 are in different directions in the region near the point 502 of the abnormality 501. Then, the normal vectors 521, 522, 523 are directed in different directions, so that diffusion (for example, light 511, 512, 513) occurs in the light reflected from the anomaly 501, and the image sensor 220 of the time correlation camera 110. The width 503 of the fringe pattern entering the arbitrary pixel 531 is increased.

  FIG. 6 is a diagram showing an example of the amplitude of light that changes in accordance with the abnormality 501 shown in FIG. In the example shown in FIG. 6, the amplitude of light is divided into a real part (Re) and an imaginary part (Im) and is represented on a two-dimensional plane. In FIG. 6, the light amplitudes 611, 612, and 613 corresponding to the lights 511, 512, and 513 in FIG. The light amplitudes 611, 612, and 613 cancel each other, and light having an amplitude 621 is incident on the arbitrary pixel 531 of the image sensor 220.

  Therefore, in the situation shown in FIG. 6, it can be confirmed that the amplitude is locally small in the region where the abnormality 501 of the inspection object 500 is imaged. In other words, when there is a region that is darker than the surroundings in the amplitude image showing the amplitude change, it can be assumed that there is local cancellation of the amplitude of the light in the region. It can be determined that an abnormality 501 has occurred at the position of the inspection object 500 corresponding to the region. Here, the case where the amplitude image is locally dark in the region corresponding to the protrusion-shaped abnormality 501 is illustrated, but the amplitude image is also locally dark in the region corresponding to the dent-like abnormality such as a scratch. Become.

  The inspection system of the embodiment is not limited to the one in which the inclination changes steeply like the abnormality 501 in FIG. 5, and can also detect an abnormality that changes gently. FIG. 7 is a diagram illustrating a second detection example of the abnormality of the inspected object by the time correlation camera 110 according to the embodiment. In the example shown in FIG. 7, the surface of the object to be inspected is flat (in other words, the normal line is parallel) in the normal state, but a gentle gradient 701 is generated on the object 700 to be inspected. In such a situation, the normal vectors 721, 722, and 723 on the gradient 701 also change gently. Accordingly, the light beams 711, 712, and 713 input to the image sensor 220 are also shifted little by little. In the example shown in FIG. 7, since the light amplitudes do not cancel each other due to the gentle gradient 701, the light amplitudes shown in FIGS. 5 and 6 hardly change. However, although the light originally projected from the screen 130 should enter the image sensor 220 as it is, the light projected from the screen 130 does not enter the image sensor 220 in a parallel state due to the gentle gradient 701. In addition, a phase change occurs in the light. Accordingly, by detecting the difference between the light phase change and the surroundings, an abnormality due to the gentle gradient 701 as shown in FIG. 7 can be detected.

  In addition, there may be an abnormality other than the surface shape of the inspection object (in other words, the distribution of the normal vector of the inspection object). FIG. 8 is a diagram illustrating a third example of detection of abnormality of an object to be inspected by the time correlation camera 110 according to the embodiment. In the example shown in FIG. 8, dirt 801 is attached to the object 800 to be inspected, so that the light irradiated from the illumination device 120 is absorbed or diffusely reflected, and the dirt 801 of the time correlation camera 110 is imaged. This represents an example in which the light intensity hardly changes in an arbitrary pixel region. In other words, in an arbitrary pixel area where the dirt 801 is imaged, changes in the light intensity cancel each other and cancel each other, resulting in almost direct current brightness.

  In such a case, in the pixel region where the dirt 801 is imaged, there is almost no light amplitude. Therefore, when an amplitude image is displayed, a region darker than the surroundings is generated. Therefore, it can be estimated that the dirt 801 is present at the position of the inspection object 800 corresponding to the region.

  As described above, in the embodiment, an area in which an abnormality may exist in the inspected object is detected by detecting the change in the amplitude of the light and the change in the phase of the light based on the time correlation image data. Can be specified.

  Returning to FIG. 1, the PC 100 will be described. The PC 100 controls the entire detection system. The PC 100 includes an arm control unit 101, an illumination control unit 102, and a control unit 103.

  The arm control unit 101 controls the arm 140 in order to change the surface of the object 150 to be imaged by the time correlation camera 110. In the embodiment, a plurality of surfaces to be imaged of the inspection object 150 are set in the PC 100 in advance. Then, every time the time correlation camera 110 completes imaging of the inspection object 150, the arm control unit 101 controls the arm 140 so that the surface on which the time correlation camera 110 is set can be imaged according to the setting. The inspection object 150 is moved. Note that the moving method of the arm 140 according to the embodiment is not limited to repeating the movement of the arm 140 every time imaging is completed and stopping the imaging before the imaging is started, and the arm 140 is continuously moved. Driving can also be included. The arm 140 may also be referred to as a transport unit, a moving unit, a position changing unit, a posture changing unit, or the like.

  The illumination control unit 102 outputs a fringe pattern irradiated by the illumination device 120 in order to inspect the inspected object 150. The illumination control unit 102 according to the embodiment transfers at least three or more stripe patterns to the illumination device 120 and instructs the illumination device 120 to switch and display the stripe patterns during the exposure time.

  FIG. 9 is a diagram illustrating an example of a fringe pattern output from the illumination control unit 102 to the illumination device 120. In accordance with the rectangular wave shown in FIG. 9B, the illumination control unit 102 performs control so that the stripe pattern in which the black region and the white region shown in FIG. 9A are set is output.

  The stripe interval for each stripe pattern irradiated in the embodiment is set according to the size of the abnormality (defect) to be detected, and detailed description thereof is omitted here.

  In addition, the angular frequency ω of the rectangular wave for outputting the fringe pattern is set to the same value as the angular frequency ω of the reference signal.

  As shown in FIG. 9, the fringe pattern output from the illumination control unit 102 can be shown as a rectangular wave, but the border area of the fringe pattern is blurred through the screen 130, that is, the bright area in the fringe pattern. By making the intensity change of light at the boundary between the (stripe region) and the dark region (interval region) gentle (dull), it can be approximated to a sine wave. FIG. 10 is a diagram illustrating an example of a wave shape representing a stripe pattern after passing through the screen 130. As shown in FIG. 10, the measurement accuracy can be improved by the wave shape approaching a sine wave. Further, a gray region in which the brightness changes in multiple steps may be added to the stripe, or a gradation may be given. Further, a stripe pattern including color stripes may be used.

  Returning to FIG. 1, the control unit 103 includes an image generation unit 104, a region specification unit 105, a feature calculation unit 106, and an abnormality determination unit 107. The control unit 103 generates an amplitude image, a phase image (time correlation image), and an intensity image based on data input from the time correlation camera 110, and there is a possibility that an abnormality exists based on the generated image. A certain area (hereinafter referred to as an abnormal candidate area) is identified. Then, the control unit 103 calculates a plurality of features (feature amounts) for determining an abnormality based on the specified abnormality candidate region, and performs multivariate analysis using the calculated plurality of features. The abnormality of the inspection object 150 is determined. In the embodiment, the control unit 103 uses the complex number instead of the time correlation image data (which can be referred to as complex time correlation image data) indicated by the polar complex number divided by the amplitude component and the phase component. It is assumed that two types of time correlation image data divided by the imaginary part are received from the time correlation camera 110.

  The image generation unit 104 generates an amplitude image, a phase image, and an intensity image based on data (image data such as intensity image data and time correlation image data) input from the time correlation camera 110. As described above, an amplitude image is an image that represents the amplitude of light entering each pixel, a phase image is an image that represents a phase of light entering each pixel, and an intensity image is a pixel-by-pixel. It is an image showing the intensity of light entering.

  Although the embodiment does not limit the calculation method of the amplitude image, for example, the image generation unit 104 determines from the pixel values C1 (x, y) and C2 (x, y) of the two types of time correlation image data. It is possible to derive each pixel value F (x, y) of the amplitude image using the following equation (8).

  Further, the image generation unit 104 derives each pixel value P (x, y) of the phase image from the pixel values C1 (x, y) and C2 (x, y) using the following equation (9). Is possible.

  Note that it is needless to say that the image generation unit 104 can generate an intensity image based on the intensity image data input from the time correlation camera 110. Therefore, here, a method for deriving each pixel value of the intensity image will be described. Is omitted.

  The area specifying unit 105 specifies an abnormality candidate area in which an abnormality may exist from each of the amplitude image, the phase image, and the intensity image generated by the image generation unit 104. For example, the region specifying unit 105 specifies a region including a position where the amplitude of light locally changes (in relation to the surroundings or the like) as an abnormal candidate region of the amplitude image, and the phase of the light changes locally. A region including the position is specified as an abnormal candidate region of the phase image. When specifying these abnormal candidate regions, various image processing such as Gaussian processing (smoothing processing), Laplacian processing, threshold processing, and the like can be used.

  Further, the region specifying unit 105 specifies a region including a position where the light intensity locally changes (in relation to the surroundings or the like) as an abnormal candidate region of the intensity image.

  Here, it is assumed that the object to be inspected 150 is a product to which metallic coating or pearl coating is applied. In this case, the surface of the inspection object 150 has the following configuration.

  FIG. 11 is a diagram illustrating a schematic configuration of a coating surface in a case where an object to be inspected 150 according to the embodiment is a product to which metallic coating or pearl coating is applied. As shown in FIG. 11, in the case where the object to be inspected 150 is a product to which metallic coating or pearl coating is applied, the surface of the object to be inspected 150 includes a base layer 1101, a color layer 1102, and a light diffusion layer. 1103 and an overcoat layer 1104.

  Here, in the example of FIG. 11, dust 1105 exists below the topcoat layer 1104, more specifically, between the color layer 1102 and the light diffusion layer 1103. The dust 1105 constitutes a so-called inner layer defect that does not appear as a change in the surface shape of the overcoat layer 1104.

  Since the inner layer defect does not appear as a change in the surface shape of the topcoat layer 1104, it is difficult to specify the region where the inner layer defect exists as an abnormal candidate region of the amplitude image and the phase image. However, since the inner layer defect affects the intensity of the reflected light from the object 150, it is possible to specify the region where the inner layer defect exists as an abnormal candidate region of the intensity image.

  By the way, the light reflection performance on the surface of the object 150 to be inspected changes according to the color (paint color) of the color layer 1102, so even if it is the same inner layer defect, it appears as a change in pixel value in the intensity image. The ease depends on the paint color. For this reason, conventionally, in order to accurately identify an abnormal candidate region as a region in which an inner layer defect exists from an intensity image, it is necessary to make a determination criterion as to whether it corresponds to an abnormal candidate region depending on the paint color. And complicated processing is required.

  Therefore, in the embodiment, when an abnormality candidate region is specified from the intensity image, the influence of the paint color is eliminated by applying a normalization process to the intensity image, and the intensity image after the influence of the paint color is excluded from the abnormality image. Decided to identify candidate areas. As a result, it is possible to make the reference used when specifying the abnormal candidate region from the intensity image the same regardless of the paint color, making it possible to more easily specify the abnormal candidate region from the intensity image. Become. Note that how to specifically identify the abnormality candidate region from the intensity image will be described later in detail together with the description of the processing executed by the region specifying unit 105, and thus further description is omitted here. .

  Returning to FIG. 1, the feature calculation unit 106 calculates a plurality of features (a plurality of features related to amplitude, phase, and intensity) for determining an abnormality from all abnormality candidate regions specified by the region specifying unit 105. . Then, the abnormality determination unit 107 determines the abnormality of the inspected object 150 by executing multivariate analysis using a plurality of features calculated by the feature calculation unit 106.

  Examples of the multivariate analysis method include the MT method (Mahalanobis Taguchi method). The MT method is a method in which a large amount of information is collected on one scale called Mahalanobis distance, and the quality of an inspection object is determined based on the magnitude of Mahalanobis distance. Specifically, in the MT method, a reference is created in advance based on data representing the characteristics of a non-defective sample with no quality problem, and the inspection target is handled based on the created standard and data representing the characteristics of the inspection target. The Mahalanobis distance to be calculated is calculated, and the quality of the inspection object is determined by comparing the calculated Mahalanobis distance with a predetermined threshold.

  In the embodiment, a technique other than the MT method may be used as a multivariate analysis technique.

  Here, in the embodiment, as described below, as one of a plurality of features related to the strength among the plurality of features used in the MT method, a difference value of the strength in the abnormal candidate region, more specifically, the strength. A difference value between a maximum value (MAX) and a minimum value (min) is used. This difference value may also be referred to as “Range”.

  In the embodiment, there are a plurality of features for determining abnormality. Therefore, in the embodiment, the feature calculation unit 106 also calculates other features other than the intensity difference value. For example, the feature calculation unit 106 detects pixel values (amplitude or amplitude) in an abnormal candidate area specified from an image other than the intensity image (amplitude image or phase image), or an area obtained by performing various image processing on the abnormal candidate area. The difference value between the maximum value and the minimum value of (phase) can be calculated as another feature.

  FIG. 12 is a diagram for explaining an abnormal candidate region extracted from each of the amplitude image, the phase image, and the intensity image, and a feature calculated based on the abnormal candidate region. In the example of FIG. 12, the region A is specified as an abnormal candidate region in all of the amplitude image, the phase image, and the intensity image, the region B is specified as an abnormal candidate region only in the amplitude image, and the region C is the phase image. And it is specified as an abnormal candidate region in the intensity image. In FIG. 12, for simplification, the regions with the same reference numerals (abnormality candidate regions) are illustrated as having the same outer shape, but the abnormal candidate region identified from the amplitude image and the phase The abnormality candidate area specified from the image and the abnormality candidate area specified from the intensity image may have different external shapes.

  As shown in FIG. 12, in the embodiment, the difference value (Range) between the maximum value and the minimum value of the intensity is calculated from all the areas A to C specified as the abnormality candidate areas. That is, in the embodiment, not only the abnormal candidate regions (regions A and C) specified from the intensity image but also other images (at least one of the amplitude image and the phase image) that are not specified from the intensity image are specified. The difference value of intensity is also calculated from the abnormal candidate region (region B).

  When an abnormal candidate region (region B) is specified from an image other than the intensity image, conventionally, an intensity difference value is calculated from the region in the intensity image corresponding to the region B of the amplitude image. However, for example, when the surface of an elongated portion whose longitudinal edge is curved is an object to be inspected (imaging target), the intensity image of that portion is relative to the curved edge portion regardless of whether there is an abnormality. Becomes darker. For this reason, conventionally, if an abnormal candidate region such as the region B is specified as being located at the curved edge portion, the intensity difference value in the abnormal candidate region is large regardless of whether there is an abnormality. Therefore, even if there is no abnormality, overdetection that is detected as having an abnormality may occur.

  Therefore, in the embodiment, in order to suppress the above-described overdetection caused by the difference in strength being increased regardless of whether there is an abnormality in the curved edge portion, the abnormality candidate area specified by the area specifying unit 105 Depending on whether or not is an area specified from the intensity image, the area for calculating the intensity difference value is made different.

  In brief, the feature calculation unit 106 according to the embodiment, when the abnormality candidate area is an area specified from at least the intensity image, the maximum intensity value in the area where the abnormality candidate area is subjected to the expansion process, A difference value between the minimum value of the intensity values in the abnormal candidate region that has not been subjected to the expansion process is calculated as an intensity difference value. In addition, when the abnormality candidate area is not an area specified from the intensity image and is an area specified from at least one of the amplitude image and the phase image, the feature calculation unit 106 performs an intensity image corresponding to the abnormality candidate area. The intensity difference value is calculated from the area corresponding to the logical product of the first area of the first area and the second area including the position where the gradient of the intensity value is the largest in the first area. Note that how the first area and the second area are set will be described later in detail together with the description of the process executed by the feature calculation unit 106, and thus further description is omitted here.

  In the embodiment, as described above, in order to more reliably determine an area in which an inner layer defect may exist as an abnormal candidate area, the abnormal candidate area is determined from the intensity image in which the influence of the paint color is eliminated by normalization processing. Was identified. However, for example, when a dark color is used as the paint color, even if there is an inner layer defect, the inner layer defect is not recognized by the human eye due to the influence of the dark color. It is desirable not to judge. Therefore, unlike the case of specifying an abnormality candidate region, it is desirable to consider the influence of the paint color when determining an abnormality.

  Therefore, in the embodiment, when calculating a plurality of features related to intensity among a plurality of features for determining abnormality, the intensity image that has not been subjected to normalization processing is used to determine the influence of the paint color when determining abnormality. Was added. That is, the feature calculation unit 106 according to the embodiment calculates the above-described intensity difference value without performing normalization processing on the intensity image.

  Hereinafter, processing executed in the embodiment will be described. FIG. 13 is a flowchart showing a series of processing executed when the inspection system of the embodiment inspects the inspected object 150. In the following, it is assumed that the inspected object 150 is already fixed to the arm 140 and arranged at the initial position of the inspection.

  As illustrated in FIG. 13, the PC 100 according to the embodiment first outputs a fringe pattern for inspecting the inspected object 150 to the illumination device 120 (S1301).

  The illuminating device 120 stores the fringe pattern input from the PC 100 (S1321). And the illuminating device 120 displays the stored fringe pattern so that it may change according to a time transition (S1322). Note that the condition for starting display by the illumination device 120 is not limited to storing the fringe pattern, and may be that the inspector performs a start operation on the illumination device 120, for example.

  Then, the control unit 103 of the PC 100 transmits an imaging start instruction to the time correlation camera 110 (S1302).

  Next, the time correlation camera 110 starts imaging the inspection object 150 and a region including the periphery of the inspection object 150 in accordance with the transmitted imaging start instruction (S1311). Then, the control unit 240 of the time correlation camera 110 generates intensity image data and time correlation image data (S1312). Then, the control unit 240 of the time correlation camera 110 outputs the intensity image data and the time correlation image data to the PC 100 (S1313).

  The control unit 103 of the PC 100 receives the intensity image data and the time correlation image data (S1303). Then, the image generation unit 104 generates an amplitude image, a phase image, and an intensity image from the received intensity image data and time correlation image data (S1304).

  Then, the area specifying unit 105 specifies an abnormality candidate area in which an abnormality may exist from each of the amplitude image, the phase image, and the intensity image (S1305). Details of the abnormality candidate area specifying process in S1305 will be described below.

  FIG. 14 is a flowchart illustrating details of the abnormality candidate area specifying process (the process of S1305 in FIG. 13) executed by the inspection system of the embodiment. Hereinafter, as an example, only the abnormality candidate area specifying process using the intensity image will be described, and the description of the abnormality candidate area specifying process using the amplitude image and the phase image will be omitted.

  As illustrated in FIG. 14, in the abnormality candidate region identification processing, the region identification unit 105 first performs Gaussian processing (smoothing processing) on the intensity image (S1401). Accordingly, even when a layer that causes random light diffusion, such as the light diffusion layer 1103 shown in FIG. 11, exists in the inspected object 150, the pixel value of the intensity image that appears by the diffusion of the light It is possible to generate an intensity image in which the influence of the minute change is canceled.

  Then, the region specifying unit 105 performs Laplacian processing on the intensity image that has been subjected to Gaussian processing (S1402). Laplacian processing is processing for calculating a change in the gradient of pixel values in an image having a plurality of pixels. Since the Laplacian process corresponds to the second order differentiation, a gradual (steady) change in the pixel value can be ignored. Therefore, according to the Laplacian process, for example, even when the surface of an elongated portion with a curved edge on the long side is an inspection target (imaging target), the intensity value of the curved edge portion is steep. If the value does not change (locally), the change in the intensity value can be ignored.

  FIG. 15 is a diagram illustrating a specific example of a Laplacian filter that can be used in the embodiment. The Laplacian filter shown in FIG. 15 is a so-called 8-neighbor Laplacian filter for taking the difference between the pixel value of the pixel to be processed and the pixel values of eight pixels adjacent to the periphery of the pixel to be processed. is there. More specifically, the Laplacian filter shown in FIG. 15 subtracts the sum of the pixel values of eight adjacent pixels around the pixel to be processed from a value obtained by multiplying the pixel value of the pixel to be processed by eight. This is a filter for setting the value to a new pixel value of the pixel to be processed. The new pixel value obtained by this calculation corresponds to the second-order differentiation of the pixel value before Laplacian processing.

  In the embodiment, a Laplacian filter having a filter coefficient different from the example shown in FIG. 15 may be used. Hereinafter, results obtained by the Laplacian filter will be briefly described together with specific examples of images.

  FIG. 16 is a diagram illustrating a specific example of the intensity image in the embodiment. FIG. 17 is a diagram illustrating a specific example of the intensity image after the Laplacian process is performed in the embodiment. As shown in FIGS. 16 and 17, according to the Laplacian filter, an image 1701 in which the influence of the edge portion is canceled can be obtained from the intensity image 1601 in which the influence of the curved edge portion appears. it can.

  Returning to FIG. 14, when the Laplacian process (S1402) is completed, the region specifying unit 105 performs a normalization process on the intensity image after the Laplacian process is performed (S1403). Thereby, the influence of the paint color is eliminated.

  Then, the region specifying unit 105 performs threshold processing on the intensity image after the normalization processing (S1403) is performed (S1404). For example, the region specifying unit 105 extracts a region having a pixel value greater than or equal to a predetermined value and an area greater than or equal to a predetermined value from the intensity image that has been subjected to the normalization process.

  Then, the region specifying unit 105 specifies the region extracted by the threshold processing (S1404) as an abnormal candidate region of the intensity image (S1405). As a result, a series of processes (abnormality candidate area specifying process) starting from S1401 ends, and the process proceeds to S1306 in FIG.

  Returning to FIG. 13, when the abnormality candidate area specifying process (S1305) ends, the feature calculation unit 106 calculates a plurality of features for determining abnormality from the abnormality candidate area (S1306). Details of the feature calculation process in S1306 will be described below.

  FIG. 18 is a flowchart showing details of the feature calculation process (the process of S1306 in FIG. 13) executed by the inspection system of the embodiment. In the following, as an example, only the feature calculation process for calculating the feature (intensity difference value) related to the intensity will be described, and the feature calculation process for calculating other features other than the intensity difference value will be described. Omitted.

  As shown in FIG. 18, the feature calculation unit 106 first performs Gaussian processing (smoothing processing) on the intensity image (S1801). The processing in S1801 is the same as the processing in S1401 in FIG. 14 described above.

  Then, the feature calculation unit 106 determines whether or not an abnormal candidate region (hereinafter referred to as a calculation target region) that is a target for calculating a feature is an abnormal candidate region specified from at least an intensity image (S1802). ).

  When it is determined that the calculation target region is at least an abnormal candidate region specified from the intensity image (S1802: Yes), the feature calculation unit 106 expands the abnormal candidate region a predetermined number of times (S1803). The expansion process is a process of adding an area for several pixels around the area to a certain area. Then, the feature calculation unit 106 calculates the maximum pixel value (intensity value after smoothing) in the area subjected to the dilation process and the pixel value (intensity value after smoothing) in the area before the dilation process is performed. ) Is calculated as one of the characteristics (characteristics regarding strength) for determining abnormality (S1804).

  On the other hand, when it is determined that the calculation target region is an abnormality candidate region identified from at least one of the amplitude image and the phase image (S1802: No), the feature calculation unit 106 determines that the calculation target region is the amplitude image and the phase image. It is determined whether or not the region is an abnormal candidate region identified from only one of the phase images (S1805).

  When it is determined that the calculation target region is an abnormality candidate region identified from only one of the amplitude image and the phase image (S1805: Yes), the feature calculation unit 106 includes the intensity image corresponding to the abnormality candidate region. The area is set as the first area (S1806).

  On the other hand, when it is determined that the calculation target region is an abnormality candidate region identified from both the amplitude image and the phase image (S1805: No), the feature calculation unit 106 includes the abnormality candidate region identified from the amplitude image, A region corresponding to the logical sum of the abnormality candidate region identified from the phase image is set as the first region (S1807).

  When the setting of the first area in S1806 or S1807 ends, the feature calculation unit 106 performs Laplacian processing on the amplitude image (S1808). Since the Laplacian process has been described above, the description thereof is omitted here.

  Then, the feature calculation unit 106 identifies a pixel having the largest pixel value in the first region in the intensity image after the Laplacian process (S1809). Then, the feature calculation unit 106 performs an expansion process for adding a region for several pixels around the pixel to the pixel specified in S1809 (S1810).

  Then, the feature calculation unit 106 sets the second region based on the region expanded in S1810 (S1811). For example, the feature calculation unit 106 can set the area itself expanded in S1810 as the second area. However, if the pixel specified in S1809 is located at the end of the first area, the area expanded in S1810 will protrude from the first area. Therefore, in this case, the feature calculation unit 106 does not set the area expanded in S1810 itself as the second area, but sets the area where the area expanded in S1810 and the first area overlap as the second area. Set as.

  Then, the feature calculation unit 106 specifies an area corresponding to the logical product of the first area specified in S1806 or S1807 described above and the second area specified in S1811, and the intensity difference is determined from the area. The value (difference value between the maximum value and the minimum value of the intensity value after smoothing) is calculated (S1812).

  Here, referring to FIG. 16 and FIGS. 19 to 21, processing until the first region and the second region are set and the amplitude attenuation amount is calculated based on the first region and the second region. The flow will be described in more detail with specific examples of images.

  As described above, FIG. 16 is a diagram illustrating a specific example of the intensity image in the embodiment. On the other hand, FIG. 19 is a diagram showing a specific example of an amplitude image in the embodiment. In the following, as an example, the abnormal candidate region is not specified from the intensity image 1601 (see FIG. 16) and the phase image (not shown), and the abnormal candidate region 1902 (see FIG. 19) is determined only from the amplitude image 1901 (see FIG. 19). The case where is specified will be described.

  When the abnormal candidate region 1902 is specified only from the amplitude image 1901, as described above, the region in the intensity image 1601 corresponding to the abnormal candidate region 1902 of the amplitude image 1901 is set as the first region. When the first region is set, the intensity image 1601 is subjected to Laplacian processing, and the pixel having the largest pixel value is specified from the first region in the image after the Laplacian processing. Then, an area on which the expansion process is performed around the pixel having the largest pixel value is set as the second area.

  FIG. 20 is a diagram illustrating a second region in a specific example (see FIG. 17) of the intensity image after the Laplacian process is performed in the embodiment. An image 1701 shown in FIG. 20 (and FIG. 17) is generated by performing Laplacian processing on the intensity image 1601 shown in FIG. The area 2001 shown in FIG. 20 is subjected to expansion processing centering on the pixel having the largest pixel value in the area corresponding to the first area in the image 1701 (area corresponding to the abnormal candidate area 1902 in FIG. 19). Represents an area. Therefore, in the example of FIG. 20, the area 2001 is set as the second area.

  When the first area and the second area are set as described above, an area corresponding to the logical product of the first area and the second area is specified as an area for which a feature relating to strength is calculated. Then, the maximum value and the minimum value of the pixel value (intensity value) are extracted from the area thus identified, and the difference value between them is calculated as a feature relating to the intensity.

  In the above description, the example in which the intensity difference value is calculated from the region corresponding to the logical product of the first region and the second region has been described. However, the region corresponding to the logical product and the second region are actually Are often the same. Therefore, in the embodiment, the intensity difference value may be calculated from the second area instead of the area corresponding to the logical product. That is, in the embodiment, it is possible to calculate the intensity difference value from the area corresponding to the logical product, and it is also possible to calculate the intensity difference value from the second area.

  FIG. 21 is a diagram illustrating a specific example of a region in which a feature relating to strength is calculated in the embodiment. FIG. 21 shows the intensity image 1601 shown in FIG. 16 with the region 1902 identified as the first region in FIG. 19 and the region 2001 identified as the second region in FIG. It is. In the example of FIG. 21, the maximum value and the minimum value of the pixel values are extracted from the area 2101 corresponding to the logical product of the area 1902 specified as the first area and the area 2001 specified as the second area. The difference value between them is calculated as a feature relating to strength.

  Returning to FIG. 18, when the difference value (characteristic regarding strength) is calculated by the process of S1804 or S1812, a series of processes (characteristic calculation process) starting from S1801 is terminated, and the process proceeds to S1307 of FIG.

  Then, returning to FIG. 13, the abnormality determination unit 107 determines the abnormality of the inspected object 150 by executing multivariate analysis based on the MT method or the like using the plurality of features calculated by the feature calculation unit 106. (S1307). The result of the abnormality determination process in S1307 is output to a display device (not shown) provided in the PC 100.

  As an output method of the result of the abnormality determination process, for example, intensity image data is displayed, and an area of intensity image data corresponding to an area where an abnormality is detected based on amplitude image data and phase image data is displayed. A method of displaying the decoration so that the inspector can recognize the abnormality can be considered. In the embodiment, the determination result is not limited to visual output, and the determination result may be output by voice or the like.

  When the process of S1307 is completed, the control unit 103 determines whether or not the inspection of the object 150 is completed (S1308). When it is determined that the inspection has not ended (S1308: No), the arm control unit 101 captures the surface of the inspection object 150 to be inspected next with the time correlation camera 110 according to a predetermined setting. The movement control of the arm 140 is performed so that it can be performed (S1309). Then, after the movement control of the arm 140 is completed, the control unit 103 transmits an imaging start instruction to the time correlation camera 110 again (S1302).

  On the other hand, when it determines with the test | inspection of the to-be-inspected object 150 having been complete | finished (S1308: Yes), the control part 103 outputs an completion | finish instruction | indication with respect to the time correlation camera 110 (S1310). As a result, a series of processes executed by the PC 100 is completed.

  Then, the time correlation camera 110 determines whether or not an end instruction has been received (S1314). If an end instruction has not been received (S1314: No), the processing from S1311 is performed again. On the other hand, when an end instruction is accepted (S1314: Yes), the process ends.

  Note that the inspecting process of the illumination device 120 may be performed by an inspector, or may be ended in accordance with an instruction from another configuration.

  With the above, a series of processes executed when the inspection system of the embodiment inspects the object to be inspected is completed.

  In the above-described embodiment, the example in which the intensity image data generated using the time correlation camera 110 and the time correlation image data are generated has been described. However, the present invention is not limited to the use of the time correlation camera 110 for generating the intensity image data and the time correlation image data, and a time correlation camera that can be realized by analog processing or an operation equivalent thereto. An imaging system may be used. For example, image data generated by a normal digital still camera is output, and the information processing apparatus superimposes a reference signal using the image data generated by the digital still camera as frame image data, thereby generating time-correlated image data. The time correlation image data may be generated by using a digital camera that superimposes a reference signal on a light intensity signal in the image sensor.

  As described above, the area specifying unit 105 according to the embodiment, when specifying an abnormality candidate area from an intensity image, performs a normalization process on the intensity image and uses the intensity image that has been subjected to the normalization process. The abnormal candidate area is specified. As a result, the influence of the paint color appearing in the intensity image is eliminated by the normalization process, so that the reference used when specifying the abnormal candidate region from the intensity image can be made identical regardless of the paint color.

  Furthermore, the feature calculation unit 106 according to the embodiment calculates a feature related to strength using a strength image that has not been subjected to normalization processing when calculating a feature related to strength among a plurality of features for determining abnormality. To do. As described above, in the abnormality determination, unlike the case of specifying the abnormality candidate region, it is desirable to consider the influence of the paint color. However, according to the embodiment, the strength that is not subjected to the normalization process Since the abnormality determination is performed in consideration of the influence of the paint color based on the feature calculated from the image, that is, the intensity image including the influence of the paint color, the above demand can be realized.

  Hereinafter, some modified examples of the embodiment will be described.

(First modification)
In the above-described embodiment, the example in which the abnormal candidate region is identified based on the local change in phase, amplitude, and intensity (difference from the surroundings) has been described. However, the abnormal candidate region is identified based on the difference from the surroundings. You are not limited to doing that. For example, as a first modification, a case in which an abnormal candidate region is specified based on a difference from reference shape data (reference data) set (acquired) in advance may be considered. In this case, it is necessary to match the position and synchronization of the spatial phase modulation illumination (stripe pattern) with the situation when the reference data is set (acquired).

  In the first modification, the region specifying unit 105 stores a phase image, an amplitude image, and an intensity image obtained from the reference surface, which are stored in advance in a storage unit (not shown), and a phase image and an amplitude image of the inspected object 150. And the intensity image, and whether or not there is a difference more than a predetermined standard for any one or more of the phase, amplitude, and intensity of light between the surface of the inspected object 150 and the reference surface judge.

  Hereinafter, in the first modification, an inspection system having the same configuration as that of the embodiment is used, and the surface of a normal inspection object is used as an example of the reference surface.

  In this case, while the illumination device 120 irradiates the fringe pattern via the screen 130, the time correlation camera 110 images the surface of a normal object to be inspected, and intensity image data, time correlation image data, Is generated. Then, the PC 100 generates a phase image, an amplitude image, and an intensity image from the intensity image data and the time correlation image data generated by the time correlation camera 110, and generates the generated phase image in a storage unit (not shown) of the PC 100. The amplitude image and the intensity image are stored. Then, the time correlation camera 110 captures an image of an inspection object for which it is desired to determine whether or not there is a possibility of abnormality, and generates intensity image data and time correlation image data. Then, after the PC 100 generates the phase image, the amplitude image, and the intensity image from the intensity image data and the time correlation image data, the phase image, the amplitude image, and the intensity image of the normal inspection object stored in the storage unit in the past. And compare. At that time, the PC 100 specifies an abnormal candidate region by comparing the phase image, the amplitude image, and the intensity image of a normal inspection object with the phase image, the amplitude image, and the intensity image of the inspection object to be inspected. It is output as data representing the characteristics of As a result, the PC 100 identifies each abnormality candidate area of the phase image, the amplitude image, and the intensity image by identifying pixels (areas) whose characteristics for identifying the abnormality candidate area are equal to or greater than a predetermined reference. Can be identified.

  With the above configuration, in the first modification, it is possible to determine whether or not there is a difference from the surface of a normal object to be inspected, in other words, whether or not there is a possibility that an abnormality exists on the surface of the object to be inspected. Note that any method may be used as a method for comparing the phase image, the amplitude image, and the intensity image, and the description thereof will be omitted.

  In the first modification, an example in which data indicating characteristics for extracting an abnormal candidate region is output based on the difference from the reference surface has been described. However, it is also conceivable to identify the abnormality candidate region by combining the technique using the difference in the reference surface described in the first modification and the technique using the difference between the surroundings described in the embodiment. Note that any technique may be used as a technique for combining the technique of the first modification and the technique of the embodiment, and the description thereof is omitted here.

(Second modification)
In the above-described embodiment, an example in which an abnormality (defect) of an inspection object is detected by moving a stripe pattern in the x direction has been described. However, when an abnormality (defect) in which the normal distribution changes sharply in the y direction perpendicular to the x direction is generated in the inspection object, the fringe pattern is moved in the y direction rather than in the x direction. It may be easier to detect defects. Therefore, as a second modification, an example in which a fringe pattern moving in the x direction and a fringe pattern moving in the y direction are alternately switched will be described.

  The illumination control unit 102 of the second modified example switches the stripe pattern output to the illumination device 120 at every predetermined time interval. Thereby, the illuminating device 120 outputs the some fringe pattern extended in the different direction with respect to one test object surface.

  FIG. 22 is a diagram illustrating a switching example of a fringe pattern output by the illumination control unit 102 of the second modification. In FIG. 22A, the illumination control unit 102 transitions the stripe pattern displayed by the illumination device 120 in the x direction. Thereafter, as illustrated in FIG. 22B, the illumination control unit 102 changes the stripe pattern displayed by the illumination device 120 in the y direction.

  Then, the control unit 103 of the PC 100 performs abnormality detection based on the time correlation image data obtained from the stripe pattern irradiation of FIG. 22A, and is obtained from the stripe pattern irradiation of FIG. Abnormality detection is performed based on the time correlation image data.

  FIG. 23 is a diagram illustrating an example in which the illumination control unit 102 according to the second modification irradiates the surface including the abnormality (defect) 2301 with a stripe pattern. In the example shown in FIG. 23, the abnormality (defect) 2301 extends in the x direction. In this case, the illumination control unit 102 sets the fringe pattern to move in the y direction that intersects the x direction, in other words, the direction that intersects the longitudinal direction of the abnormality (defect) 2301. With this setting, detection accuracy can be improved.

  FIG. 24 shows the relationship between the abnormality (defect) 2301 and the fringe pattern on the screen 130 when the fringe pattern is changed in the y direction in FIG. 23, in other words, the direction orthogonal to the longitudinal direction of the abnormality (defect) 2301. FIG. As shown in FIG. 24, when an abnormality (defect) 2301 having a narrow width in the y direction and a longitudinal direction in the x direction crossing the y direction occurs, the light emitted from the illumination device 120 is In the y direction that intersects the x direction, the cancellation of the amplitude of light increases. For this reason, the PC 100 can detect the abnormality (defect) 2301 from the amplitude image data corresponding to the fringe pattern moved in the y direction.

  In the inspection system of the second modified example, when the longitudinal direction of the defect generated in the inspection object is random, the fringe pattern is displayed in a plurality of directions (for example, the x direction and the y direction crossing the x direction). Thus, the defect can be detected regardless of the shape of the defect, and the abnormality (defect) detection accuracy can be improved. Further, by projecting a fringe pattern that matches the shape of the abnormality, the abnormality detection accuracy can be improved.

(Third Modification)
Further, the second modification described above is not limited to the method of switching the fringe pattern when performing abnormality detection in the x direction and abnormality detection in the y direction. Therefore, as a third modification, an example in which the fringe pattern output from the illumination control unit 102 to the illumination device 120 is simultaneously moved in the x direction and the y direction will be described.

  FIG. 25 is a diagram illustrating an example of a fringe pattern output from the illumination control unit 102 of the third modified example to the illumination device 120. In the example shown in FIG. 25, the illumination control unit 102 moves the fringe pattern in the direction 2501.

  The stripe pattern shown in FIG. 25 includes a stripe pattern with one period 2502 in the x direction and a stripe pattern with one period 2503 in the y direction. That is, the stripe pattern shown in FIG. 25 has a plurality of stripes extending in the intersecting directions having different widths. Here, in the third modified example, it is necessary to make the width of the stripe pattern in the x direction different from the width of the stripe pattern in the y direction. Thereby, when generating the time correlation image data corresponding to the x direction and the time correlation image data corresponding to the y direction, the corresponding reference signals can be made different. In addition, since the period (frequency) of the change in the intensity of light due to the stripe pattern may be changed, the moving speed of the stripe pattern (stripe) may be changed instead of changing the width of the stripe.

  Then, the time correlation camera 110 generates time correlation image data corresponding to the stripe pattern in the x direction based on the reference signal corresponding to the stripe pattern in the x direction, and based on the reference signal corresponding to the stripe pattern in the y direction. Thus, time correlation image data corresponding to the stripe pattern in the y direction is generated. Then, the control unit 103 of the PC 100 detects abnormality based on the time correlation image data corresponding to the stripe pattern in the x direction, and then detects abnormality based on the time correlation image data corresponding to the stripe pattern in the y direction. I do. Thereby, in the 3rd modification, it becomes possible to detect regardless of the direction where a defect has occurred, and the detection accuracy of abnormality (defect) can be improved.

  The inspection program executed by the PC 100 of the above-described embodiment is a file in an installable format or an executable format, and is a computer such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk). It is recorded on a readable recording medium and provided.

  Further, the inspection program executed by the PC 100 of the above-described embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. Further, the inspection program executed on the PC 100 according to the above-described embodiment may be provided or distributed via a network such as the Internet.

  Although several embodiments and modifications of the present invention have been described, these embodiments and modifications are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments and modifications can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and equivalents thereof.

  DESCRIPTION OF SYMBOLS 100 ... PC (inspection system), 104 ... Image generation part, 105 ... Area specification part, 106 ... Feature calculation part, 107 ... Abnormality determination part, 110 ... Time correlation camera, 120 ... Illumination device (illumination part), 130 ... Screen (Lighting part).

Claims (12)

A planar illumination unit that gives periodic temporal and spatial changes in the intensity of light to the object to be inspected;
An image generation unit that generates an amplitude image, a phase image, and an intensity image from image data output from a time correlation camera or an imaging system that performs an equivalent operation;
An area specifying unit for specifying an abnormality candidate area in which an abnormality may exist in the inspection object from each of the amplitude image, the phase image, and the intensity image;
A feature calculator that calculates a plurality of features for determining the abnormality from the abnormality candidate region;
An abnormality determination unit that determines the abnormality by performing multivariate analysis using the plurality of features;
With
When specifying the abnormal candidate region from the intensity image, the region specifying unit performs a normalization process on the intensity image, and uses the intensity image after the normalization process, Identify
Inspection system. The region specifying unit, when specifying the abnormal candidate region from the intensity image, performs a Laplacian process on the intensity image, performs a normalization process on the intensity image after the Laplacian process, Using the intensity image after being processed, the abnormal candidate region is identified.
The inspection system according to claim 1. The region specifying unit, when specifying the abnormal candidate region from the intensity image, performs Gaussian processing on the intensity image, performs Laplacian processing on the intensity image after the Gaussian processing, and performs the Laplacian processing. The intensity image after being subjected to normalization processing, and using the intensity image after the normalization processing has been performed, the abnormal candidate region is identified,
The inspection system according to claim 2. The feature calculation unit calculates the feature related to the intensity using the intensity image that has not been subjected to the normalization process when calculating the feature related to the strength among the plurality of features.
The inspection system according to any one of claims 1 to 3. The feature calculation unit corresponds to the abnormal candidate area when the abnormal candidate area is not an area specified from the intensity image and is an area specified from at least one of the amplitude image and the phase image. Difference between the maximum value and the minimum value of the intensity value in the area corresponding to the logical product of the first area in the intensity image and the second area including the position where the change in the gradient of the pixel value is the largest in the first area. A value, or a difference value between the maximum value and the minimum value of the intensity values in the second region, is calculated as a feature relating to the intensity.
The inspection system according to claim 4. The feature calculation unit, when the abnormality candidate area is not an area specified from the intensity image and is an area specified from both the amplitude image and the phase image, the abnormality specified from the amplitude image An area corresponding to the logical sum of the candidate area and the abnormal candidate area specified from the phase image is set as the first area.
The inspection system according to claim 5. The feature calculation unit performs a Laplacian process on the intensity image, and performs an expansion process on the pixel having the largest pixel value in the first area of the intensity image after the Laplacian process is performed. Set as the second area,
The inspection system according to claim 5 or 6. The feature calculation unit sets a region where the region subjected to the expansion process and the first region overlap as the second region,
The inspection system according to claim 7. The feature calculation unit performs Gaussian processing on the intensity image, and sets the first region and the second region using the intensity image after the Gaussian processing is performed.
The inspection system according to any one of claims 5 to 8. When the abnormality candidate area is an area specified from the intensity image, the feature calculation unit has a maximum value of the intensity value in an area obtained by performing an expansion process on the abnormality candidate area specified from the intensity image; A difference value between the minimum value of the intensity value in the abnormality candidate region that has not been subjected to the expansion process is calculated as a feature related to the intensity,
The inspection system according to any one of claims 5 to 9. The abnormality determination unit determines the abnormality by executing a determination process using an MT (Mahalanobis Taguchi) method as the multivariate analysis.
The inspection system according to claim 1. Amplitude from image data output from a time-correlation camera or an imaging system that operates equivalently to image the object illuminated by a planar illumination unit that gives periodic temporal and spatial changes in light intensity An image generating step for generating an image, a phase image, and an intensity image;
An area specifying step for specifying an abnormal candidate area in which an abnormality may exist in the inspection object from each of the amplitude image, the phase image, and the intensity image;
A feature calculating step of calculating a plurality of features for determining the abnormality from the abnormality candidate region;
An abnormality determination step for determining the abnormality by performing multivariate analysis using the plurality of features;
With
In the region specifying step, when the abnormality candidate region is specified from the intensity image, the abnormality image is subjected to normalization processing, and the abnormality candidate region is used using the intensity image after the normalization processing is performed. Including the step of identifying
Inspection method.