JP6909378B2 - Inspection system - Google Patents

Description

Embodiments of the present invention relate to inspection systems.

Conventionally, a technique has been proposed in which an object to be inspected is irradiated with light, the reflected light from the surface of the object to be inspected is imaged as image data, and an abnormality in the object to be inspected is detected based on a change in brightness of the image data. Has been done.

At that time, a technique has been proposed in which the intensity of the light applied to the object to be inspected is periodically changed and an abnormality is detected based on the change in the brightness of the captured image data.

Japanese Unexamined Patent Publication No. 2014-2125

In this type of inspection system, even if an abnormality such as a scratch or a protrusion on the inspection surface can be detected, it may be difficult to detect an abnormality in which the shape change is relatively gradual.

Therefore, one of the problems of the present invention is, for example, to obtain an inspection system in which an abnormality in which the shape changes relatively slowly is easily detected.

The inspection system of the embodiment is provided by a surface illumination unit that gives a periodic time change of the light intensity by the spatial movement of the fringe pattern of the light intensity, and an imaging system that operates with a time-correlated camera or the like. A feature corresponding to the distribution of the normal vector of the inspection surface from the time-correlation image generator that generates the time-correlation image, which is a complex image represented by the intensity and the phase, and the surroundings. An arithmetic processing unit that calculates a feature that detects anomalies based on at least one of the difference and the difference from the reference surface, and anomaly discrimination that distinguishes the inspection surface into a normal region and an abnormal region based on the feature that detects the anomaly. The arithmetic processing unit includes a unit and a phase retrograde region detection unit that detects a phase retrograde region in which the sign of the directional differential value in the direction along the image plane of the phase of the time correlation image is different from the sign of the normal region. Including, the abnormality determination unit determines an abnormality region based on the phase retrograde region.

FIG. 1 is a diagram showing a configuration example of the inspection system of the embodiment. FIG. 2 is a block diagram showing the configuration of the time correlation camera of the embodiment. FIG. 3 is a conceptual diagram showing frames accumulated in chronological order by the time-correlated camera of the embodiment. FIG. 4 is a diagram showing an example of a striped pattern illuminated by the lighting device of the embodiment. FIG. 5 is a diagram showing a first example of detecting an abnormality of an inspected object by the time correlation camera of the embodiment. FIG. 6 is a diagram showing an example of the amplitude of light that changes according to the abnormality when the abnormality shown in FIG. 5 is present in the object to be inspected. FIG. 7 is a diagram showing a second detection example of an abnormality of the inspected object by the time correlation camera of the embodiment. FIG. 8 is a diagram showing a third detection example of an abnormality of the inspected object by the time correlation camera of the embodiment. FIG. 9 is a diagram showing an example of a fringe pattern output by the lighting control unit of the embodiment to the lighting device. FIG. 10 is a diagram showing an example of the shape of a wave representing a striped pattern after passing through the screen of the embodiment. FIG. 11 is a flowchart showing the procedure of the abnormality detection processing based on the amplitude in the abnormality detection processing unit of the embodiment. FIG. 12 is a flowchart showing a procedure of phase-based abnormality detection processing in the abnormality detection processing unit of the embodiment. FIG. 13 is a flowchart showing a procedure of abnormality detection processing based on amplitude and intensity in the abnormality detection processing unit of the embodiment. FIG. 14 is a flowchart showing a procedure of inspection processing of an inspected object in the inspection system of the embodiment. FIG. 15 is a diagram showing an example of switching the fringe pattern output by the illumination control unit of the second modification. FIG. 16 is a diagram showing an example in which the illumination control unit of the modified example 2 irradiates the surface including an abnormality (defect) with a striped pattern. FIG. 17 is a diagram showing the relationship between the abnormality (defect) and the fringe pattern on the screen when the fringe pattern is changed in the y direction. FIG. 18 is a diagram showing an example of a fringe pattern output by the lighting control unit of the modification 3 to the lighting device. FIG. 19 is a block diagram showing an example of an abnormality detection processing unit included in the inspection system of the embodiment. FIG. 20 is a side view of a sample including an abnormal region in which the shape changes relatively slowly. FIG. 21 is a diagram showing an example of a phase change in the vicinity of the abnormal region. FIG. 22 is an explanatory diagram of a phase reversal according to the inclination of the inspection surface. FIG. 23 is a diagram showing an example of a phase image of a sample obtained by the inspection system of the embodiment and including an abnormality in which the shape change is relatively gradual. FIG. 24 is a graph showing an example of a two-dimensional power spectrum in the spatial frequency region of the time-correlated image of the sample of FIG. 23. FIG. 25 is a diagram showing an example of a phase image obtained by the inspection system of the embodiment and obtained by performing an inverse Fourier transform on the frequency domain of FIG. 24.

<Basic configuration of time-correlated camera>
The inspection system of this embodiment will be described. The inspection system 1 of the first embodiment has various configurations for inspecting an inspected object. FIG. 1 is a diagram showing a configuration example of the inspection system of the present embodiment. As shown in FIG. 1, the inspection system 1 of the present embodiment includes a PC 100, a time correlation camera 110, a lighting device 120, a screen 130, and a moving mechanism 140. The time correlation camera 110 is an example of an imaging unit.

The moving mechanism 140 is used to fix the inspected object 150, and changes the position and orientation of the surface of the inspected object 150 that can be photographed by the time correlation camera 110 according to the control from the PC 100.

The lighting device 120 is a device that irradiates the object to be inspected 150 with light, and can control the intensity of the irradiated light in units of regions according to the stripe pattern from the PC 100. Further, the illuminating device 120 can control the intensity of light in the region unit according to the periodic transition of time. In other words, the illuminating device 120 can provide periodic temporal and spatial changes in light intensity. A specific light intensity control method will be described later.

The screen 130 diffuses the light output from the lighting device 120 and then irradiates the object to be inspected 150 with the light in a surface manner. The screen 130 of the present embodiment surfacely irradiates the object to be inspected 150 with light given a periodic time change and a spatial change input from the lighting device 120. An optical system component (not shown) such as a Fresnel lens for condensing light may be provided between the illumination device 120 and the screen 130.

In this embodiment, an example in which the lighting device 120 and the screen 130 are combined to form a surface irradiation unit that gives a periodic time change and a spatial change in light intensity will be described. It is not limited, and for example, LEDs may be arranged in a plane to form an illumination unit.

The time correlation camera 110 includes an optical system 210, an image sensor 220, a data buffer 230, a control unit 240, and a reference signal output unit 250. FIG. 2 is a block diagram showing the configuration of the time correlation camera 110 of the present embodiment.

The optical system 210 includes a photographing lens and the like, transmits a light flux from an external subject (including an object to be inspected) of the time-correlation camera 110, and forms an optical image of the subject formed by the light flux.

The image sensor 220 is a sensor capable of outputting the intensity of light incident through the optical system 210 as a light intensity signal at high speed for each pixel.

The light intensity signal of the present embodiment is obtained by irradiating a subject (including an object to be inspected) with light by the lighting device 120 of the inspection system and receiving the reflected light from the subject by the image sensor 220.

The image sensor 220 is, for example, a sensor that can be read out at a higher speed than that of a conventional sensor, and has a two-dimensional plane in which pixels are arranged in two directions, a row direction (x direction) and a column direction (y direction). It shall be configured. Then, each pixel of the image sensor 220 is set to pixels P (1,1), ..., P (i, j), ..., P (X, Y) (note that the image size of this embodiment is X). × Y.). The reading speed of the image sensor 220 is not limited, and may be the same as the conventional one.

The image sensor 220 receives a light beam from the subject (including the object to be inspected) transmitted by the optical system 210 and performs photoelectric conversion, thereby indicating a light intensity signal (photographing) indicating the intensity of the light reflected from the subject. A two-dimensional planar frame composed of signals) is generated and output to the control unit 240. The image sensor 220 of the present embodiment outputs the frame every readable unit time.

The control unit 240 of the present embodiment is composed of, for example, a CPU, a ROM, a RAM, and the like, and by executing an inspection program stored in the ROM, the transfer unit 241 and the reading unit 242, and the intensity image superimposing unit 243. The first multiplier 244, the first correlation image superimposition unit 245, the second multiplier 246, the second correlation image superimposition unit 247, and the image output unit 248 are realized. It should be noted that the present invention is not limited to the realization by a CPU or the like, and may be realized by an FPGA or an ASIC.

The transfer unit 241 stores the frames composed of the light intensity signals output from the image sensor 220 in the data buffer 230 in chronological order.

The data buffer 230 stores frames composed of light intensity signals output from the image sensor 220 in chronological order.

FIG. 3 is a conceptual diagram showing frames accumulated in chronological order by the time correlation camera 110 of the present embodiment. As shown in FIG. 3, the data buffer 230 of the present embodiment contains a plurality of light intensity signals G (1,1, t) for each time t (t = t0, t1, t2, ..., Tn). A plurality of frames Fk (k = 1, 2, ..., N) composed of a combination of G (i, j, t), ..., G (X, Y, t) are arranged in chronological order. Accumulate. One frame created at time t is a light intensity signal G (1,1, t), ..., G (i, j, t), ..., G (X, Y, t). It is composed.

The light intensity signal (imaging signal) G (1,1, t), ..., G (i, j, t), ..., G (X, Y, t) of the present embodiment includes the frame image Fk ( Each pixel P (1,1), ..., P (i, j), ..., P (X, Y) constituting k = 1, 2, ..., N) is associated with each other.

The frame output from the image sensor 220 is composed of only the light intensity signal, and in other words, it can be considered as monochrome image data. In this embodiment, an example in which the image sensor 220 generates monochrome image data in consideration of resolution, sensitivity, cost, etc. will be described, but the image sensor 220 is not limited to a monochrome image sensor. Instead, an image sensor for color may be used.

Returning to FIG. 2, the reading unit 242 of the present embodiment receives the light intensity signals G (1,1, t), ..., G (i, j, t), ..., G (X,) from the data buffer 230. Y, t) is read out in chronological order in frame units and output to the first multiplier 244, the second multiplier 246, and the intensity image superimposing unit 243.

The time correlation camera 110 of the present embodiment generates image data for each output destination of the reading unit 242. In other words, the time-phase camera 110 creates three types of image data.

The time-correlation camera 110 of the present embodiment generates intensity image data and two types of time-correlation image data as three types of image data. The present embodiment is not limited to generating three types of image data, and may be a case where the intensity image data is not generated or a case where one type or three or more types of time-correlated image data is generated. .. The time correlation camera 110 is an example of a time correlation image generation unit.

As described above, the image sensor 220 of the present embodiment outputs a frame composed of a light intensity signal for each unit time. However, in order to generate normal image data, a light intensity signal for the exposure time required for shooting is required. Therefore, in the present embodiment, the intensity image superimposition unit 243 superimposes a plurality of frames for the exposure time required for shooting to generate intensity image data. Each pixel value (value representing the intensity of light) G (x, y) of the intensity image data can be derived from the following equation (1). The exposure time is the time difference between t0 and tn.

As a result, intensity image data in which the subject (including the object to be inspected) is photographed is generated as in the case of photographing with a conventional camera. Then, the intensity image superimposition unit 243 outputs the generated intensity image data to the image output unit 248.

The time-correlated image data is image data showing a change in the intensity of light according to a time transition. That is, in the present embodiment, for each frame in chronological order, the light intensity signal included in the frame is multiplied by the reference signal indicating the time transition, and the reference signal, the light intensity signal, and the time that is the multiplication result are multiplied. A time correlation value frame composed of correlation values is generated, and a time correlation image data is generated by superimposing a plurality of time correlation value frames.

By the way, in order to detect an abnormality of the object to be inspected by using the time-correlated image data, it is necessary to change the light intensity signal input to the image sensor 220 in synchronization with the reference signal. For this purpose, as described above, the illuminating device 120 is determined to irradiate surface light so as to periodically give a time change and spatial movement of fringes through the screen 130.

In this embodiment, two types of time-correlated image data are generated. The reference signal may be a signal representing a time transition, but in the present embodiment, a complex sine wave e- ^jωt is used. The angular frequency is ω and the time is t. The angular frequency ω is such that the complex sine wave e ^-jωt representing the reference signal correlates with one cycle of the above-mentioned exposure time (in other words, intensity image data, time required to generate a time-correlated image). It shall be set. In other words, the planar and dynamic light formed by the illuminating unit such as the illuminating device 120 and the screen 130 has a first period (time period) at each position on the surface (reflecting surface) of the inspected object 150. In addition to giving a change in the irradiation intensity over time, it also gives an increase / decrease distribution of the spatial irradiation intensity in a second cycle (spatial cycle) along at least one direction along the surface. When this surface light is reflected on a surface, it is complex-modulated according to the specifications of the surface (distribution of normal vector, etc.). The time-correlation camera 110 receives the light complex-modulated on the surface and performs orthogonal detection (orthogonal demodulation) using the reference signal of the first period to obtain time-correlation image data as a complex signal. By modulation / demodulation based on such complex time correlation image data, features corresponding to the distribution of the surface normal vector can be detected.

The complex sine wave e- ^jωt can also be expressed as e- ^jωt = cos (ωt) -j · sin (ωt). Therefore, each pixel value C (x, y) of the time-correlated image data can be derived from the following equation (2).

In the present embodiment, in the equation (2), two types of time-correlated image data are divided into a pixel value C1 (x, y) representing a real part and a pixel value C2 (x, y) representing an imaginary part. Generate.

Therefore, the reference signal output unit 250 generates and outputs different reference signals for the first multiplier 244 and the second multiplier 246, respectively. Reference signal output section 250 of this embodiment outputs the first reference signal cosωt corresponding to the real part of the complex sine wave e ^-Jeiomegati the first multiplier 244, the imaginary part of the complex sine wave e ^-Jeiomegati The corresponding second reference signal sinωt is output to the second multiplier 246. As described above, the reference signal output unit 250 of the present embodiment describes an example of outputting two types of reference signals represented as time functions of a sine wave and a cosine wave forming a Hilbert transform pair with each other, but the reference signal is a time. Any reference signal such as a function that changes according to a time transition may be used.

Then, the first multiplier 244 outputs the real number part cosωt of ^{the complex sine wave e -jωt} input from the reference signal output unit 250 for each light intensity signal of the frame in units of frames input from the reading unit 242. Multiply.

The first correlation image superimposition unit 245 performs a process of superimposing the multiplication result of the first multiplier 244 on a pixel-by-pixel basis for a plurality of frames for the exposure time required for shooting. As a result, each pixel value C1 (x, y) of the first time-correlated image data is derived from the following equation (3).

Then, the second multiplier 246 multiplies the light intensity signal of the frame input from the reading unit 242 by the imaginary part sinωt of ^{the complex sine wave e -jωt input from the reference signal output unit 250.}

The second correlation image superimposition unit 247 performs a process of superimposing the multiplication result of the second multiplier 246 on a pixel-by-pixel basis for a plurality of frames for the exposure time required for shooting. As a result, each pixel value C2 (x, y) of the second time-correlated image data is derived from the following equation (4).

By performing the above-mentioned processing, it is possible to generate two types of time-correlated image data, in other words, time-correlated image data having two degrees of freedom.

Moreover, this embodiment does not limit the type of a reference signal. For example, in the present embodiment ^{, two types of time-correlated image data of the real part and the imaginary part of the complex sine wave e-jωt} are created, but two types of image data are generated by the amplitude of light and the phase of light. You may.

The time-correlated camera 110 of the present embodiment can create a plurality of systems as time-correlated image data. This makes it possible to create two types of time-correlated image data for each of the widths of the stripes, for example, when the light is irradiated with a combination of stripes having a plurality of types of widths. For this purpose, the time correlation camera 110 includes a combination of two multipliers and two correlation image superimposition units for a plurality of systems, and the reference signal output unit 250 has an angular frequency suitable for each system. The reference signal by ω can be output.

Then, the image output unit 248 outputs two types of time-correlated image data and intensity image data to the PC 100. As a result, the PC 100 detects the abnormality of the inspected object by using the two types of time-correlated image data and the intensity image data. For that purpose, it is necessary to irradiate the subject with light.

The lighting device 120 of the present embodiment irradiates a striped pattern that moves at high speed. FIG. 4 is a diagram showing an example of a striped pattern illuminated by the lighting device 120 of the present embodiment. In the example shown in FIG. 4, the stripe pattern is scrolled (moved) in the x direction. The white area is the bright area corresponding to the stripes, and the black area is the interval area (dark area) corresponding to the stripes.

In the present embodiment, the fringe pattern irradiated by the illuminating device 120 is moved by one cycle at the exposure time at which the time-correlation camera 110 captures the intensity image data and the time-correlation image data. As a result, the illuminating device 120 gives a periodic time change of the light intensity by the spatial movement of the fringe pattern of the light intensity. In the present embodiment, by associating the time during which the fringe pattern of FIG. 4 moves by one cycle with the exposure time, each pixel of the time-correlated image data relates to at least one cycle of light intensity signal of the fringe pattern. Information is embedded.

As shown in FIG. 4, in the present embodiment, an example in which the illuminating device 120 irradiates a fringe pattern based on a square wave will be described, but a wave other than the square wave may be used. By using a diffusing member that diffuses the illumination light, it is possible to blur the light-dark boundary region of the rectangular wave. The screen 130 is an example of a diffusion member.

In the present embodiment, the fringe pattern irradiated by the illuminating device 120 is represented as A (1 + cos (ωt + kx)). That is, the fringe pattern includes a plurality of fringes repetitively (periodically). The intensity of the light applied to the object to be inspected can be adjusted between 0 and 2A, and the phase of the light is kx. k is the wave number of the stripe. x is the direction in which the phase changes.

Then, the fundamental frequency component of the light intensity signal f (x, y, t) of each pixel of the frame can be expressed by the following equation (5). As shown by the equation (5), the brightness of the stripe changes in the x direction.

f (x, y, t) = A (1 + cos (ωt + kx))
= A + A / 2 {e ^{j (ωt + kx)} + e ^{-j (ωt + kx)} } …… (5)

As shown by the equation (5), the intensity signal of the fringe pattern irradiated by the illuminating device 120 can be considered as a complex number.

Then, the light from the lighting device 120 is reflected from the subject (including the object to be inspected) and input to the image sensor 220.

Therefore, the light intensity signal G (x, y, t) input to the image sensor 220 can be the light intensity signal f (x, y, t) of each pixel of the frame when the illumination device 120 is irradiated. Therefore, by substituting the equation (5) into the equation (1) for deriving the intensity image data, the equation (6) can be derived. The phase is kx.

From the formula (6), it can be confirmed that a value obtained by multiplying the exposure time T by the median value A of the light intensity output by the lighting device 120 is input to each pixel of the intensity image data. Further, by substituting the equation (5) into the equation (2) for deriving the time-correlated image data, the equation (7) can be derived. AT / 2 is the amplitude and kx is the phase.

Thereby, the time-correlation image data represented by the complex number represented by the equation (7) can be replaced with the above-mentioned two types of time-correlation image data. That is, the time-correlated image data composed of the real part and the imaginary part described above includes the phase change and the amplitude change in the light intensity change irradiated to the inspected object. In other words, the PC 100 of the present embodiment can detect a phase change of the light emitted from the lighting device 120 and a change in the amplitude of the light based on the two types of time-correlated image data. Therefore, the PC 100 of the present embodiment has, based on the time-correlated image data and the intensity image data, the amplitude image data representing the amplitude of the light entering each pixel and the phase image data representing the phase change of the light entering each pixel. And generate.

Further, the PC 100 detects an abnormality of the inspected object based on the generated amplitude image data and the phase image data.

By the way, when the surface shape of the object to be inspected has an abnormality due to unevenness, the distribution of the normal vector on the surface of the object to be inspected has a change corresponding to the abnormality. Further, when an abnormality such as absorbing light occurs on the surface of the object to be inspected, the intensity of the reflected light changes. A change in the distribution of the normal vector is detected as at least one of a phase change and an amplitude change of light. Therefore, in the present embodiment, at least one of the phase change and the amplitude change of the light corresponding to the change in the distribution of the normal vector is detected by using the time-correlated image data and the intensity image data. This makes it possible to detect an abnormality in the surface shape. Next, the relationship between the abnormality of the object to be inspected, the normal vector, and the phase change or amplitude change of light will be described.

FIG. 5 is a diagram showing a first detection example of an abnormality of an inspected object by the time correlation camera 110 of the first embodiment. In the example shown in FIG. 5, it is assumed that the object to be inspected 500 has a protrusion-shaped abnormality 501. In this situation, it can be confirmed that the normal vectors 521, 522, and 523 are oriented in different directions in the region near the point 502 of the abnormality 501. Then, since the normal vectors 521, 522, and 523 are oriented in different directions, the light reflected from the abnormality 501 is diffused (for example, light 511, 512, 513), and the image sensor 220 of the time correlation camera 110 is generated. The width 503 of the striped pattern that fits in any pixel 531 of the above is widened.

FIG. 6 is a diagram showing an example of the amplitude of light that changes according to the abnormality when the abnormality 501 shown in FIG. 5 is present in the inspected body 500. In the example shown in FIG. 6, the amplitude of light is divided into a real part (Re) and an imaginary part (Im) and represented on a two-dimensional plane. In FIG. 6, the amplitudes of the light corresponding to the light 511, 512, and 513 of FIG. 5 are shown as 611, 612, and 613. Then, the amplitudes 611, 612, and 613 of the light cancel each other out, and the light having the amplitude 621 is incident on the arbitrary pixel 531 of the image sensor 220.

Therefore, in the situation shown in FIG. 6, it can be confirmed that the amplitude is small in the region where the abnormality 501 of the inspected body 500 is imaged. In other words, in the amplitude image data showing the amplitude change, if there is a region that is darker than the surroundings, it can be inferred that the amplitudes of the lights cancel each other out in that region. It can be determined that the abnormality 501 has occurred at the position of the corresponding object 500 to be inspected.

The inspection system 1 of the present embodiment can detect not only a steeply changing inclination as in the abnormality 501 of FIG. 5 but also a slowly changing abnormality. FIG. 7 is a diagram showing a second detection example of an abnormality of the inspected object by the time correlation camera 110 of the first embodiment. In the example shown in FIG. 7, the surface of the inspected object is normally flat (in other words, the normals are parallel), but the inspected object 700 has a gentle gradient 701. In such a situation, the normal vectors 721, 722, 723 on the gradient 701 also change slowly. Therefore, the lights 711, 712, and 713 input to the image sensor 220 also shift little by little. In the example shown in FIG. 7, since the light amplitudes do not cancel each other out due to the gentle gradient 701, the light amplitudes as shown in FIGS. 5 and 6 hardly change. However, the light originally projected from the lighting device 120 and the screen 130 should enter parallel to the image sensor as it is, but due to the gentle gradient 701, the light projected from the lighting device 120 and the screen 130 is parallel to each other. Since it does not enter the image sensor, the light undergoes a phase change. Therefore, by detecting the difference in the phase change of light from the surroundings and the like, it is possible to detect an abnormality due to a gentle gradient 701 as shown in FIG.

In addition, abnormalities may occur in addition to the surface shape of the object to be inspected (in other words, the distribution of the normal vector of the object to be inspected). FIG. 8 is a diagram showing a third detection example of an abnormality of the inspected object by the time correlation camera 110 of the first embodiment. In the example shown in FIG. 8, since the dirt 801 is attached to the object 800 to be inspected, the light emitted from the lighting device 120 is absorbed or diffusely reflected, and the dirt 801 of the time correlation camera 110 is photographed. It shows an example in which the intensity of light hardly changes in an arbitrary pixel region. In other words, in an arbitrary pixel region in which the dirt 801 is photographed, the light intensity causes phase cancellation, the vibration component is canceled, and the brightness becomes almost DC brightness.

In such a case, since there is almost no amplitude of light in the pixel region in which the dirt 801 is photographed, when the amplitude image data is displayed, there is a region that is darker than the surroundings. Therefore, it can be estimated that there is an abnormality 801 such as dirt at the position of the inspected body 800 corresponding to the region.

As described above, in the present embodiment, it is possible to estimate that the object to be inspected has an abnormality by detecting the change in the amplitude of the light and the change in the phase of the light based on the time-correlated image data.

Returning to FIG. 1, the PC 100 will be described. The PC 100 controls the entire detection system. The PC 100 includes a movement mechanism control unit 101, a light emission control unit 102, a control unit 103, and a storage unit 109. The storage unit 109 stores data used for arithmetic processing, arithmetic processing results, and the like.

The moving mechanism control unit 101 controls the moving mechanism 140 in order to change the surface of the object to be inspected 150 to be imaged by the time correlation camera 110. The moving mechanism 140 is, for example, a robot arm. In the present embodiment, in the PC 100, a plurality of surfaces to be imaged of the object to be inspected 150 are set. Then, each time the time-correlation camera 110 finishes photographing the object to be inspected 150, the movement mechanism 140 is covered so that the movement mechanism control unit 101 can photograph the surface on which the time-correlation camera 110 is set according to the setting. The inspection body 150 is moved. The present embodiment is not limited to repeatedly moving the moving mechanism 140 each time the shooting is completed and stopping the moving mechanism 140 before the shooting starts, and the moving mechanism 140 may be continuously driven. .. The moving mechanism 140 may also be referred to as a transporting portion, a moving portion, a gripping portion, a position changing portion, a posture changing portion, or the like.

The light emission control unit 102 outputs a fringe pattern irradiated by the lighting device 120 in order to inspect the object to be inspected 150. The light emission control unit 102 of the present embodiment delivers at least three or more striped patterns to the lighting device 120, and instructs the lighting device 120 to switch and display the striped patterns during the exposure time. The light emission control unit 102 may also be referred to as a lighting control unit.

FIG. 9 is a diagram showing an example of a fringe pattern output by the light emission control unit 102 to the lighting device 120. The light emission control unit 102 controls so that the fringe pattern in which the black region and the white region shown in FIG. 9A are set is output according to the rectangular wave shown in FIG. 9B.

The interval between the stripes for each stripe pattern to be irradiated in the present embodiment is set according to the size of the abnormality (defect) to be detected, and detailed description thereof will be omitted here.

Further, the angular frequency ω of the square wave for outputting the fringe pattern is set to the same value as the angular frequency ω of the reference signal.

As shown in FIG. 9, the fringe pattern output by the light emission control unit 102 can be shown as a square wave, but the boundary region of the fringe pattern is blurred by passing through the screen 130 (diffusion member), that is, fringes. By slowing (blunting) the change in light intensity at the boundary between the bright region (striped region) and the dark region (interval region) in the pattern, it can be approximated to a sine wave. FIG. 10 is a diagram showing an example of the shape of a wave representing a striped pattern after passing through the screen 130. As shown in FIG. 10, when the shape of the wave approaches a sine wave, the measurement accuracy can be improved. In addition, a gray area in which the brightness changes in multiple stages may be added to the stripes, or a gradation may be given. Further, a fringe pattern including color fringes may be used.

Returning to FIG. 1, the control unit 103 includes an amplitude-phase image generation unit 104 and an abnormality detection processing unit 105, and is based on the intensity image data input from the time correlation camera 110 and the time correlation image data. A process is performed to calculate a feature that corresponds to the distribution of the normal vector of the inspection surface of the inspected object 150 and detects an abnormality due to a difference from the surroundings. In this embodiment, in order to perform the inspection, instead of the time correlation image data (referred to as complex time correlation image data) indicated by the complex number, two types of the complex number correlation image data are divided into a real part and an imaginary part. The time-correlated image data of the above is received from the time-correlated camera 110. The amplitude-phase image generation unit 104 (control unit 103) is an example of an arithmetic processing unit. The abnormality detection processing unit 105 is an example of an arithmetic processing unit and an abnormality determination unit.

The amplitude-phase image generation unit 104 generates amplitude image data and phase image data based on the intensity image data input from the time correlation camera 110 and the time correlation image data.

The amplitude image data is image data representing the amplitude of light entering each pixel. The phase image data is image data representing the phase of light entering each pixel.

The present embodiment does not limit the calculation method of the amplitude image data, but for example, the amplitude-phase image generation unit 104 has pixel values C1 (x, y) and C2 (x, y) of two types of time-correlation image data. From y), each pixel value F (x, y) of the amplitude image data can be derived using the equation (8).

Then, in the present embodiment, it is possible to determine whether or not there is an abnormality region based on the pixel value (amplitude) of the amplitude image data and the pixel value of the intensity image data. For example, an abnormality occurs in a region where the pixel value (AT) of the intensity image data divided by 2 and the amplitude of the amplitude image data (AT / 2 if there is no cancellation) match to some extent. I can guess that it is not. On the other hand, in the regions where they do not match, it can be inferred that the amplitudes cancel each other out. The specific method will be described later.

Similarly, the amplitude-phase image generation unit 104 uses the equation (9) from the pixel values C1 (x, y) and C2 (x, y) to obtain each pixel value P (x, y) of the phase image data. Can be derived.

The abnormality detection processing unit 105 is a feature corresponding to the distribution of the normal vector of the surface to be inspected by the amplitude image data and the phase image data generated by the amplitude-phase image generation unit 104, and is different from the surroundings. , Detects anomalous features of the subject 150 to be inspected. In this embodiment, an example using the amplitude distribution of the complex time correlation image as a feature corresponding to the distribution of the normal vector will be described. The amplitude distribution of the complex time correlation image is data showing the amplitude distribution of each pixel of the complex time correlation image, and corresponds to the amplitude image data.

Next, the abnormality detection processing based on the amplitude in the abnormality detection processing unit 105 of the present embodiment will be described. FIG. 11 is a flowchart showing the procedure of the processing in the abnormality detection processing unit 105 of the present embodiment.

First, the abnormality detection processing unit 105 uses the amplitude value (representing pixel value) of light stored in each pixel of the amplitude image data as a reference (for example, the center) to mean the average amplitude in the N × N region. The value is subtracted (step S1101) to generate the average difference image data of the amplitude. The average difference image data of the amplitude corresponds to the gradient of the amplitude. It is assumed that an appropriate value of the integer N is set according to the embodiment.

Next, the abnormality detection processing unit 105 performs mask processing using a predetermined amplitude threshold value on the average difference image data of the amplitude generated by the subtraction (step S1102).

Further, the abnormality detection processing unit 105 calculates the standard deviation for each pixel in the mask area of the average difference image data (step S1103). In this embodiment, the method based on the standard deviation will be described, but the method is not limited to the case where the standard deviation is used, and for example, an average value or the like may be used.

Then, the abnormality detection processing unit 105 detects a pixel whose amplitude pixel value obtained by subtracting the average is smaller than −4.5σ (σ: standard deviation) as a region having an abnormality (defect) (step S1104).

According to the processing procedure described above, the abnormality of the inspected object can be detected from the amplitude value of each pixel (in other words, the distribution of the amplitude). However, the present embodiment is not limited to detecting anomalies from the amplitude distribution of the complex time correlation image. As a feature corresponding to the distribution of the normal vector of the surface to be inspected, the gradient of the phase distribution may be used. Therefore, an example using the gradient of the phase distribution will be described next.

Next, the phase-based abnormality detection processing in the abnormality detection processing unit 105 of the present embodiment will be described. FIG. 12 is a flowchart showing the procedure of the processing in the abnormality detection processing unit 105 of the present embodiment.

First, the abnormality detection processing unit 105 subtracts the average phase value in the N × N region from the phase value (pixel value representing) of the light for each pixel of the phase image data with the pixel as a reference (for example, the center). (Step S1201), phase average difference image data is generated. The average difference image data of the phase corresponds to the gradient of the phase.

Next, the abnormality detection processing unit 105 compares the size (absolute value) of the average difference image data of the phase generated by the subtraction with the threshold value, and determines the pixels in which the size of the average difference image data is equal to or larger than the threshold value. , It is detected as a pixel having an abnormality (defect) (step S1202).

Based on the detection result of S1202, the abnormality detection processing unit 105 can determine the unevenness based on the positive / negative of the average difference image data, that is, the magnitude relationship between the phase value of the pixels and the average phase value (step S1203). Which of the pixel phase value and the average phase value is larger to be convex depends on the setting of each part, but if the magnitude relationship is different, the unevenness is different.

Anomalies can be detected from the gradient of the phase distribution obtained by another method. For example, in the abnormality detection processing unit 105, as another method, the magnitude of the difference between the average vector in the N × N region of the normalized time-correlated image data and the vector of each normalized pixel is larger than the threshold value. If it is also large, it can be detected as a pixel having an abnormality (defect). Further, the abnormality of the inspected object may be detected based on the information corresponding to the phase distribution, not limited to the gradient of the phase distribution.

Next, the abnormality detection processing based on the amplitude and the intensity in the abnormality detection processing unit 105 of the present embodiment will be described. FIG. 13 is a flowchart showing the procedure of the processing in the abnormality detection processing unit 105 of the present embodiment.

First, the abnormality detection processing unit 105 uses the following equation (100) for each pixel from the time-correlation image data and the intensity image data, and the amplitude (pixel value representing) C (x, y) (formula (formula). The ratio R (x, y) of (see 7)) and the intensity (pixel value representing) G (x, y) (see equation (6)) is calculated (step S1301).

R (x, y) = C (x, y) / G (x, y) ... (100)

Next, the abnormality detection processing unit 105 compares the ratio R (x, y) with the threshold value, and selects pixels having a ratio R (x, y) value equal to or less than the corresponding threshold value as pixels having an abnormality (defect). (Step S1302). Further, the abnormality detection processing unit 105 compares the ratio R (x, y) with the threshold value, and determines that the pixel whose value of the ratio R (x, y) is equal to or larger than another corresponding threshold value is uneven (dirt, etc.). It is detected as a certain pixel (step S1303). When the amplitude cancellation (attenuation) becomes remarkable due to an abnormality in the distribution of the normal vector, the amplitude is much lower than the intensity. On the other hand, although the distribution of the normal vector is not so abnormal, when the light absorption becomes remarkable due to the dirt on the surface of the inspected object 150 or the like, the intensity is much lower than the amplitude. Therefore, the abnormality detection processing unit 105 can detect the abnormality type in steps S1302 and S1303.

Next, the inspection process of the object to be inspected in the inspection system of the present embodiment will be described. FIG. 14 is a flowchart showing the procedure of the above-described processing in the inspection system of the present embodiment. It is assumed that the inspected body 150 is already fixed to the moving mechanism 140 and is arranged at the initial position of the inspection.

The PC 100 of the present embodiment outputs a striped pattern for inspecting the object to be inspected to the lighting device 120 (step S1401).

The lighting device 120 stores the striped pattern input from the PC 100 (step S1421). Then, the lighting device 120 displays the stored fringe pattern so as to change according to the time transition (step S1422). The condition for the lighting device 120 to start the display is not limited to the time when the striped pattern is stored, and may be, for example, when the inspector performs a start operation on the lighting device 120.

Then, the control unit 103 of the PC 100 transmits a shooting start instruction to the time correlation camera 110 (step S1402).

Next, the time-correlated camera 110 starts imaging the region including the inspected object 150 according to the transmitted imaging instruction (step S1411). Next, the control unit 240 of the time correlation camera 110 generates intensity image data and time correlation image data (step S1412). Then, the control unit 240 of the time correlation camera 110 outputs the intensity image data and the time correlation image data to the PC 100 (step S1413).

The control unit 103 of the PC 100 receives the intensity image data and the time-correlated image data (step S1403). Then, the amplitude-phase image generation unit 104 generates the amplitude image data and the phase image data from the received intensity image data and the time correlation image data (step S1404).

Then, the abnormality detection processing unit 105 performs abnormality detection control of the inspected object based on the amplitude image data and the phase image data (step S1405). Then, the abnormality detection processing unit 105 outputs the abnormality detection result to a display device (not shown) included in the PC 100 (step S1406).

As an output example of the abnormality detection result, the inspector displays an abnormality in the intensity image data area corresponding to the area in which the abnormality is detected based on the amplitude image data and the phase image data while displaying the intensity image data. It is conceivable to display decorations so that they can be recognized. Further, the output is not limited to the output based on the visual sense, and it may be output that an abnormality is detected by voice or the like.

The control unit 103 determines whether or not the inspection of the object to be inspected has been completed (step S1407). When it is determined that the inspection has not been completed (step S1407: No), the moving mechanism control unit 101 photographs the surface of the object to be inspected next with the time correlation camera 110 according to a predetermined setting. The movement of the arm is controlled so as to be possible (step S1408). After the movement control of the arm is completed, the control unit 103 again transmits a shooting start instruction to the time correlation camera 110 (step S1402).

On the other hand, when the control unit 103 determines that the inspection of the object to be inspected has been completed (step S1407: Yes), the control unit 103 outputs an end instruction to the time correlation camera 110 (step S1409), and ends the process.

Then, the time correlation camera 110 determines whether or not the end instruction has been accepted (step S1414). If the end instruction is not accepted (step S1414: No), the process is performed again from step S1411. On the other hand, when the end instruction is received (step S1414: Yes), the process ends.

The termination process of the lighting device 120 may be performed by the inspector or may be terminated according to an instruction from another configuration.

Further, in the present embodiment, an example of generating the intensity image data generated by using the time correlation camera 110 and the time correlation image data has been described. However, it is not limited to using the time-correlation camera 110 to generate the intensity image data and the time-correlation image data, and a time-correlation camera that can be realized by analog processing and an operation equivalent thereto can be used. An imaging system may be used. For example, the image data generated by a normal digital still camera is output, and the information processing apparatus uses the image data generated by the digital still camera as frame image data and superimposes a reference signal to obtain time-correlated image data. It may be generated, or time-correlated image data may be generated by using a digital camera that superimposes a reference signal on the light intensity signal in the image sensor.

(Modification example 1)
In the present embodiment, an example of detecting a feature related to an abnormality based on the difference from the surroundings has been described, but the feature is not limited to the detection based on the difference from the surroundings, and the reference shape is used. The feature may be detected based on the difference from the data (reference data, for example, time correlation data, amplitude image data, phase image data, etc.). In this case, it is necessary to align and synchronize the spatial phase modulation illumination (striped pattern) with the case of the reference data.

In this modification, the abnormality detection processing unit 105 contains the amplitude image data and the phase image data obtained from the reference surface, which are stored in the storage unit 109 in advance, and the amplitude image data and the phase image data of the inspected object 150. Is compared, and it is determined whether or not there is a difference of a predetermined reference or more with respect to any one or more of the amplitude of light and the phase of light between the surface of the object to be inspected 150 and the reference surface.

In this modification, an inspection system having the same configuration as that of the first embodiment is used, and a normal surface of the object to be inspected is used as a reference surface.

While the illuminating device 120 illuminates the pattern through the screen 130, the time-correlated camera 110 images the surface of a normal subject to be inspected and generates time-correlated image data. Then, the PC 100 inputs the time correlation image data generated by the time correlation camera 110, generates the amplitude image data and the phase image data, and stores the amplitude image data and the phase image data in the storage unit 109 of the PC 100. .. Then, the time correlation camera 110 takes an image of the object to be inspected for which it is desired to determine whether or not an abnormality has occurred, and generates time correlation image data. Then, the PC 100 generates the amplitude image data and the phase image data from the time-correlation image data, and then compares the amplitude image data and the phase image data of the normal inspected object stored in the storage unit 109. At that time, the comparison result between the amplitude image data and the phase image data of the normal inspected object and the amplitude image data and the phase image data of the inspected object to be inspected is used as data showing the feature of detecting an abnormality. Output. Then, when the feature for detecting the abnormality is equal to or higher than the predetermined standard, it can be inferred that there is an abnormality in the object to be inspected 150.

Thereby, in this modified example, it can be determined whether or not there is a difference from the surface of the normal object to be inspected, in other words, whether or not an abnormality has occurred on the surface of the object to be inspected. Since any method may be used as the method for comparing the amplitude image data and the phase image data, the description thereof will be omitted.

Further, in this modification, an example of outputting data showing the characteristics of detecting anomalies based on the difference from the reference surface has been described, but the difference from the reference surface and the surroundings shown in the first embodiment have been described. May be combined with the difference in the above to calculate the characteristics for detecting anomalies. Any method may be used for the combination method, and the description thereof will be omitted.

(Modification 2)
In the first embodiment, an example in which an abnormality (defect) of an inspected object is detected by moving a stripe pattern in the x direction has been described. However, when an abnormality (defect) in which the distribution of normals changes sharply in the y direction perpendicular to the x direction occurs in the inspected object, the fringe pattern is moved in the y direction rather than the fringe pattern is moved in the x direction. It may be easier to detect defects. Therefore, in the modified example, an example in which the fringe pattern moving in the x direction and the fringe pattern moving in the y direction are alternately switched will be described.

The light emission control unit 102 of this modification switches the fringe pattern to be output to the lighting device 120 at predetermined time intervals. As a result, the lighting device 120 outputs a plurality of striped patterns extending in different directions on one inspection surface.

FIG. 15 is a diagram showing an example of switching the fringe pattern output by the light emission control unit 102 of this modified example. In FIG. 15A, the light emission control unit 102 shifts the fringe pattern displayed by the lighting device 120 in the x direction. After that, as shown in (B), the light emission control unit 102 shifts the fringe pattern displayed by the lighting device 120 in the y direction.

Then, the control unit 103 of the PC 100 performs abnormality detection based on the time correlation image data obtained from the stripe pattern irradiation of FIG. 15 (A), and is obtained from the stripe pattern irradiation of FIG. 15 (B). Anomaly detection is performed based on the time-correlated image data.

FIG. 16 is a diagram showing an example in which the light emission control unit 102 of this modified example irradiates the surface including the abnormality (defect) 1601 with a striped pattern. In the example shown in FIG. 16, the abnormality (defect) 1601 extends in the x direction. In this case, the light emission control unit 102 is set so that the fringe pattern moves in the y direction intersecting in the x direction, in other words, in the direction intersecting the longitudinal direction of the abnormality (defect) 1601. With this setting, the detection accuracy can be improved.

FIG. 17 is a diagram showing the relationship between the abnormality (defect) 1701 and the fringe pattern on the lighting device 120 when the fringe pattern is changed in the y direction, in other words, in the direction orthogonal to the longitudinal direction of the defect 1701. .. As shown in FIG. 17, when the width is narrow in the y direction and an abnormality (defect) 1701 having the x direction intersecting the y direction as the longitudinal direction occurs, the light emitted from the illuminating device 120 is emitted. The cancellation of the amplitude of light increases in the y direction intersecting the x direction. Therefore, the PC 100 can detect the abnormality (defect) from the amplitude image data corresponding to the fringe pattern moved in the y direction.

In the inspection system of this modified example, when the longitudinal direction of the defect occurring in the inspected object is random, the stripe pattern is displayed in a plurality of directions (for example, the x direction and the y direction intersecting the x direction). Therefore, the defect can be detected regardless of the shape of the defect, and the accuracy of detecting an abnormality (defect) can be improved. Further, by projecting a fringe pattern that matches the shape of the abnormality, the accuracy of detecting the abnormality can be improved.

(Modification example 3)
Further, the above-described modification 2 is not limited to the method of switching the fringe pattern when performing the abnormality detection in the x direction and the abnormality detection in the y direction. Therefore, in the third modification, an example in which the light emission control unit 102 simultaneously moves the fringe pattern output to the lighting device 120 in the x-direction and the y-direction will be described.

FIG. 18 is a diagram showing an example of a fringe pattern output to the lighting device 120 by the light emission control unit 102 of this modified example. In the example shown in FIG. 18, the light emission control unit 102 moves the fringe pattern in the direction 1801.

The fringe pattern shown in FIG. 18 includes a fringe pattern with one cycle of 1802 in the x direction and a fringe pattern with one cycle of 1803 in the y direction. That is, the fringe pattern shown in FIG. 18 has a plurality of fringes having different widths and extending in intersecting directions. It is necessary to make the width of the stripe pattern in the x direction different from the width of the stripe pattern in the y direction. Thereby, when the time-correlated image data corresponding to the x direction and the time-correlated image data corresponding to the y direction are generated, the corresponding reference signals can be made different. Since the cycle (frequency) of the change in light intensity due to the fringe pattern may be changed, the moving speed of the fringe pattern (strip) may be changed instead of changing the width of the fringe.

Then, the time correlation camera 110 generates time-correlation image data corresponding to the fringe pattern in the x direction based on the reference signal corresponding to the fringe pattern in the x direction, and is based on the reference signal corresponding to the fringe pattern in the y direction. The time-correlated image data corresponding to the fringe pattern in the y direction is generated. After that, the control unit 103 of the PC 100 performs abnormality detection based on the time-correlation image data corresponding to the fringe pattern in the x direction, and then detects the abnormality based on the time-correlation image data corresponding to the fringe pattern in the y direction. I do. As a result, in this modified example, detection can be performed regardless of the direction in which the defect has occurred, and the accuracy of detecting an abnormality (defect) can be improved.

<Abnormality judgment based on phase reversal (1)>
FIG. 19 is a block diagram of the abnormality detection processing unit 105, FIG. 20 is a side view of a sample including an abnormality region Aa in which the shape change is relatively gradual, and FIG. 21 shows a phase change in the vicinity of the abnormality region Aa. FIG. 22 is an explanatory diagram for reversing the phase.

As shown in FIG. 19, the abnormality detection processing unit 105 includes a phase differential value calculation unit 105a, a phase retrograde region detection unit 105b, and an abnormality determination unit 105c. The phase differential value calculation unit 105a and the phase retrograde region detection unit 105b are examples of arithmetic processing units.

The phase differential value calculation unit 105a calculates the directional differential value of the phase in each pixel. The directional derivative is the directional derivative in each of the directions along the image plane, for example, the x-direction and the y-direction. The directional derivative can be calculated based on the phase difference of adjacent pixels, and the phase difference of adjacent pixels is calculated, for example, based on the division of the complex number (complex luminance value) of each pixel determined by the amplitude and phase. Can be done.

The phase retrograde region detection unit 105b detects a phase retrograde region in which the phase reverses in the direction along the image plane. The phase retrograde region is a region in which the phase increases in the direction opposite to the normal region (general region). The normal area is an area in which the abnormality to be detected is not confirmed.

The phase-reverse region may appear near the boundary between the abnormal region Aa and the normal region An, as shown in FIG. 20, in which the shape changes relatively slowly. Therefore, it is possible to identify the abnormal region Aa by detecting the phase reversal region. In the part where the shape changes suddenly, the region where the phase is reversed becomes narrower (or shorter). Therefore, the detection of the phase reversal region is suitable for detecting an abnormal region Aa in which the shape change is relatively gradual, for example, a gradual swelling due to accumulation of paint droplets during painting, a gradual dent, or the like. ..

In the example of FIG. 21, in the phase forward region Ap, the phase φ of each pixel increases toward the x direction, whereas in the phase retrograde region Ar, the phase φ of each pixel decreases toward the x direction. .. Since the range of phase φ is limited to −π to + π, in the region where the phase is increasing, the phase skips to −π when it reaches + π, and the phase is decreasing. Then, the phase skips to + π at the position where it reaches −π. As shown in FIG. 21, the sign of the directional differential value in the x direction along the image plane of the phase φ in the phase forward region Ap is + (positive), whereas the x of the phase φ in the phase retrograde region Ar is x. The sign of the directional differential value of the direction is- (negative). That is, the phase retrograde region Ar is a region in which the sign of the directional differential value of the phase φ (− in FIG. 22) is different from the sign of the directional differential value of the phase φ of the phase forward region Ap (+ in FIG. 22). ..

As shown in FIG. 22, when the inspection surface gradually rises toward the x direction, that is, when the gradient of the inspection surface in the x direction gradually increases, the phase from point 1 to point 7 increases toward the x direction. However, the phase starts to reverse from the position between the points 7 and 8, and decreases from the point 8 to the point 11 in the x direction. Further, the phase returns to the forward direction from the position between the points 11 and 12, and the phase increases from the point 12 to the point 14 in the x direction. In this case, the region between the points 1 and 7 and the region between the points 12 and 14 is the phase forward region Ap, and the region between the points 8 and 11 is the phase retrograde region Ar. Become.

The phase retrograde region detection unit 105b detects the phase retrograde region Ap and the phase retrograde region Ar by grouping and labeling each pixel according to the sign of the directional derivative value of the phase. That is, when the phase retrograde region detection unit 105b has the same sign of the directional differential value of the phase for a plurality of pixels adjacent to each other, the phase retrograde region Ap or the phase forward region Ap or Let the phase retrograde region Ar be. The phase reversal region detection unit 105b may be subjected to morphology processing such as expansion, contraction, opening, and closing, for example. Further, the phase retrograde region detection unit 105b can also be referred to as a phase retrograde region detection unit.

The abnormality determination unit 105c determines the abnormality region Aa based on the detection result by the phase reversal region detection unit 105b.

Specifically, the abnormality determination unit 105c is, for example, a case where at least a part of the phase retrograde region Ar is located in the target region M (see FIGS. 20, 23, 25), and the phase retrograde region Ar is the case. When the area (the number of pixels included) is within a predetermined range, the phase reversal region Ar can be determined as the abnormal region Aa. The target area M is an example of a predetermined area. Further, when the width is within a predetermined range, it means that the width is larger than the first threshold value and smaller than the second threshold value. In this case, the first threshold value can be set to distinguish from abnormalities such as minute irregularities that are not to be detected, for example. Also, the second threshold can be set empirically, for example, based on the size of a typical anomalous region Aa.

Further, in the abnormality determination unit 105c, for example, when the phase forward region Ap is sandwiched between two phase retrograde regions Ar determined to be the abnormal region Aa, or in the phase retrograde region Ar determined to be the abnormal region Aa. When it is surrounded, when it is adjacent to the phase retrograde region Ar determined to be the abnormal region Aa, and when the width of the phase forward region Ap is smaller than the third threshold value, the relevant case is concerned. The phase forward region Ap can be determined as the abnormal region Aa. Since the phase retrograde region Ar appears near the boundary between the abnormal region Aa and the normal region An whose shape changes relatively slowly, the phase forward region Ap satisfying such a condition can be presumed to be the abnormal region Aa. .. The third threshold can be set empirically, for example, based on the size of a typical anomalous region Aa.

Further, the anomaly determination unit 105c can determine the phase retrograde region Ar as the anomaly region Aa when the representative value (for example, the average value) of the amplitude of the phase retrograde region Ar is smaller than the fourth threshold value. The fourth threshold value may be set based on, for example, the amplitude of a typical abnormal region Aa, or may be set as a ratio to the amplitude of a normal region around the abnormal region Aa. The fourth threshold is an example of a predetermined value.

<Abnormality judgment based on phase reversal (2)>
Further, the phase retrograde region detection unit 105b may detect the phase retrograde region Ar based on the frequency filtering in the spatial frequency region of the time-correlated image. Specifically, the phase retrograde domain detection unit 105b executes (1) a two-dimensional Fourier transform (discrete Fourier transform, fast Fourier transform) in the x-direction and the y-direction on the complex time correlation image, and (2) ) In the spatial frequency domain, the frequency domain whose frequency code is + (positive) or-(negative) is extracted, and (3) two-dimensional inverse Fourier transform (inverse discrete Fourier transform, inverse fast Fourier transform) for the extracted frequency domain. ) To obtain a frequency-filtered time-correlated image. When the sign of the directional derivative of the phase is + (positive), the sign of the frequency is + (positive), and when the sign of the directional derivative of the phase is- (negative), the sign of the frequency is- (negative). It becomes. Therefore, by such a method, the phase reversal region Ar can be detected more easily or more reliably.

FIGS. 23 to 25 show an example of the verification result of the detection of the phase reversal region Ar by the frequency filtering in the spatial frequency region. FIG. 23 is an example of a phase image of a sample including an abnormality (region A1) in which the shape change is relatively gradual, and FIG. 24 is a two-dimensional power spectrum in the spatial frequency region of the time-correlated image of the sample of FIG. 23. 25 is a phase image corresponding to the frequency domain SFF of FIG. 24, that is, a phase image obtained by performing an inverse Fourier transform on the frequency domain SFF. In FIG. 23, a black-and-white striped pattern shows how the phase changes from −π (black) to + π (white) on the inspection surface. The horizontal axis of FIG. 24 is the frequency in the x direction of FIGS. 23 and 24, and the vertical axis of FIG. 24 is the frequency of the y direction of FIGS. 23 and 24. Further, in the frequency domain SFF, the range of the frequency fx in the x direction is 0 <fx <Lx, and the range of the frequency fy in the y direction is −Ly <fy <+ Ly. In FIG. 25, for convenience, black and white are reversed from those in FIG. 23, and the portion without data is shown in white.

Looking at FIG. 23, it can be seen that the phase is disturbed in the portion of the region A1 in the broken line DL. According to another more detailed analysis, the traveling direction of the phase of the region A1 is opposite to that of the peripheral portion (normal region An). That is, in the normal region An, the phase increases toward the right direction (opposite direction in the x direction) of FIG. 23, and in the region A1, the phase increases toward the left direction (x direction) in FIG. 23. Here, as shown in FIG. 24, the phase retrograde domain detection unit 105b (1) executes a two-dimensional Fourier transform on the time-correlated image of FIG. 23 in the x-direction and the y-direction, and (2) further, x. The + (positive) frequency domain SFF in the direction is extracted, and (3) the frequency domain SFF is subjected to a two-dimensional inverse Fourier transform to obtain a phase image as shown in FIG. 25. In this case, the phase retrograde region Ar is detected as the region A1 having a frequency of + (positive), that is, the region A1 having a directional differential value of the phase of + (positive).

As described above, in the present embodiment, in the phase retrograde region detection unit 105b, the sign of the directional differential value in the direction (x direction, y direction) along the image plane of the phase of the time-correlation image is the normal region An. The phase retrograde region Ar different from the symbol is detected, and the abnormality determination unit 105c determines the abnormality region Aa based on the detection result of the phase retrograde region detection unit 105b. The phase-reverse region Ar appears, for example, at the boundary between the abnormal region Aa and the normal region An whose shape changes relatively slowly. Therefore, according to the present embodiment, for example, based on the sign of the directional derivative value of the phase, the abnormal region Aa in which the shape change is relatively gradual, which is difficult to find by the conventional method, can be more easily or more reliably obtained. It can be determined.

Further, in the present embodiment, when at least a part of the phase retrograde region Ar is included in the target region M (predetermined region) (location condition), the abnormality determination unit 105c makes the phase retrograde region Ar abnormal. It may be determined as the region Aa, or the abnormality determination unit 105c determines that the phase retrograde region Ar is the abnormal region Aa when the width of the phase retrograde region Ar is within a predetermined range (a condition of the width). Alternatively, the abnormality determination unit 105c abnormalizes the phase retrograde region Ar when the representative value of the amplitude of the phase retrograde region Ar is smaller than the fourth threshold value (a predetermined value) (amplitude condition). It may be determined as the region Aa. Therefore, according to the present embodiment, for example, the abnormal region Aa can be detected more accurately according to the place where the phase reversal region Ar appears, the width, the amplitude, and the like. Note that these conditions may be OR conditions (logical sum). Further, the abnormal region Aa may be determined on the condition of specifications other than the location (position), the width (size), and the amplitude (for example, the shape, the extending direction, etc.). Further, the abnormal region Aa may be determined by various combinations of these conditions.

Further, in the present embodiment, the phase retrograde region detection unit 105b detects the phase retrograde region Ar based on the frequency filtering in the spatial frequency region of the time-correlated image. Therefore, according to the present embodiment, for example, the phase reversal region Ar can be detected more easily and more accurately, and the abnormal region Aa can be discriminated.

The inspection program executed on the PC100 of the above-described embodiment is a file in an installable format or an executable format on a computer such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versailles Disk). It is recorded and provided on a readable recording medium.

Further, the inspection program executed by the PC 100 of the above-described embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading via the network. Further, the inspection program and the calibration program executed by the PC100 of the above-described embodiment may be configured to be provided or distributed via a network such as the Internet.

Although some embodiments and modifications of the present invention have been described, these embodiments and modifications are presented as examples and are not intended to limit the scope of the invention. These novel embodiments and modifications can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

1 ... Inspection system, 104 ... Amplitude-phase image generation unit (calculation processing unit), 105a ... Phase differential value calculation unit (calculation processing unit), 105b ... Phase retrograde region detection unit (calculation processing unit), 105c ... Abnormality determination unit , 110 ... Time-correlated camera (imaging unit, time-correlated image generation unit), 120 ... Illumination device (illumination unit), Aa ... Abnormal region, M ... Target region (predetermined region), Ar ... Phase-reverse region, x, y … Direction along the image plane.

Claims (5)

A surface illumination unit that gives a periodic time change of light intensity by spatial movement of a fringe pattern of light intensity,
A time-correlation image generator that generates a time-correlation image, which is a complex image represented by intensity and phase, by a time-correlation camera or an imaging system that operates equivalently.
From the time correlation image, the arithmetic processing unit that calculates the feature corresponding to the distribution of the normal vector of the inspection surface and detects the abnormality by at least one of the difference from the surroundings and the difference from the reference surface. ,
An abnormality discriminating unit that distinguishes the inspection surface into a normal region and an abnormal region based on the feature of detecting the abnormality,
With
The arithmetic processing unit includes a phase retrograde region detection unit that detects a phase retrograde region in which the sign of the directional differential value in the direction along the image plane of the phase of the time-correlated image is different from the sign of the normal region.
The abnormality determination unit is an inspection system that determines an abnormality region based on the phase retrograde region. The inspection system according to claim 1, wherein the abnormality determination unit determines the phase retrograde region as an abnormal region when at least a part of the phase retrograde region is included in a predetermined region. The inspection system according to claim 1 or 2, wherein the abnormality determination unit determines the phase retrograde region as an abnormal region when the width of the phase retrograde region is within a predetermined range. The abnormality determination unit according to any one of claims 1 to 3, wherein when the representative value of the amplitude of the phase retrograde region is smaller than a predetermined value, the abnormality determination unit determines the phase retrograde region as an abnormal region. Inspection system. The inspection system according to any one of claims 1 to 4, wherein the phase retrograde region detection unit detects the phase retrograde region based on frequency filtering in the spatial frequency region of the time-correlated image.

Images