JP6420131B2 - Inspection system and inspection method - Google Patents

Description

  The present invention relates to an inspection system and an inspection method.

  Conventionally, there has been proposed a technique for irradiating an object to be inspected, imaging reflected light from the surface of the object to be inspected as image data, and detecting an abnormality of the object to be inspected based on a change in luminance of the image data. Has been.

  In this case, a technique has been proposed in which an inspected object is irradiated with line illumination having a different wavelength, and an abnormality is detected based on a change in luminance of the captured image data.

JP 2014-092477 A

  However, in the prior art, the purpose is to obtain specular reflection light, and it is difficult to inspect the state in the transparent body. Furthermore, in the prior art, since the captured image data does not change the light intensity, the captured image data naturally includes information on time transitions when the light intensity is changed. Not. For this reason, when detecting an abnormality of the object to be inspected with the captured image data, the detection accuracy may be lowered.

  In order to solve the above-described problems and achieve the object, the inspection system of the present invention is a planar illumination unit that gives a periodic temporal change in the light intensity by moving the fringe pattern caused by the difference in light intensity. And the optical signal reflected from the inspection object during the imaging interval using a time-correlation camera or an imaging system that operates equivalently, in which the period of the fringe pattern generated in the illumination unit and the imaging interval match. A first image generating unit that generates intensity image data indicating a luminance value obtained from the above and amplitude image data calculated from a luminance change of an optical signal reflected from the inspection target during an imaging interval; Based on the intensity image data and amplitude image data generated by the generation unit, the specular reflection image data indicating the component reflected from the surface of the transparent layer to be inspected and the component reflected from the inside of the transparent layer are shown. A second image generation unit for generating a distributed reflection image data, and comprising a.

FIG. 1 is a diagram illustrating a configuration example of an inspection system according to the first embodiment. FIG. 2 is a block diagram showing the configuration of the time correlation camera of the first embodiment. FIG. 3 is a conceptual diagram showing frames accumulated in time series in the time correlation camera of the first embodiment. FIG. 4 is a diagram illustrating an example of a fringe pattern irradiated by the illumination device of the first embodiment. FIG. 5 is a diagram illustrating a first detection example of abnormality of an object to be inspected by the time correlation camera according to the first embodiment. FIG. 6 is a diagram showing an example of the amplitude of light that changes in accordance with the abnormality when the abnormality shown in FIG. FIG. 7 is a diagram illustrating a second example of detection of abnormality of the inspected object by the time correlation camera according to the first embodiment. FIG. 8 is a diagram illustrating a third example of detection of abnormality of an object to be inspected by the time correlation camera of the first embodiment. FIG. 9 is a diagram illustrating an example of a fringe pattern output from the illumination control unit according to the first embodiment to the illumination device. FIG. 10 is a diagram illustrating an example of a wave shape representing a stripe pattern after passing through the screen of the first embodiment. FIG. 11 is a diagram illustrating light reflected from the clear layer and the inner layer of the object to be inspected. FIG. 12 is a diagram illustrating regular reflection components and diffuse reflection components received by the time correlation camera of the first embodiment. FIG. 13 is a diagram illustrating a concept for generating image data indicating a diffuse reflection component. FIG. 14 is a flowchart illustrating a procedure of abnormality detection processing based on amplitude image data in the abnormality detection processing unit of the first embodiment. FIG. 15 is a flowchart illustrating a procedure of an inspection process for an object to be inspected in the inspection system according to the first embodiment. FIG. 16 is a diagram illustrating a switching example of a fringe pattern output from the illumination control unit according to the first modification. FIG. 17 is a diagram illustrating an example in which the illumination control unit of Modification 1 irradiates the surface including the abnormality (defect) with a stripe pattern. FIG. 18 is a diagram showing a relationship between an abnormality (defect) and a stripe pattern on the screen when the stripe pattern is changed in the y direction. FIG. 19 is a diagram illustrating an example of a fringe pattern output to the illumination device by the illumination control unit of the present modification.

(First embodiment)
The inspection system of this embodiment will be described. The inspection system of the first embodiment has various configurations for inspecting an object to be inspected. FIG. 1 is a diagram illustrating a configuration example of an inspection system according to the present embodiment. As shown in FIG. 1, the inspection system of this embodiment includes a PC 100, a time correlation camera 110, a lighting device 120, a screen 130, and an arm 140.

  The arm 140 is used to fix the inspection object 150 and changes the position and orientation of the surface of the inspection object 150 that can be imaged by the time correlation camera 110 according to control from the PC 100.

  The illuminating device 120 is a device that irradiates light to the object 150 to be inspected, and can control the intensity of irradiated light in units of regions in accordance with a stripe pattern from the PC 100. Furthermore, the illuminating device 120 can control the intensity | strength of the light of the said area unit according to periodic time transition. In other words, the lighting device 120 can give a periodic temporal change and a spatial change of the light intensity. A specific light intensity control method will be described later.

  The screen 130 diffuses the light output from the illuminating device 120 and then irradiates the test object 150 with light in a plane. The screen 130 according to the present embodiment irradiates the object 150 in a surface with the light input from the illumination device 120 and subjected to periodic time change and space change. An optical system component (not shown) such as a condensing Fresnel lens may be provided between the illumination device 120 and the screen 130.

  In addition, although this embodiment demonstrates the example which comprises the planar irradiation part which combines the illuminating device 120 and the screen 130, and gives the periodic time change and spatial change of light intensity, such a combination is demonstrated. For example, the illumination unit may be configured by arranging LEDs in a plane.

  The time correlation camera 110 includes an optical system 210, an image sensor 220, a data buffer 230, a control unit 240, and a reference signal output unit 250. FIG. 2 is a block diagram showing the configuration of the time correlation camera 110 of the present embodiment.

  The optical system 210 includes a photographic lens and the like, transmits a light beam from a subject (including an object to be inspected) outside the time correlation camera 110, and forms an optical image of the subject formed by the light beam.

  The image sensor 220 is a sensor that can output the intensity of light incident through the optical system 210 as a light intensity signal at high speed for each pixel.

  The light intensity signal of the present embodiment is a signal obtained by the illumination device 120 of the inspection system irradiating a subject (including an object to be inspected) with light and the image sensor 220 receiving reflected light from the subject.

  The image sensor 220 is, for example, a sensor that can be read out at a higher speed than a conventional sensor, and has a two-dimensional planar shape in which pixels are arranged in two kinds of directions: a row direction (x direction) and a column direction (y direction). It shall be configured. Each pixel of the image sensor 220 is defined as a pixel P (1,1),..., P (i, j),..., P (X, Y) (Note that the image size in this embodiment is X X Y). Note that the reading speed of the image sensor 220 is not limited and may be the same as the conventional one.

  The image sensor 220 receives a light beam from a subject (including an object to be inspected) transmitted by the optical system 210 and photoelectrically converts the light intensity signal (photographing) indicating the intensity of light reflected from the subject. Signal) is generated and output to the control unit 240. The image sensor 220 according to the present embodiment outputs the frame for each readable unit time.

  The control unit 240 according to the present embodiment includes, for example, a CPU, a ROM, a RAM, and the like. By executing an inspection program stored in the ROM, the transfer unit 241, the reading unit 242, and the intensity image superimposing unit 243. And a first multiplier 244, a first correlation image superimposing unit 245, a second multiplier 246, a second correlation image superimposing unit 247, and an image output unit 248. Note that the present invention is not limited to implementation with a CPU or the like, and may be implemented with an FPGA or an ASIC.

  The transfer unit 241 stores the frames composed of the light intensity signals output from the image sensor 220 in the data buffer 230 in time series order.

  The data buffer 230 accumulates frames composed of light intensity signals output from the image sensor 220 in time series.

  FIG. 3 is a conceptual diagram showing frames accumulated in time series in the time correlation camera 110 of the present embodiment. As shown in FIG. 3, the data buffer 230 of the present embodiment stores a plurality of light intensity signals G (1, 1, t), every time t (t = t0, t1, t2,..., Tn). .., G (i, j, t),..., G (X, Y, t) are combined into a plurality of frames Fk (k = 1, 2,..., N) in chronological order. Accumulated. Note that one frame created at time t is a light intensity signal G (1, 1, t),..., G (i, j, t), ..., G (X, Y, t). Composed.

  A light intensity signal (imaging signal) G (1,1, t),..., G (i, j, t),. Each pixel P (1,1),..., P (i, j),..., P (X, Y) constituting k = 1, 2,.

  The frame output from the image sensor 220 includes only a light intensity signal, in other words, it can be considered as monochrome image data. In this embodiment, an example in which the image sensor 220 generates monochrome image data in consideration of resolution, sensitivity, cost, and the like will be described. However, the image sensor 220 is not limited to a monochrome image sensor. Alternatively, a color image sensor may be used.

  Returning to FIG. 2, the reading unit 242 of the present embodiment receives the light intensity signals G (1,1, t),..., G (i, j, t),. Y, t) are read out in frame-by-frame order and output to the first multiplier 244, the second multiplier 246, and the intensity image superimposing unit 243.

  The time correlation camera 110 of the present embodiment generates image data for each output destination of the reading unit 242. In other words, the time phase camera 110 creates three types of image data.

  The time correlation camera 110 of this embodiment generates intensity image data and two types of time correlation image data as three types of image data. Note that the present embodiment is not limited to generating three types of image data, and it may be possible to generate no intensity image data or to generate one or more types of time-correlated image data. .

  As described above, the image sensor 220 of the present embodiment outputs a frame composed of a light intensity signal every unit time. However, in order to generate normal image data, a light intensity signal corresponding to the exposure time necessary for photographing is required. Therefore, in the present embodiment, the intensity image superimposing unit 243 generates intensity image data by superimposing a plurality of frames for an exposure time necessary for photographing. In addition, each pixel value (value representing the intensity of light) G (x, y) of the intensity image data can be derived from the following equation (1). The exposure time is the time difference between t0 and tn. The intensity image data is image data indicating a luminance value by an optical signal reflected from the inspection object while being imaged by the time correlation camera 110 (during an imaging interval).

  Thereby, the intensity image data in which the subject (including the object to be inspected) is photographed is generated in the same manner as the conventional camera photographing. Then, the intensity image superimposing unit 243 outputs the generated intensity image data to the image output unit 248.

  The time correlation image data is image data indicating changes in light intensity according to time transition. That is, in the present embodiment, for each frame in time series order, the light intensity signal included in the frame is multiplied by the reference signal indicating the time transition, and the reference signal, the light intensity signal, and the time that is the multiplication result Temporal correlation image data is generated by generating a temporal correlation value frame composed of correlation values and superimposing a plurality of temporal correlation value frames.

  By the way, in order to detect abnormality of the object to be inspected using the time correlation image data, it is necessary to change the light intensity signal input to the image sensor 220 in synchronization with the reference signal. For this purpose, as described above, the illumination device 120 performs planar light irradiation that periodically gives temporal changes and spatial movement of the stripes via the screen 130.

In this embodiment, two types of time correlation image data are generated. The reference signal may be a signal representing a time transition, but in the present embodiment, a complex sine wave e ^−jωt is used. It is assumed that the angular frequency ω and time t. The angular frequency ω is such that the complex sine wave e ^−jωt representing the reference signal correlates with one period of the above-described exposure time (in other words, the time required to generate the intensity image data and the time correlation image). It shall be set. In other words, the planar and dynamic light formed by the illumination unit 120 and the illumination unit such as the screen 130 is in a first period (time period) at each position on the surface (reflection surface) of the inspection object 150. And a distribution of increase or decrease in spatial irradiation intensity in a second period (spatial period) along at least one direction along the surface. When this planar light is reflected by the surface, it is complex-modulated according to the specifications of the surface (normal vector distribution, etc.). The time correlation camera 110 receives the light complex-modulated on the surface and performs quadrature detection (orthogonal demodulation) using the reference signal of the first period, thereby obtaining time correlation image data as a complex signal. By modulation / demodulation based on such complex time correlation image data, it is possible to detect features corresponding to the surface normal vector distribution.

The complex sine wave e ^−jωt can also be expressed as e ^−jωt = cos (ωt) −j · sin (ωt). Accordingly, each pixel value C (x, y) of the time correlation image data can be derived from the following equation (2).

  In this embodiment, in the formula (2), two types of time correlation image data are divided into a pixel value C1 (x, y) representing the real part and a pixel value C2 (x, y) representing the imaginary part. Generate.

For this reason, the reference signal output unit 250 generates and outputs different reference signals for the first multiplier 244 and the second multiplier 246, respectively. Reference signal output section 250 of this embodiment outputs the first reference signal cosωt corresponding to the real part of the complex sine wave e ^-Jeiomegati the first multiplier 244, the imaginary part of the complex sine wave e ^-Jeiomegati The corresponding second reference signal sin ωt is output to the second multiplier 246. As described above, the reference signal output unit 250 according to the present embodiment describes an example in which two types of reference signals expressed as time functions of a sine wave and a cosine wave that form a Hilbert transform pair are described. Any reference signal that changes with time transition such as a function may be used.

The first multiplier 244 corresponds to the real part of the complex sine wave e ^−jωt input from the reference signal output unit 250 for each light intensity signal of the frame in units of frames input from the reading unit 242. The first reference signal cosωt to be multiplied.

  The first correlation image superimposing unit 245 performs a process of superimposing the multiplication result of the first multiplier 244 on a pixel-by-pixel basis for a plurality of frames for an exposure time necessary for photographing. Thereby, each pixel value C1 (x, y) of the first time correlation image data is derived from the following equation (3).

Then, the second multiplier 246 corresponds to the second imaginary part of the complex sine wave e ^−jωt input from the reference signal output unit 250 with respect to the light intensity signal of the frame input from the reading unit 242. Multiply the reference signal sin ωt.

  The second correlation image superimposing unit 247 performs a process of superimposing the multiplication result of the second multiplier 246 on a pixel-by-pixel basis for a plurality of frames corresponding to the exposure time necessary for photographing. Thereby, each pixel value C2 (x, y) of the second time correlation image data is derived from the following equation (4).

  By performing the processing described above, two types of time correlation image data, in other words, time correlation image data having two degrees of freedom can be generated.

Further, the present embodiment does not limit the type of reference signal. For example, in this embodiment, two types of time-correlated image data of the real part and the imaginary part of the complex sine wave e ^−jωt are created, but two types of image data based on the light amplitude and the light phase are generated. May be.

  Note that the time correlation camera 110 of the present embodiment can create a plurality of systems as time correlation image data. Thereby, for example, when light in which stripes having a plurality of types of widths are combined is irradiated, two types of time correlation image data based on the real part and the imaginary part described above can be created for each stripe width. For this purpose, the time correlation camera 110 includes a combination of two multipliers and two correlation image superimposing units for a plurality of systems, and the reference signal output unit 250 has an angular frequency suitable for each system. The reference signal by ω can be output.

  Then, the image output unit 248 outputs two types of time correlation image data and intensity image data to the PC 100. As a result, the PC 100 detects an abnormality of the object to be inspected using the two types of time correlation image data and the intensity image data. For that purpose, it is necessary to irradiate the subject with light.

  The illuminating device 120 of this embodiment irradiates the fringe pattern which moves at high speed. FIG. 4 is a diagram illustrating an example of a fringe pattern irradiated by the illumination device 120 of the present embodiment. In the example shown in FIG. 4, the stripe pattern is scrolled (moved) in the x direction. A white area is a bright area corresponding to the stripe, and a black area is an interval area (dark area) corresponding to the stripe.

  In this embodiment, a projector is used as the illumination device 120. Then, the light irradiated from the illumination device 120 that is a projector is irradiated to the subject (including the object to be inspected) through the screen.

  In the present embodiment, the fringe pattern irradiated by the illuminating device 120 is moved by one period with the exposure time when the time correlation camera 110 captures the intensity image data and the time correlation image data. Thereby, the illuminating device 120 gives the time change of the light intensity periodically by the spatial movement of the stripe pattern of the light intensity. In the present embodiment, the time during which the fringe pattern in FIG. 4 moves by one period corresponds to the exposure time, so that each pixel of the time-correlated image data relates to at least the light intensity signal for one period of the fringe pattern. Information is embedded.

  As shown in FIG. 4, in the present embodiment, an example in which the illumination device 120 irradiates a stripe pattern based on a rectangular wave will be described, but other than a rectangular wave may be used. In the present embodiment, the illumination device 120 is irradiated through the screen 130, so that the bright and dark boundary region of the rectangular wave can be blurred.

  First, the case where the fringe pattern irradiated by the illumination device 120 is expressed as A + Acos (ωt + kx) will be described. That is, the stripe pattern includes a plurality of stripes repeatedly (periodically). It should be noted that the intensity of light applied to the object to be inspected can be adjusted between 0 and 2A, and is the light phase kx. k is the wave number of the stripe. x is the direction in which the phase changes.

  The fundamental frequency component of the light intensity signal f (x, y, t) of each pixel in the frame can be expressed as the following equation (5). As shown in Expression (5), the brightness of the stripe changes in the x direction.

f (x, y, t) = A (1 + cos (ωt + kx))
= A + A / 2 {e ^{j (ωt + kx)} + e ^{−j (ωt + kx)} } (5)

  As shown in Expression (5), the intensity signal of the fringe pattern irradiated by the illumination device 120 can be considered as a complex number.

  Then, the light from the illumination device 120 is reflected and input to the image sensor 220 from the subject (including the object to be inspected).

  Therefore, the light intensity signal G (x, y, t) input to the image sensor 220 can be used as the light intensity signal f (x, y, t) of each pixel of the frame when the illumination device 120 is irradiated. Therefore, when Expression (5) is substituted into Expression (1) for deriving intensity image data, Expression (6) can be derived. The phase is kx.

  From Expression (6), it can be confirmed that each pixel of the intensity image data is input with a value obtained by multiplying the exposure time T by the intermediate value A of the intensity of the light output from the illumination device 120. Further, when Expression (5) is substituted into Expression (2) for deriving time correlation image data, Expression (7) can be derived. Note that AT / 2 is the amplitude and kx is the phase.

  Thereby, the time correlation image data shown by the complex number shown by Formula (7) is replaceable with the two types of time correlation image data mentioned above. That is, the above-described time correlation image data composed of the real part and the imaginary part includes a phase change and an amplitude change due to a change in light intensity irradiated on the test object. In other words, the PC 100 according to the present embodiment can detect the phase change of the light emitted from the illumination device 120 and the light amplitude change based on the two types of time correlation image data. Therefore, the PC 100 according to the present embodiment, based on the time correlation image data and the intensity image data, amplitude image data representing the amplitude of light entering each pixel and phase image data representing the phase change of light entering each pixel. And can be generated. And abnormality of a to-be-inspected object is detected by using any one or more among the produced | generated amplitude image data and phase image data.

  By the way, when an abnormality based on the unevenness occurs in the surface shape of the object to be inspected, a change corresponding to the abnormality occurs in the distribution of normal vectors on the surface of the object to be inspected. Further, when an abnormality that absorbs light occurs on the surface of the object to be inspected, the intensity of the reflected light changes. A change in the normal vector distribution is detected as at least one of a phase change and an amplitude change of light. Thus, in the present embodiment, using the time correlation image data and the intensity image data, at least one of the light phase change and the amplitude change corresponding to the change in the normal vector distribution is detected. Thereby, the abnormality of the surface shape can be detected. Next, the relationship between the abnormality of the inspected object, the normal vector, and the phase change or amplitude change of light will be described.

  FIG. 5 is a diagram illustrating a first detection example of abnormality of an object to be inspected by the time correlation camera 110 according to the first embodiment. In the example shown in FIG. 5, it is assumed that the inspected object 500 has a projecting shape abnormality 501. In this situation, it can be confirmed that the normal vectors 521, 522, and 523 are in different directions in the region near the point 502 of the abnormality 501. The normal vectors 521, 522, 523 are directed in different directions, so that diffusion (for example, light 511, 512, 513) is generated in the light reflected from the anomaly 501, and the image sensor 220 of the time correlation camera 110. The width 503 of the fringe pattern entering the arbitrary pixel 531 is increased.

  FIG. 6 is a diagram showing an example of the amplitude of light that changes in accordance with the abnormality 501 shown in FIG. In the example shown in FIG. 6, the amplitude of light is divided into a real part (Re) and an imaginary part (Im) and is represented on a two-dimensional plane. In FIG. 6, the light amplitudes 611, 612, and 613 corresponding to the lights 511, 512, and 513 in FIG. The light amplitudes 611, 612, and 613 cancel each other, and light having an amplitude 621 is incident on the arbitrary pixel 531 of the image sensor 220.

  Therefore, in the situation shown in FIG. 6, it can be confirmed that the amplitude is small in the region where the abnormality 501 of the inspection object 500 is imaged. In other words, when there is a region that is darker than the surroundings in the amplitude image data showing the change in amplitude, it can be assumed that there is a cancellation of the amplitude of light in the region, It can be determined that an abnormality 501 has occurred at the position of the corresponding inspection object 500.

  The inspection system according to the present embodiment is not limited to the one in which the inclination changes steeply like the abnormality 501 in FIG. 5, and can also detect an abnormality that changes gently. FIG. 7 is a diagram illustrating a second example of detection of abnormality of an object to be inspected by the time correlation camera 110 according to the first embodiment. In the example shown in FIG. 7, the surface of the object to be inspected is flat (in other words, the normal line is parallel) in the normal state, but a gentle gradient 701 is generated on the object 700 to be inspected. In such a situation, the normal vectors 721, 722, and 723 on the gradient 701 also change gently. Accordingly, the light beams 711, 712, and 713 input to the image sensor 220 are also shifted little by little. In the example shown in FIG. 7, since the light amplitudes do not cancel each other due to the gentle gradient 701, the light amplitudes shown in FIGS. 5 and 6 hardly change. However, although the light originally projected from the screen 130 should enter the image sensor as it is, the light projected from the screen 130 does not enter the image sensor in a parallel state because of the gentle gradient 701. A phase change occurs in the light. Accordingly, by detecting the difference between the light phase change and the surroundings, an abnormality due to the gentle gradient 701 as shown in FIG. 7 can be detected.

  In addition, there may be an abnormality other than the surface shape of the inspection object (in other words, the distribution of the normal vector of the inspection object). FIG. 8 is a diagram illustrating a third example of detection of abnormality of an object to be inspected by the time correlation camera 110 according to the first embodiment. In the example shown in FIG. 8, since the dirt 801 is attached to the object 800, the light irradiated from the illumination device 120 is absorbed or diffusely reflected, and the dirt 801 of the time correlation camera 110 is photographed. This shows an example in which light hardly changes in intensity in an arbitrary pixel region. In other words, in an arbitrary pixel area where the dirt 801 is photographed, the light intensity causes a phase cancellation, the vibration component is canceled, and an almost DC brightness is shown.

  In such a case, in the pixel region where the dirt 801 is imaged, there is almost no light amplitude, and therefore when the amplitude image data is displayed, a region darker than the surroundings is generated. Therefore, it can be estimated that there is an abnormality 801 such as dirt at the position of the inspection object 800 corresponding to the region.

  As described above, in the present embodiment, it is possible to estimate that the object to be inspected is abnormal by detecting the change in the amplitude of the light and the change in the phase of the light based on the time correlation image data.

  Returning to FIG. 1, the PC 100 will be described. The PC 100 controls the entire detection system. The PC 100 includes an arm control unit 101, an illumination control unit 102, and a control unit 103.

  The arm control unit 101 controls the arm 140 in order to change the surface of the object 150 to be imaged by the time correlation camera 110. In the present embodiment, in the PC 100, a plurality of surfaces to be imaged of the inspected object are set. Then, every time the time correlation camera 110 finishes photographing the object to be inspected 150, the arm control unit 101 can photograph the surface on which the time correlation camera 110 is set according to the setting. Move 150. The present embodiment is not limited to repeating the movement of the arm each time shooting is completed and stopping the shooting before the shooting is started, and the arm 140 may be continuously driven. The arm 140 may also be referred to as a transport unit, a moving unit, a position changing unit, a posture changing unit, or the like.

  The illumination control unit 102 outputs a fringe pattern irradiated by the illumination device 120 in order to inspect the inspected object 150. The illumination control unit 102 of this embodiment transfers at least three or more stripe patterns to the illumination device 120 and instructs the illumination device 120 to switch and display the stripe patterns during the exposure time.

  FIG. 9 is a diagram illustrating an example of a fringe pattern output from the illumination control unit 102 to the illumination device 120. In accordance with the rectangular wave shown in FIG. 9B, the illumination control unit 102 performs control so that the stripe pattern in which the black region and the white region shown in FIG. 9A are set is output.

  In this embodiment, the stripe interval for each stripe pattern to be irradiated is set according to the size of the abnormality (defect) to be detected, and detailed description thereof is omitted here.

  In addition, the angular frequency ω of the rectangular wave for outputting the fringe pattern is set to the same value as the angular frequency ω of the reference signal.

  As shown in FIG. 9, the fringe pattern output from the illumination control unit 102 can be shown as a rectangular wave, but the border area of the fringe pattern is blurred through the screen 130, that is, the bright area in the fringe pattern. By making the intensity change of light at the boundary between the (stripe region) and the dark region (interval region) gentle (dull), it can be approximated to a sine wave. FIG. 10 is a diagram illustrating an example of a wave shape representing a stripe pattern after passing through the screen 130. As shown in FIG. 10, the measurement accuracy can be improved by the wave shape approaching a sine wave. Further, a gray region in which the brightness changes in multiple steps may be added to the stripe, or a gradation may be given. Further, a stripe pattern including color stripes may be used.

  Returning to FIG. 1, the control unit 103 includes an amplitude-phase image generation unit 104, a reflection component image generation unit 105, and an abnormality detection processing unit 106, and intensity image data input from the time correlation camera 110, Based on the time correlation image data, a process for calculating a characteristic corresponding to the distribution of the normal vector of the inspection target surface of the inspection object 150 and detecting an abnormality based on a difference from the surroundings is performed. In the present embodiment, in order to perform the inspection, two types of real number and imaginary part of complex number correlation image data are used instead of time correlation image data (referred to as complex time correlation image data) indicated by complex numbers. Are received from the time correlation camera 110.

  The amplitude-phase image generation unit 104 generates amplitude image data and phase image data based on the intensity image data input from the time correlation camera 110 and the time correlation image data.

  The amplitude image data is image data representing the amplitude of light entering each pixel, calculated from the change in luminance of the optical signal reflected from the inspection target surface during imaging by the time correlation camera 110 (during the imaging interval). And The phase image data is image data representing the phase of light entering each pixel.

  Although the present embodiment does not limit the calculation method of the amplitude image data, for example, the amplitude-phase image generation unit 104 has pixel values C1 (x, y) and C2 (x, From y), each pixel value F (x, y) of the amplitude image data can be derived using Equation (8).

  By the way, the surface to be inspected of the object to be inspected is composed of, for example, a base, a surface coated on the base, and a plurality of layers such as a transparent layer (hereinafter also referred to as a clear layer). May be. In such a case, if it is possible to detect which layer has an abnormality, the abnormality can be detected more accurately.

  FIG. 11 is a diagram illustrating light reflected from the clear layer and the inner layer of the object to be inspected. In the example shown in FIG. 11, the device under test 1101 includes an inner layer 1111 and a clear layer 1112 painted on the surface of the inner layer 1111.

  The illumination light emitted from the illumination device 120 is specularly reflected on the surface of the clear layer 1112, passes through the light 1121 toward the time correlation camera 110 and the inside of the clear layer 1112, and enters the inside of the inner layer 1111. A large number of reflections are repeated inside the inner layer 1111, resulting in diffused light that is emitted from the inner layer 1111 and the clear layer 1112, and is divided into light 1122 that travels to the time correlation camera 110. The light 1121 specularly reflected on the surface of the clear layer 1112 is referred to as a specular reflection component, and the light 1122 that enters the inner layer 1111 and is emitted from the inner layer 1111 and the clear layer 1112 as diffused light is also referred to as a diffuse reflection component.

  Thus, the time correlation camera 110 receives a combination of the regular reflection component and the diffuse reflection component.

  FIG. 12 is a diagram illustrating regular reflection components and diffuse reflection components received by the time correlation camera 110. As shown in FIG. 12, the regular reflection component 1201 changes in luminance value according to a cycle in which the luminance value of the stripe pattern irradiated by the illumination device 120 changes. The amplitude B of the regular reflection component 1201 is determined based on the regular reflectance of the object to be inspected or actual measurement.

  Then, the reflection component image generation unit 105 of the present embodiment divides the intensity image data into a regular reflection component 1201 and a diffuse reflection component 1202. Thereby, the inspection of the surface of the clear layer 1112 based on the regular reflection component 1201 and the inspection inside the inner layer 1111 of the clear layer 1112 based on the diffuse reflection component 1202 can be realized.

  In the example shown in FIG. 12, the equation f ′ (x, y, t) for obtaining the luminance received by the time correlation camera 110 is represented by the following equation (9). It is assumed that the light phase is kx.

G ′ (x, y, t) = A + B cos (ωt + kx) (9)

  G ′ (x, y, t) is a luminance value for each imaging interval, in other words, intensity image data when the clear layer is applied can be derived from the following equation (10).

  As shown in Expression (10), in the example shown in FIG. 12, intensity image data that is a total value of light intensity signals received by the time correlation camera 110 in one period of the fringe pattern irradiated by the illumination device 120 is: Become AT.

  Moreover, although Formula (8), Formula (3), and Formula (4) were cases in which the diffuse reflection component was not taken into consideration, the component amplitude-phase image generation unit 104 is represented by Formula (8), Formula (3), G ′ (x, y, t) shown in Expression (9) is substituted for G (x, y, z) in Expression (4). As a result, the equation (11) can be derived as the amplitude data F ′ (x, y) when the clear layer is applied.

  Note that C1 '(x, y) and C2' (x, y) shown in Expression (11) are derived from Expressions (12) and (13) below.

  Similarly, the amplitude-phase image generation unit 104 uses the equation (14) from the pixel values C1 ′ (x, y) and C2 ′ (x, y) to change the phase when the clear layer is applied. Each pixel value P ′ (x, y) of the image data can be derived.

  Then, the reflection component image generation unit 105 of the present embodiment inspects the object to be inspected based on the intensity image data AT and the amplitude image data F ′ (x, y) generated by the amplitude-phase image generation unit 104. Regular reflection image data reflected from the surface of the transparent layer of the surface and diffuse reflection image data reflected from the inside of the transparent layer of the inspection surface of the inspection object are generated.

  The amplitude image data corresponds to ½ of the changing regular reflection component. Therefore, the reflection component image generation unit 105 calculates image data of the regular reflection component by doubling the amplitude image data F ′ (x, y).

  As described above, intensity image data = regular reflection component image data + diffuse reflection component image data. Accordingly, the reflection component image generation unit 105 derives the diffuse reflection component image data from the intensity image data AT−regular reflection component image data 2 · F ′ (x, y).

  FIG. 13 is a diagram illustrating a concept for generating image data indicating diffuse reflection components (hereinafter referred to as diffuse reflection image data). As shown in FIG. 13, diffuse reflection image data can be generated by subtracting amplitude image data * 2 from intensity image data.

  And in this embodiment, the abnormality of the clear layer surface of a to-be-inspected object can be detected by detecting abnormality based on the image data of a regular reflection component. Furthermore, by detecting the abnormality based on the diffuse reflection image data, it is possible to detect the abnormality of the inner layer inside the clear layer of the object to be inspected.

  The abnormality detection processing unit 106 detects an abnormality of the inspection target surface using the amplitude image data generated by the reflection component image generation unit 105 and the diffuse reflection data.

  Next, the abnormality detection process based on the amplitude image data in the abnormality detection processing unit 106 of the present embodiment will be described. FIG. 14 is a flowchart showing a procedure of the processing in the abnormality detection processing unit 106 of the present embodiment.

  First, the abnormality detection processing unit 106 uses the amplitude value of light (a pixel value representing) stored in each pixel of the image data of the regular reflection component (twice the amplitude) as a reference (for example, the center). As a result, the average amplitude value of the N × N region is subtracted (step S1401), and the average difference image data of the amplitude is generated. The average difference image data of the amplitude corresponds to the gradient of the amplitude. The integer N is set to an appropriate value according to the embodiment.

  Next, the abnormality detection processing unit 106 performs mask processing using a predetermined amplitude threshold value on the average difference image data of the amplitude generated by subtraction (step S1402).

  Further, the abnormality detection processing unit 106 calculates a standard deviation for each pixel within the mask area of the average difference image data (step S1403). In the present embodiment, a method based on the standard deviation will be described. However, the method is not limited to the case where the standard deviation is used. For example, an average value or the like may be used.

  Then, the abnormality detection processing unit 106 detects a pixel having an average pixel value smaller than −4.5σ (σ: standard deviation) as a region having an abnormality (defect) (step S1404).

  By the processing procedure described above, an abnormality on the surface of the clear layer of the object to be inspected can be detected from the amplitude value (in other words, amplitude distribution) of each pixel.

  And the abnormality detection process part 106 of this embodiment detects abnormality about the inner surface of the clear layer of a to-be-inspected object using diffuse reflection image data. Note that the abnormality detection method using the diffuse reflection image data may be a method similar to a conventionally used method, and the description thereof is omitted.

  Further, the abnormality detection processing unit 106 according to the present embodiment is not limited to the abnormality detection method using the amplitude image data and the abnormality detection method using the diffusion image data. An abnormality detection method used, an abnormality detection method using a comparison result between intensity image data and other image data, or the like may be used.

  Next, an inspection process for an object to be inspected in the inspection system of the present embodiment will be described. FIG. 15 is a flowchart showing a procedure of the above-described processing in the inspection system of the present embodiment. It is assumed that the device under test 150 is already fixed to the arm 140 and arranged at the initial position of the inspection.

  The PC 100 of this embodiment outputs a fringe pattern for inspecting the object to be inspected to the illumination device 120 (step S1501).

  The illuminating device 120 stores the fringe pattern input from the PC 100 (step S1521). And the illuminating device 120 displays the stored fringe pattern so that it may change according to a time transition (step S1522). Note that the conditions under which the illuminating device 120 starts display are not limited when the fringe pattern is stored, and may be, for example, when the inspector performs a start operation on the illuminating device 120.

  Then, the control unit 103 of the PC 100 transmits a shooting start instruction to the time correlation camera 110 (step S1502).

  Next, the time correlation camera 110 starts imaging for an area including the inspection object 150 in accordance with the transmitted imaging start instruction (step S1511). Next, the control unit 240 of the time correlation camera 110 generates intensity image data and time correlation image data (step S1512). Then, the control unit 240 of the time correlation camera 110 outputs the intensity image data and the time correlation image data to the PC 100 (step S1513).

  The control unit 103 of the PC 100 receives the intensity image data and the time correlation image data (step S1503). Then, the amplitude-phase image generation unit 104 generates amplitude image data and phase image data from the received intensity image data and time correlation image data (step S1504).

  Then, the reflection component image generation unit 105 generates regular reflection component image data and diffuse reflection component image data based on the intensity image data and the amplitude image data (step S1505).

  Then, the abnormality detection processing unit 106 performs abnormality detection control of the object to be inspected based on the image data of the regular reflection component and the image data of the diffuse reflection component (step S1506). Then, the abnormality detection processing unit 106 outputs the abnormality detection result to a display device (not shown) included in the PC 100 (step S1507).

  As an output example of the abnormality detection result, the intensity image data is displayed, and the intensity image data area corresponding to the area where the abnormality is detected based on the image data of the regular reflection component and the image data of the diffuse reflection component It is conceivable to display the decoration so that the inspector can recognize the abnormality. Further, the output is not limited to visual output, and it may be output that an abnormality has been detected by voice or the like.

  The control unit 103 determines whether or not the inspection of the inspection object has been completed (step S1508). When it is determined that the inspection has not been completed (step S1508: No), the arm control unit 101 can photograph the surface of the inspection object to be inspected with the time correlation camera 110 according to a predetermined setting. In this way, arm movement control is performed (step S1509). After the arm movement control is completed, the control unit 103 transmits an imaging start instruction to the time correlation camera 110 again (step S1502).

  On the other hand, when it is determined that the inspection of the object to be inspected has ended (step S1508: Yes), the control unit 103 outputs an end instruction to the time correlation camera 110 (step S1510) and ends the process.

  Then, the time correlation camera 110 determines whether an end instruction has been received (step S1514). If an end instruction has not been received (step S1514: NO), the processing is performed again from step S1511. On the other hand, if an end instruction has been received (step S1514: YES), the process ends.

  Note that the inspecting process of the illumination device 120 may be performed by an inspector, or may be ended in accordance with an instruction from another configuration.

  In the present embodiment, the example in which the intensity image data generated using the time correlation camera 110 and the time correlation image data are generated has been described. However, the present invention is not limited to the use of the time correlation camera 110 for generating the intensity image data and the time correlation image data, and a time correlation camera that can be realized by analog processing or an operation equivalent thereto. An imaging system may be used. For example, image data generated by a normal digital still camera is output, and the information processing apparatus superimposes a reference signal using the image data generated by the digital still camera as frame image data, thereby generating time-correlated image data. The time correlation image data may be generated by using a digital camera that superimposes a reference signal on a light intensity signal in the image sensor.

(Modification 1)
In the first embodiment, the example in which the fringe pattern is moved in the x direction to detect an abnormality (defect) of the inspection object has been described. However, when an abnormality (defect) in which the normal distribution changes sharply in the y direction perpendicular to the x direction is generated in the inspection object, the fringe pattern is moved in the y direction rather than in the x direction. It may be easier to detect defects. Therefore, in the modification, an example will be described in which a fringe pattern moving in the x direction and a fringe pattern moving in the y direction are alternately switched.

  The illumination control unit 102 of this modification switches the fringe pattern output to the illumination device 120 at predetermined time intervals. Thereby, the illuminating device 120 outputs the some fringe pattern extended in the different direction with respect to one test object surface.

  FIG. 16 is a diagram illustrating a switching example of a fringe pattern output by the illumination control unit 102 of the present modification. In FIG. 16A, the illumination control unit 102 causes the stripe pattern displayed by the illumination device 120 to transition in the x direction. Thereafter, as illustrated in FIG. 16B, the illumination control unit 102 causes the stripe pattern displayed by the illumination device 120 to transition in the y direction.

  Then, the control unit 103 of the PC 100 performs abnormality detection based on the time correlation image data obtained from the stripe pattern irradiation of FIG. 16A, and is obtained from the stripe pattern irradiation of FIG. Abnormality detection is performed based on the time correlation image data.

  FIG. 17 is a diagram illustrating an example in which the illumination control unit 102 according to the present modification irradiates the surface including the abnormality (defect) 1601 with a fringe pattern. In the example shown in FIG. 17, an abnormality (defect) 1601 extends in the x direction. In this case, the illumination control unit 102 sets the fringe pattern to move in the y direction that intersects the x direction, in other words, in the direction that intersects the longitudinal direction of the abnormality (defect) 1601. With this setting, detection accuracy can be improved.

  FIG. 18 is a diagram showing the relationship between an abnormality (defect) 1701 and the stripe pattern on the screen 130 when the stripe pattern is changed in the y direction, in other words, in a direction orthogonal to the longitudinal direction of the defect 1601. As shown in FIG. 18, when an abnormality (defect) 1701 having a narrow width in the y direction and a longitudinal direction in the x direction intersecting the y direction occurs, the light emitted from the illumination device 120 is In the y direction that intersects the x direction, the cancellation of the amplitude of light increases. Therefore, the PC 100 can detect the abnormality (defect) from the amplitude image data corresponding to the stripe pattern moved in the y direction.

  In the inspection system of this modification, when the longitudinal direction of the defect generated in the inspection object is random, the fringe pattern is displayed in a plurality of directions (for example, the x direction and the y direction crossing the x direction). Thus, the defect can be detected regardless of the shape of the defect, and the detection accuracy of the abnormality (defect) can be improved. Further, by projecting a fringe pattern that matches the shape of the abnormality, the abnormality detection accuracy can be improved.

(Modification 2)
In addition, the first modification described above is not limited to the method of switching the fringe pattern when performing abnormality detection in the x direction and abnormality detection in the y direction. Therefore, in the second modification, an example in which the fringe pattern output from the illumination control unit 102 to the illumination device 120 is moved simultaneously in the x direction and the y direction will be described.

  FIG. 19 is a diagram illustrating an example of a fringe pattern output to the illumination device 120 by the illumination control unit 102 of the present modification. In the example shown in FIG. 19, the illumination control unit 102 moves the fringe pattern in the direction 1801.

  The fringe pattern shown in FIG. 19 includes a fringe pattern having one cycle 1802 in the x direction and includes a fringe pattern having one cycle 1803 in the y direction. That is, the stripe pattern shown in FIG. 19 has a plurality of stripes extending in intersecting directions having different widths. It is necessary to make the width of the stripe pattern in the x direction different from the width of the stripe pattern in the y direction. Thereby, when generating the time correlation image data corresponding to the x direction and the time correlation image data corresponding to the y direction, the corresponding reference signals can be made different. In addition, since the period (frequency) of the light intensity change by the stripe pattern only needs to be changed, the moving speed of the stripe pattern (stripe) may be changed instead of changing the width of the stripe.

  Then, the time correlation camera 110 generates time correlation image data corresponding to the stripe pattern in the x direction based on the reference signal corresponding to the stripe pattern in the x direction, and based on the reference signal corresponding to the stripe pattern in the y direction. Thus, time correlation image data corresponding to the stripe pattern in the y direction is generated. After that, the control unit 103 of the PC 100 detects an abnormality based on the time correlation image data corresponding to the stripe pattern in the x direction, and then detects an abnormality based on the time correlation image data corresponding to the stripe pattern in the y direction. I do. Thereby, in this modification, it becomes possible to detect regardless of the direction in which the defect occurs, and the detection accuracy of the abnormality (defect) can be improved.

  In the inspection system according to the above-described embodiment, the inspection of the surface of the clear layer using the image data of the specular reflection component and the inspection of the inside of the clear layer using the image data of the diffuse reflection component are realized. be able to.

  The inspection program executed by the PC 100 of the above-described embodiment is a file in an installable format or an executable format, and is a computer such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk). It is recorded on a readable recording medium and provided.

  Further, the inspection program executed by the PC 100 of the above-described embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. Further, the inspection program executed on the PC 100 according to the above-described embodiment may be provided or distributed via a network such as the Internet.

  Although some embodiments and modifications of the present invention have been described, these embodiments and modifications are presented as examples and are not intended to limit the scope of the invention. These novel embodiments and modifications can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and equivalents thereof.

  DESCRIPTION OF SYMBOLS 100 ... PC, 101 ... Arm control part, 102 ... Illumination control part, 103 ... Control part, 104 ... Amplitude-phase image generation part, 105 ... Reflection component image generation part, 106 ... Abnormality detection processing part, 110 ... Time correlation camera , 120 ... Illuminating device, 130 ... Screen, 140 ... Arm, 210 ... Optical system, 220 ... Image sensor, 230 ... Data buffer, 240 ... Control part, 241 ... Transfer part, 242 ... Reading part, 243 ... Superposition for intensity image 244 ... first multiplier 245 ... first correlated image superimposing unit 246 ... second multiplier 247 ... second correlated image superimposing unit 248 ... image output unit 250 ... reference signal Output part.

Claims (3)

A planar illumination unit that gives periodic temporal changes in the intensity of the light by moving the fringe pattern caused by the difference in the intensity of the light,
Light reflected from the inspection object during the imaging interval using a time-correlation camera or an imaging system that operates equivalently, in which the period of the fringe pattern generated by the illuminating unit coincides with the imaging interval A first image generating unit that generates intensity image data indicating a luminance value by a signal and amplitude image data calculated from a luminance change of an optical signal reflected from an inspection object during the imaging interval;
Regular reflection image data indicating a component reflected from the surface of the transparent layer to be inspected based on the intensity image data and the amplitude image data generated by the first image generation unit, and the transparent layer A diffuse reflection image data indicating a component reflected from the inside of the second image generation unit,
Inspection system equipped with. A detection unit for detecting an abnormality of the inspection object based on any one of the regular reflection image data and the diffuse reflection image data;
The inspection system according to claim 1, further comprising: A time correlation in which the period of the fringe pattern generated by a planar illumination unit that gives a periodic temporal change in the intensity of light coincides with the imaging interval by moving the fringe pattern caused by the difference in light intensity. Intensity image data indicating a luminance value by an optical signal reflected from the inspection object during the imaging interval and light reflected from the inspection object during the imaging interval using a camera or an imaging system that performs an equivalent operation Generating amplitude image data calculated from the luminance change of the signal,
Based on the intensity image data and the amplitude image data, specular reflection image data showing a component reflected from the surface of the transparent layer to be inspected, and diffuse reflection showing a component reflected from the inside of the transparent layer Generate image data,
Inspection method.