JP7079218B2 - Imaging device - Google Patents

Description

The present invention relates to an image pickup apparatus, for example, a technique for recognizing a three-dimensional shape of an object by imaging the object.

Patent Document 1 discloses a surface defect inspection method using a phase shift. In this method, a step of synthesizing light and dark patterns having different wavelengths (colors) after shifting the phases, a step of irradiating the combined illumination pattern on the inspection object, and an image of a reflection pattern from the inspection object are imaged. It also includes a step of decomposing the captured image for each wavelength and discriminating unevenness defects on the surface of the inspection object.

Patent Document 2 discloses a surface defect inspection method capable of detecting unevenness defects on the surface even if the inspection object has a laminated structure and the surface is a transparent glossy surface. In this method, first, a first original image is obtained by irradiating an inspection object with linear light in which light and dark are alternately present and photographing the reflected light with a line camera. Then, the image processing unit stores in advance the second original image obtained by photographing the reference object having no unevenness defect on the surface, and the inspection target object is based on the first and second original images. Detects unevenness defects existing on the surface of.

Japanese Unexamined Patent Publication No. 2013-113657 Japanese Unexamined Patent Publication No. 2009-222683

Machine vision technology is used in a wide range of applications such as inspection of defects, foreign matter, cracks, etc. inside or on the surface of products, recognition of characters printed on the surface of products, barcode reading, food inspection, etc., in factories. It has become an indispensable technology for automation. In recent years, the development of machine vision technology that utilizes three-dimensional measurement to grasp an object three-dimensionally has been progressing for the purpose of analyzing irregularities such as defects and markings in products.

As such a three-dimensional shape measurement technique, Patent Document 1 and Patent Document 2 disclose a phase shift illumination technique. When an object is irradiated with periodic illumination in which light and dark are alternately present, the pattern of transmitted light and reflected light is distorted on an uneven surface. The phase shift illumination technique is an imaging technique that emphasizes unevenness by synthesizing the distortion of light and darkness by image processing. However, for example, the surface of an object may be artificially provided with irregularities for the purpose of imparting optical performance to the product or improving design. Since the phase shift lighting technique is highly sensitive to unevenness, even slight irregularities provided artificially are emphasized. As a result, defects, markings, etc. that should be originally detected may be buried in the slight unevenness, which may make detection difficult.

The present invention has been made in view of the above, and one of the objects thereof is to provide an image pickup apparatus capable of obtaining an image in which unevenness in an object is selectively emphasized.

The above and other objects and novel features of the invention will become apparent from the description and accompanying drawings herein.

Among the inventions disclosed in the present application, the outline of a typical embodiment is as follows.

The image pickup apparatus according to one embodiment includes a lighting unit, a periodic pattern generation unit, an image pickup unit, and an image processing unit. The illumination unit irradiates the object with light having a corresponding luminance distribution for each of a plurality of periodic patterns having different phases. The periodic pattern generation unit generates a periodic pattern and controls the illumination unit so that the luminance distribution corresponds to each of the plurality of periodic patterns. The image pickup unit generates a plurality of captured image data by capturing a transmission pattern or a reflection pattern from an object for each of a plurality of periodic patterns. The image processing unit calculates the image data of the composite processed image representing the characteristics of the three-dimensional shape of the object by using the plurality of captured image data. Here, the periodic pattern is a pattern in which a completely dark portion having the minimum pixel value is generated in a finite area in the illumination unit, and the image processing unit receives a plurality of images when calculating the image data of the composite processed image. The minimum pixel value is extracted for each pixel for image data.

According to the image pickup apparatus of the one embodiment, it is possible to selectively emphasize unevenness in an object.

It is a schematic diagram which shows the whole structure example of the image pickup apparatus according to Embodiment 1 of this invention. (A) and (b) are schematic views showing an example of a periodic pattern generated by the periodic pattern generation unit of FIG. 1 and emitted from the illumination unit. It is a figure which shows the detailed example of the periodic pattern in FIG. 2 (a). (A) and (b) are diagrams illustrating an example of the processing content of the periodic pattern generation unit in FIG. It is a schematic diagram which shows the structural example of the image pickup part in FIG. It is a schematic diagram which shows the configuration example and the operation example around the image processing part in FIG. It is a flowchart which shows the detailed operation example around the image processing part of FIG. (A) and (b) are conceptual diagrams for explaining the difference in transmission pattern depending on the presence or absence of unevenness of the object in FIG. FIG. 6 is a conceptual diagram for explaining a difference in transmission pattern depending on the presence or absence of unevenness of an object when the periodic pattern is phase-shifted. FIG. 6 is a conceptual diagram showing how the transmission pattern changes according to the shape of the unevenness when a completely dark portion is provided. FIG. 10 is a conceptual diagram showing a change in the value of the image data of the composite processed image with respect to the inclination angle of the unevenness. It is a figure which shows an example of the periodic pattern used in the image pickup apparatus according to Embodiment 2 of this invention. It is a schematic diagram which shows the whole structure example of the image pickup apparatus according to Embodiment 2 of this invention. It is a schematic diagram which shows the whole structure example of the image pickup apparatus according to Embodiment 3 of this invention. It is a schematic diagram which shows the whole structure example of the image pickup apparatus different from FIG. It is a schematic diagram which shows the structural example of the image processing part of FIG. 1 in the image pickup apparatus according to Embodiment 4 of this invention. It is a schematic diagram which shows an example of the processing content of the 3rd calculation part in FIG. It is a schematic diagram which shows the structural example of a part of the image pickup apparatus according to Embodiment 5 of this invention. It is a figure explaining an example of the processing content of the image recognition part in FIG. It is a figure which shows the structural example of the object recognized by the image recognition part in FIG. It is a schematic diagram which shows the structural example of a part of the image pickup apparatus according to Embodiment 7 of this invention. FIG. 1 is a schematic diagram showing an example of how the periodic pattern applied to an object changes according to the distance between the illuminated portion and the object.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, in all the drawings for explaining the embodiment, the same members are designated by the same reference numerals in principle, and the repeated description thereof will be omitted. Further, the "light" described in the specification refers to an electromagnetic wave, and may be, for example, visible light, microwave, terahertz wave, infrared ray, ultraviolet ray, X-ray, or the like.

(Embodiment 1)
<< Outline of imaging device >>
FIG. 1 is a schematic view showing an overall configuration example of the image pickup apparatus according to the first embodiment of the present invention. As shown in FIG. 1, the image pickup apparatus 1 roughly includes a periodic pattern generation unit 10, an illumination unit 11, an adjustment unit 12, an image pickup unit 13, an image processing unit 14, and an output unit 15. It is equipped. In FIG. 1, for illustration purposes, optical paths are indicated by dashed arrows and electrical signals are indicated by solid arrows. The periodic pattern generation unit 10 and the image processing unit 14 are composed of, for example, an information processing device such as an electronic computer, a semiconductor device such as an FPGA (Field Programmable Gate Array) or a microcontroller, and the like.

The adjusting unit 12 is composed of, for example, a connector port or the like that enables reception of an electrical information signal from the outside. An external input device such as a keyboard or a mouse is connected to the adjusting unit 12. Further, an electric signal from an information processing device or a semiconductor device may be input to the adjusting unit 12. The output unit 15 is composed of, for example, a connector port or the like that enables an information signal from the image processing unit 14 to be transmitted to an external worker or the like. A display device, an audio output device, or the like is connected to the output unit 15. Further, the output unit 15 may output an electric signal directed to the information processing device or the semiconductor device.

The illumination unit 11 irradiates the object 2 with light having a corresponding luminance distribution for each of a plurality of periodic patterns having different phases. In the lighting unit 11, typically, like a display, pixels that output light to the outside are arranged in a two-dimensional manner, and the brightness value of the light output from each pixel is modulated by a control signal from the outside. It is a display device that emits modulated light. Further, the illumination unit 11 may be a display device such as a projector that switches the brightness of each pixel according to the reflection direction of the micromirrors arranged in two dimensions. The method of varying the brightness of light temporally and spatially is not particularly limited to the method using a display or a projector.

The periodic pattern generation unit 10 generates a reference periodic pattern, generates a plurality of periodic patterns having different phases from each other using the periodic pattern, and illuminates the luminance distribution so as to correspond to each of the plurality of periodic patterns. The unit 11 is controlled. The adjusting unit 12 receives various parameters used when generating the periodic pattern from the outside, and transmits a control signal representing the various parameters to the periodic pattern generation unit 10. The image pickup unit 13 generates a plurality of captured image data by capturing a transmission pattern or a reflection pattern from the object 2 for each of the plurality of periodic patterns. The image processing unit 14 calculates the image data of the composite processed image representing the characteristics of the three-dimensional shape of the object 2 by using the plurality of captured image data. The image data of the synthesized image is output to the outside via the output unit 15.

<< Details of periodic pattern generation unit and adjustment unit >>
2 (a) and 2 (b) are schematic views showing an example of a periodic pattern generated by the periodic pattern generation unit of FIG. 1 and emitted from the illumination unit. The periodic pattern spatially has a periodicity of period L, and there are a bright portion having a large pixel value and a dark portion having a small pixel value during one cycle. For example, FIG. 2A shows a periodic pattern having periodicity in one direction in space. Assuming that one direction having periodicity is the X direction for convenience and the direction perpendicular to it is the Y direction, in the periodic pattern as shown in FIG. 2A, the two-dimensional arrangement direction of the pixels coincides with the X and Y directions. It will be.

However, the direction having periodicity is not limited to this, and may be any direction with respect to the pixel arrangement. For example, the periodic pattern shown in FIG. 2B is a donut-shaped pattern, and the direction having periodicity is the radial direction. In the following, in order to make the explanation easier to understand, it is assumed that a periodic pattern as shown in FIG. 2A is used. At this time, the pixel value in the x-th pixel counted in the X direction and the y-th pixel counted in the Y direction from the origin position at the lower left is expressed as I (x, y). The periodic pattern generation unit 10 transmits the pixel value I (x, y) as an electrical control signal to the illumination unit 11, and the illumination unit 11 emits light having a brightness proportional to the pixel value I (x, y). Emit from each pixel.

FIG. 3 is a diagram showing a detailed example of the periodic pattern in FIG. 2 (a). FIG. 3 shows a diagram in which the distribution of the pixel values output from the illumination unit 11 in the periodic direction is enlarged by substantially one cycle. In the specification, a region having a high pixel value (for example, a pixel region) with respect to the average value of the maximum value and the minimum value of the pixel value is called a bright part, and a region having a low pixel value is called a dark part. Further, the region where the pixel value is maximum is referred to as a perfect bright portion 30, and the region where the pixel value is minimum is referred to as a complete dark portion 31. In other words, the completely bright portion 30 is a region where the brightness is the highest, and the perfect dark portion 31 is a region where the brightness is the lowest.

Here, as one of the main features of the image pickup apparatus 1 of FIG. 1, the periodic pattern generated by the periodic pattern generation unit 10 has a finite complete dark portion 31 in the illumination unit 11 as shown in FIG. It is a pattern that causes only the area. That is, for example, it is not a sine wave generally used in phase shift illumination technology, nor is it like a triangular wave, and a periodic pattern including a completely dark portion 31 is used.

The adjusting unit 12 in FIG. 1 outputs, for example, an external control signal representing the period L of the periodic pattern and the ratio D of the completely dark area 31 in the period L to the periodic pattern generation unit 10, thereby outputting the periodic pattern generation unit 10. 10 indicates the area of the complete dark portion 31 in the periodic pattern. Further, although the details will be described later, the adjusting unit 12 receives a control signal instructing the physical arrangement relationship between the lighting unit 11, the object 2, and the imaging unit 13, and uses this control signal for the physical arrangement relationship. The area of the complete dark portion 31 may be indirectly determined by outputting to various adjustable actuators (not shown). Alternatively, the adjusting unit 12 may adjust the area of the completely dark portion 31 by combining these two methods.

When the periodic pattern generation unit 10 receives a control signal regarding the ratio D of the period L and the complete dark unit 31 from the adjustment unit 12, the periodic pattern generation unit 10 internally determines the initial position _X0 of the periodic pattern as shown in FIG. 4 (b). Then, a control signal representing the pixel value I (x, y) is generated and transmitted to the illumination unit 11. 4 (a) and 4 (b) are diagrams illustrating an example of the processing content of the periodic pattern generation unit in FIG. 1. In FIG. 4A, it is assumed that I ₀ (x, y) is a pixel value for one cycle determined by the period L and the ratio D of the complete dark portion 31. The ratio of the fully bright portion 30 in the pixel value I ₀ (x, y) and the pixel value in the region other than the completely bright portion 30 and the completely dark portion 31 are arbitrary.

The initial position X ₀ of the periodic pattern is 1 when the pixel value I (x, y) to be displayed on the illumination unit 11 is generated based on the pixel value I ₀ (x, y) as in the equation (1). Reference position coordinates to start the cycle. That is, as shown in FIG. 4 (b), the periodic pattern generation unit 10 periodically sets the pixel value I ₀ (x, y) shown in FIG. 4 (a) as the reference position coordinates with the initial position X ₀ as the reference position coordinates. Pixel values I (x, y) are generated by arranging them. Further, the periodic pattern generation unit 10 sequentially shifts the initial position X ₀ to generate a plurality of periodic patterns having different phases from each other.

<< Details of the imaging unit >>
FIG. 5 is a schematic view showing a configuration example of the imaging unit in FIG. 1. For example, the transmitted light from the object 2 illuminated by the illumination unit 11 is incident on the image pickup unit 13. The object 2 is a medium having transparency in the wavelength range of the light emitted from the illumination unit 11, such as a film, a window, a glass cup, or a lens. When the wavelength range is in the terahertz band, it may be a food or rubber material having transparency in the terahertz band.

In FIG. 5, the imaging unit 13 includes an imaging optical unit 130 and a light detection unit 131. The imaging optical unit 130 forms an image of the transmitted light of the object 2 on the light receiving surface of the light detection unit 131 by using at least one optical element such as a lens or a concave mirror that refracts or reflects light to collect the light. .. For example, a telecentric optical system may be used as an optical system that suppresses a change in magnification due to blurring in the optical axis direction.

The photodetector 131 is a two-dimensional area sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. It is assumed that the light receiving surface of the photodetector 131 has a conjugate relationship with a predetermined surface on which the unevenness of the object 2 is to be observed. For example, when it is desired to observe swelling due to wrinkles or bubbles on the film surface, the imaging optical unit 130 may be arranged so that the film surface is a conjugated surface with respect to the light receiving surface.

In addition to the imaging optical unit 130, a wavelength-selective filter such as a bandpass filter may be provided in the path of the transmitted light to have a role of removing external light other than the desired wavelength. Further, for the purpose of protecting dust and dirt from entering the imaging optical unit 130, a window material or the like may be provided in the path of transmitted light. The light detection unit 131 generates a pixel value for each pixel by converting the light illuminance distribution on the light receiving surface into a two-dimensional electric signal, and transmits the pixel value as image pickup image data to the image processing unit 14. Here, in order to distinguish the image value output from the imaging unit 13 from the pixel value I (x, y) in the irradiation unit 11, it is referred to as I'(x, y). The image value I'(x, y) is counted in the xth and Y directions in the light receiving surface of the photodetector 131 in the X direction from the end, as in the case of the pixel value I (x, y). Represents the pixel value in the y-th pixel.

<< Details around the image processing section >>
FIG. 6 is a schematic view showing a configuration example and an operation example around the image processing unit in FIG. 1. The image processing unit 14 of FIG. 6 includes an image pickup control unit 140, a first storage medium 141, a first calculation unit 142, and a second calculation unit 143. As a premise for performing image processing, the periodic pattern generation unit 10 generates a plurality of periodic patterns while sequentially shifting the initial positions _X0 . The illumination unit 11 emits light corresponding to each periodic pattern, and the imaging unit 13 generates one captured image data (frame data) for one periodic pattern to form a plurality of periodic patterns. Generate a plurality of corresponding captured image data. At this time, the periodic pattern generation unit 10 transmits a control signal indicating that the illumination in the illumination unit 11 has started to the image pickup control unit 140, and the image pickup control unit 140 responds to the corresponding image pickup. Start importing image data.

Specifically, the image pickup control unit 140 acquires the pixel value _Ik '(x, y) which is the kth image pickup image data (kth frame data) corresponding to the kth period pattern, and the pixel value is the pixel value. I _k '(x, y) is stored in the first storage medium 141. As the first storage medium 141, a volatile memory such as a register included in the image processing unit 14 may be used, or a non-volatile memory arranged outside may be used. Further, the zero point of the pixel value I _k '(x, y) shall be calibrated using the pixel value of the completely dark portion 31 in the transmitted light image captured when the object 2 is not arranged.

When the storage of the k-th frame data in the first storage medium 141 is completed, the image pickup control unit 140 transmits a control signal indicating the completion to the periodic pattern generation unit 10. In order to generate the (k + 1) th periodic pattern, which is the next periodic pattern, the periodic pattern generation unit 10 shifts the initial position X ₀ by a predetermined value. The initial position X ₀ is controlled based on “X ₀ = L × k / N”, for example, in the range from 0 to the period L, and as a result, the phase of the period pattern is sequentially in the range of 0 to 2 × π. shift. Here, N represents the number of periodic patterns (in other words, the number of frames). Here, the phase shift is performed by changing the initial position X ₀ linearly with respect to k, but the phase shift method is not particularly limited to this.

When the image pickup control unit 140 completes the acquisition (storing in the first storage medium 141) of the frame data (k = 0, 1, ..., N-1) for one set in this way, the first calculation is performed. A control signal indicating acquisition completion is transmitted to unit 142. In response to this, the first calculation unit 142 extracts the minimum pixel value and the maximum pixel value for each pixel for all the frame data stored in the first storage medium 141, and the minimum pixel value. The image data I _Min '(x, y) of the minimum value image composed of and the image data I _Max '(x, y) of the maximum value image composed of the maximum pixel values are calculated. That is, as shown in the equation (2), the maximum value image and the minimum value image have the maximum value and the minimum value among all frame data (k = 0, 1, ..., N-1) for each pixel. It is an image calculated and rearranged in two dimensions.

Next, the second arithmetic processing unit 143 calculates the ratio of the difference between the pixel values of the maximum value image and the minimum value image to the sum of the pixel values of the maximum value image and the minimum value image for each pixel of the maximum value image and the minimum value image. Calculate image data. Specifically, the second arithmetic processing unit 143 exchanges the image data I _Max '(x, y) of the maximum value image and the image data I _Min '(x, y) of the minimum value image of the equation (2). The image data Γ (x, y) of the synthesized image is calculated based on the equation (3). Equation (3) is known as a contrast calculation equation.

The image processing unit 14 uses the plurality of captured image data (one set of frame data (k = 0, 1, ..., N-1)) obtained from the imaging unit 13 in this way to form a composite processed image. Image data Γ (x, y) is calculated. The image data Γ (x, y) of the synthesized image is data representing the characteristics of the three-dimensional shape of the object 2, although the details will be described later. The image data Γ (x, y) of the composite processed image is output from the output unit 15 as an image information signal. When the second arithmetic processing unit 143 completes the output of the image data Γ (x, y), the second arithmetic processing unit 143 transmits a control signal instructing the start of imaging for the next set to the imaging control unit 140.

FIG. 7 is a flowchart showing a detailed operation example around the image processing unit of FIG. First, in step S702, the period pattern generation unit 10 distributes the pixel values I ₀ (x, y) within one period L based on the parameters related to the ratio D between the period L and the completely dark part input from the adjustment unit 12. To decide. Subsequently, in step S703, the periodic pattern generation unit 10 generates the pixel value I (x, y) of the periodic pattern by determining the initial position X ₀ based on the current frame, and the pixel value I (x, y). ) Is transmitted to the lighting unit 11 as a control signal. The illumination unit 11 irradiates the object 2 with light having a luminance distribution based on the pixel value I (x, y) for a predetermined time. Further, the periodic pattern generation unit 10 transmits the information of the initial position X ₀ to the image processing unit 14 in order to cause the image processing unit 14 to associate the initial position X ₀ with the frame.

Next, in step S704, the image pickup unit 13 captures an image by detecting the intensity of the transmitted light from the object 2 for each pixel, and transmits the image capture data of the image to the image processing unit 14. Subsequently, in step S705, the image processing unit (specifically, the image pickup control unit 140) stores the image pickup data from the image pickup unit 13 in the first storage medium 141, and when the storage is completed, a control signal indicating to that effect. Is transmitted to the periodic pattern generation unit 10 to instruct the periodic pattern generation unit 10 to generate a periodic pattern at the next initial position _X0 .

In the processes of steps S703 to S705, in step S706, the periodic pattern generation unit 10 generates the final (N-1st) periodic pattern, and the image processing unit 14 acquires the final frame data corresponding to the final (N-1st) periodic pattern. Will be done. When the image processing unit 14 acquires the final frame data, in step S707, the image processing unit 14 uses one set of frame data (k = 0, 1, ..., N-1) stored in the first storage medium 141. , The image data Γ (x, y) of the composite processed image is calculated based on the equation (2) and the equation (3), and it is transmitted to the output unit 15. By exchanging the determination process of step S706 and the composition process of step S707, the image processing unit 14 calculates the maximum value / minimum value comparison in the equation (2) for the sequentially received image pickup data. Line processing may be performed.

<< Details of composite processed image >>
As described above, the image data Γ (x, y) of the composite processed image is data representing the characteristics of the three-dimensional shape of the object 2. Specifically, the image data Γ (x, y) is data in which the unevenness of the object 2 is selectively emphasized according to the shape of the unevenness (specifically, the inclination angle). This will be described in detail. 8 (a) and 8 (b) are conceptual diagrams for explaining the difference in the transmission pattern depending on the presence or absence of unevenness of the object in FIG. FIG. 9 is a conceptual diagram for explaining the difference in the transmission pattern depending on the presence or absence of unevenness of the object when the periodic pattern is phase-shifted in FIG.

As shown in FIG. 8B, in the unevenness of the object 2, the light rays incident in the oblique direction from the illumination unit 11 are refracted, and the refracted light rays are incident on the image pickup unit 13. Since the boundary surface is inclined in the unevenness as compared with the flat boundary surface without unevenness, a group of light rays emitted from a wide range in the illumination unit 11 is incident on a predetermined pixel in the image pickup unit 13 due to the refraction of the light rays. As a result, in the image acquired by the image pickup unit 13, the illuminance distribution in the illumination unit 11 is averaged in the pixels in which the light rays transmitted through the unevenness are imaged. In other words, since the light rays from the bright part and the light rays from the dark part of the periodic pattern can be incident on the pixel at a ratio according to the shape of the unevenness, they cancel each other out and the illuminance distribution is averaged.

As a result, even if the phase of the periodic pattern is shifted within the range of 0 to 2 × π, the pixel value of the pixel corresponding to the unevenness does not change significantly in each imaging data (frame data). Therefore, as shown in FIG. 9, the difference between the maximum value and the minimum value that can be taken when the phase is shifted is smaller in the pixel corresponding to the unevenness than in the pixel in which the light ray transmitted through the flat boundary surface is imaged. Since the image data Γ (x, y) of the composite processed image defined by the equation (3) is proportional to the value obtained by normalizing the difference between the minimum value and the maximum value by the average value, the sharpness due to the unevenness is increased. The degree is quantified. Therefore, the image data Γ (x, y) of one set of composite processed images obtained by the image processing unit 14 is data that reflects the refraction angle of light rays (in other words, the inclination angle of the unevenness) in the unevenness of the surface. ..

Next, it will be described that the image data Γ (x, y) of the composite processed image can selectively emphasize the unevenness according to the shape of the unevenness (specifically, the inclination angle). The shape (inclination angle) of the unevenness to be emphasized can be adjusted by the ratio D of the completely dark portion 31 in the periodic pattern generation unit 10. FIG. 10 is a conceptual diagram showing how the transmission pattern changes according to the shape of the unevenness in the case where the completely dark portion is provided in FIG. If the unevenness (specifically, the inclination angle of the unevenness) is comparatively gentle, the refraction angle of the transmitted light beam is small, so that the sharpness of the transmitted pattern is not so low as compared with the periodic pattern. At this time, since the complete dark portion 31 remains even if the sharpness is lowered due to the provision of the complete dark portion 31, the minimum value when the phase is shifted remains zero. As a result, the value of the image data Γ (x, y) defined by the equation (3) is 1, which is the same as the flat boundary surface without unevenness.

On the other hand, when the unevenness is steep, the refraction angle of the transmitted light beam is large, so that the sharpness of the transmitted pattern is significantly lower than that of the periodic pattern. As a result, the completely dark portion 31 disappears, and the minimum value when the phase is shifted becomes a finite value. As a result, the value of the image data Γ (x, y) defined in the equation (3) becomes smaller than 1, and the sharpness of the transmission pattern is reflected as the value of the image data Γ (x, y). Become.

FIG. 11 is a conceptual diagram showing a change in the value of the image data of the composite processed image with respect to the inclination angle of the unevenness in FIG. By providing the completely dark portion 31, as shown in FIG. 11, if the unevenness is gentle, the value of the image data Γ (x, y) of the composite processed image is kept at 1, so that there is no sensitivity to the unevenness. It is possible to provide a dead zone. On the other hand, when the inclination angle of the unevenness exceeds a certain threshold value, the completely dark portion 31 disappears due to the decrease in sharpness, the value of the image data Γ (x, y) becomes smaller than 1, and the sensitivity to the unevenness is generated.

The threshold value of the inclination angle of the unevenness can be variably set by adjusting the period L of the periodic pattern and the ratio D of the completely dark portion via the adjusting unit 12. Specifically, as the ratio D of the completely dark portion 31 is increased, the value of the image data Γ (x, y) becomes smaller than 1 only in the pixels that image the unevenness of the steeper tilt angle. Therefore, for example, when the ratio D of the completely dark portion 31 is sufficiently increased, only the unevenness of the steep inclination angle is emphasized, and a composite processed image in which the unevenness of the gentle inclination angle is not visible can be obtained. Will be possible. As described above, when the method of the first embodiment is used, only the unevenness having an inclination of the threshold value or more is emphasized, so that there is an advantage that an image having higher contrast can be obtained.

Here, the equations (2) and (3) are used as examples of the arithmetic methods in the image processing unit 14, but the arithmetic methods are not necessarily limited to the equations (2) and (3). That is, as can be seen from FIGS. 10 and 11, in the method of the first embodiment, roughly, the presence / absence of the remaining complete dark portion 31 and the image data of the minimum value image of the formula (2) I _Min '(x, y). ) Is used to determine the shape (tilt angle) of the unevenness. Therefore, when calculating the image data Γ (x, y) of the composite processed image, the image processing unit 14 targets at least a plurality of captured image data in the first storage medium 141 and minimizes pixels for each pixel. The process of extracting the value (I _Min '(x, y)) may be performed.

<< How to adjust the threshold >>
As a method of adjusting the threshold value in FIG. 11, as described above, an image adjustment for adjusting the period L of the periodic pattern generated by the periodic pattern generation unit 10 and the ratio D of the completely dark portion (in other words, the area of the completely dark portion in the periodic pattern) is adjusted. The method can be mentioned. Further, as another adjustment method, there is an arrangement adjustment method for adjusting the physical arrangement relationship between the illumination unit 11, the object 2, and the image pickup unit 13. FIG. 22 is a schematic diagram showing an example of how the periodic pattern irradiated to the object changes according to the distance between the illuminated portion and the object in FIG. 1. The adjustment of the distance between the illumination unit 11 and the object 2 is performed by outputting the corresponding control signal from the adjustment unit 12 to various actuators, as described above.

As shown in FIG. 22, for example, as the distance between the illumination unit 11 and the object 2 is increased, the sharpness of the periodic pattern on the actual irradiation surface of the object 2 is generated by the periodic pattern generation unit 10. It will be lower than the periodic pattern to be performed. As the sharpness of the periodic pattern decreases, the ratio D of the completely dark portion 31 also decreases accordingly. In this way, it is also possible to adjust the area of the completely dark portion 31 in the periodic pattern by adjusting the distance between the illumination unit 11 and the object 2. Further, by adjusting the distance between the object 2 and the image pickup unit 13, the area of the completely dark portion 31 can be adjusted by the same mechanism.

Therefore, it is possible to use the arrangement adjustment method instead of the image adjustment method, and it is also possible to use the image adjustment method and the arrangement adjustment method together. For example, in the image adjustment method, the setting resolution of the ratio (area) D of the completely dark portion 31 depends on the number of pixels of the illumination unit 11 of the display or the like, so that the ratio D and eventually the setting resolution of the threshold may be insufficient in some cases. There is. Therefore, for example, if the ratio D is adjusted by the image adjustment method to some extent, the adjusted image is used, and the distance is linearly adjusted by using the arrangement adjustment method, the threshold setting resolution can be increased. become.

<< Main effect of Embodiment 1 >>
As described above, by using the image pickup apparatus of the first embodiment, it is possible to obtain an image in which unevenness in an object is selectively emphasized. Alternatively, it is possible to obtain an image in which unevenness that is not desired to be seen is invisible. As a result of these, for example, when the image pickup device is applied to a surface defect inspection device, it is possible to distinguish between the artificial unevenness formed on the surface of the object and the unevenness associated with the defect by the difference in the inclination angle. This makes it possible to improve inspection accuracy.

Although the method of adjusting the area of the completely dark portion 31 is used here, it is also possible to use the method of adjusting the area of the completely bright portion 30 instead. In this case, the periodic pattern is a pattern in which the illumination unit 11 is provided with a completely bright portion having the maximum pixel value by a finite area. Further, when calculating the image data Γ (x, y) of the composite processed image, the image processing unit 14 extracts the maximum pixel value at least for each pixel of a plurality of captured image data from the imaging unit 13. do it. However, the pixel value of the completely bright portion 30 is more likely to change according to the fluctuation of the illuminance of the illumination portion 11 than the pixel value of the completely dark portion 31. As a result, the accuracy of the threshold value decreases according to the fluctuation of the illuminance. Therefore, from such a viewpoint, it is desirable to use a method of adjusting the area of the completely dark portion 31.

(Embodiment 2)
<< Details of cycle pattern >>
FIG. 12 is a diagram showing an example of a periodic pattern used in the image pickup apparatus according to the second embodiment of the present invention. Unlike the case of FIG. 3, the periodic pattern shown in FIG. 12 is a rectangular wavy pattern in which the bright part and the dark part are composed of only the completely bright part 30 and the completely dark part 31, respectively. When the same configuration as in the case of the first embodiment is used as the image pickup apparatus, the rectangular wavy pattern is generated by the periodic pattern generation unit 10 of FIG. In this case, since the periodic pattern generation unit 10 only needs to generate a periodic pattern consisting of binary pixel values I (x, y), simplification of processing and the like can be realized.

<< Outline of imaging device >>
When generating the periodic pattern as shown in FIG. 12, not only the configuration as shown in FIG. 1 but also the configuration as shown in FIG. 13 can be used. FIG. 13 is a schematic view showing an overall configuration example of the image pickup apparatus according to the second embodiment of the present invention. The image pickup apparatus 1 shown in FIG. 13 includes a movable stage control unit 16 and an illumination unit 11 different from FIG. 1 in addition to the image pickup unit 13, the image processing unit 14, and the output unit 15 as in the case of FIG. ..

The illumination unit 11 of FIG. 13 includes a uniform illumination unit 110 that emits uniform light, and a light-shielding unit 111 that is arranged between the uniform illumination unit 110 and the object 2. The light-shielding portion 111 includes an opening formed periodically and a closed opening provided between the openings to generate a periodic pattern consisting of light that passes through the opening and is blocked by the closed opening, and targets the periodic pattern. Irradiate the object 2. As a result, the periodic pattern is provided with a completely dark portion 31 having the lowest brightness in a finite area based on the closure. The movable stage control unit 16 controls the object 2 to be irradiated with a plurality of periodic patterns having different phases from each other by moving the light-shielding unit 111 in the periodic direction.

The uniform illumination unit 110 may be a display, a projector, or the like as in the case of FIG. 1, but unlike the case of FIG. 1, an illuminance distribution is not required. Therefore, for example, a lamp, an LED (light emitted diode), a laser, or the like can be used. The light emitted from the light source may be uniformly illuminated by using an appropriate optical system. The light-shielding portion 111 is, for example, a plate having a rectangular opening, a film coated with ink, or the like, and is a medium in which a region that absorbs or reflects light and a region that transmits light are periodically arranged. The period L of the light-shielding portion 111 and the ratio D of the completely dark portion 31, that is, the region that absorbs or reflects are adjusted in advance so as to have a desired sensitivity. Sensitivity adjustment is performed by exchanging the light-shielding portion 111.

Although not shown, the light-shielding unit 111 is equipped with an actuator such as an electric stage that moves the light-shielding unit 111 by a certain distance in one direction based on a control signal from the movable stage control unit 16. When the light-shielding unit 111 moves a certain distance, the movable stage control unit 16 obtains an image of a control signal for capturing and storing image pickup data, as in the case of the periodic pattern generation unit 10 described with reference to FIGS. 6 and 7. It is transmitted to the processing unit 14. Further, when the movable stage control unit 16 receives the control signal indicating that the storage is completed from the image processing unit 14, the movable stage control unit 16 moves the light-shielding unit 111 again by a certain distance.

<< Main effect of Embodiment 2 >>
As described above, by using the image pickup apparatus of the second embodiment, the following effects can be further obtained in addition to the various effects described in the first embodiment. First, by setting the periodic pattern for illuminating the object 2 as a rectangular wavy signal which is a binary signal, the number of control signals to the illumination unit 11 can be reduced and control can be facilitated. Further, a simple configuration such as a uniform lighting unit 110 can be applied to the lighting unit 11. Further, by physically shifting the periodic pattern using the light-shielding unit 111, the phase shift can be performed in steps smaller than the pixel size in the display or the like, so that the composite processed image calculated by the image processing unit 14 can be performed. It becomes possible to improve the accuracy of the image data Γ (x, y) of. Specifically, it is possible to reduce the error of the values of the image data I _Min '(x, y) and I _Max '(x, y) of the minimum value image and the maximum value image.

(Embodiment 3)
<< Outline of imaging device >>
FIG. 14 is a schematic view showing an overall configuration example of the image pickup apparatus according to the third embodiment of the present invention. FIG. 15 is a schematic view showing an overall configuration example of an image pickup apparatus different from that of FIG. The image pickup apparatus of FIG. 1 images the transmission pattern from the object 2, but the image pickup device of the third embodiment captures the reflection pattern from the object 2. This makes it possible to observe irregularities on the surface of the object 2 that absorbs light in a desired wavelength range.

The image pickup apparatus 1 of FIGS. 14 and 15 both include a periodic pattern generation unit 10, an illumination unit 11, an adjustment unit 12, an image pickup unit 13, an image processing unit 14, and an output unit 15 as in the case of FIG. In this case, instead of the periodic pattern generation unit 10 and the illumination unit 11 of FIG. 1, the uniform illumination unit 110, the light shielding unit 111, and the movable stage control unit 16 shown in FIG. 13 may be used. Unlike the first and second embodiments, the object 2 is a medium that is not transparent but reflective in the wavelength range of the light emitted from the illumination unit 11. For example, when the wavelength range is in the visible range, products with characters engraved on the surface, metal or ceramic wall surfaces having irregularities such as cracks and scratches, etc. can be considered as observation targets. The periodic pattern emitted from the illumination unit 11 may be incident on the object 2 from an oblique direction as shown in FIG. 14, or a beam splitter 17 is arranged as shown in FIG. 15 to arrange a coaxial epi-illumination type illumination. May be.

<< Main effect of Embodiment 3 >>
As described above, by using the image pickup apparatus of the third embodiment, it becomes possible to obtain the various effects described in the first embodiment and the like even on the medium in which the object 2 is reflective. For example, by using the image pickup apparatus of the third embodiment, it is possible to selectively emphasize the unevenness of the marking and the ink, so that it is possible to make the printing that is difficult to read very finely easy to read. Alternatively, it can be applied to the purpose of finding fine cracks in large-scale equipment used for industrial applications and infrastructure applications.

(Embodiment 4)
<< Details of the image processing unit >>
FIG. 16 is a schematic view showing a configuration example of the image processing unit of FIG. 1 in the image pickup apparatus according to the fourth embodiment of the present invention. The image processing unit 14 shown in FIG. 16 has a third calculation unit 144 in addition to the image pickup control unit 140, the first storage medium 141, the first calculation unit 142, and the second calculation unit 143 shown in FIG. And a second storage medium 145. For example, the second calculation unit 143 calculates the first image data and the second image data as the image data Γ (x, y) of the composite processed image, and stores these in the second storage medium 145. .. For example, the first image data is data obtained when the area of the completely dark part in the periodic pattern is the first area, and the second image data is the data obtained when the area of the completely dark part in the periodic pattern is the first area. It is the data obtained when it is a second area different from.

The third calculation unit 144 receives a control signal from the second calculation unit 143 to the effect that the storage of the image data is completed, and the third calculation unit 144 and the first image data stored in the second storage medium 145 and having different thresholds from each other. The image data of another composite processed image is calculated using the second image data. As a specific example, the third calculation unit 144 calculates the difference between the first image data and the second image data for each pixel. FIG. 17 is a schematic diagram showing an example of the processing content of the third arithmetic unit in FIG.

As shown in FIG. 17, for example, by calculating the difference between the first image data and the second image data, the image data Γ (x, y) of the synthesized image having sensitivity in a certain threshold range is calculated. It will be possible to do. As a result, for example, it is possible to calculate image data in which unevenness having an inclination angle equal to or higher than a certain threshold value is emphasized, but image data in which unevenness having an inclination angle equal to or higher than the first threshold value and equal to or lower than the second threshold value is emphasized.

<< Main effect of Embodiment 4 >>
As described above, by using the image pickup apparatus of the fourth embodiment, in addition to the various effects described in the first embodiment and the like, data with further improved selectivity for the shape of the unevenness (specifically, the inclination angle) can be obtained. Will be possible. As a result, for example, when the image pickup device is applied to a surface defect inspection device, the inspection accuracy can be further improved.

(Embodiment 5)
<< Outline of imaging device >>
FIG. 18 is a schematic view showing a configuration example of a part of the image pickup apparatus according to the fifth embodiment of the present invention. The image pickup device 1 shown in FIG. 18 is, for example, a periodic pattern generation unit 10, an illumination unit 11 (not shown), an adjustment unit 12, an image pickup unit 13 (not shown), an image processing unit 14, and an output unit similar to those in FIG. In addition to 15, an image recognition unit 18 is provided. The image recognition unit 18 causes the image processing unit 14 to calculate the image data Γ (x, y) of the composite processed image while changing the area of the completely dark portion (threshold in FIG. 11) via the adjustment unit 12, and the image concerned. The three-dimensional shape of the object 2 is recognized based on the data Γ (x, y). Strictly speaking, the information recognized by the image recognition unit 18 is the information on the refraction angle of the light beam, but it is substantially the inclination angle of the unevenness in the object 2.

FIG. 19 is a diagram illustrating an example of the processing content of the image recognition unit in FIG. The image recognition unit 18 transmits a predetermined parameter to the adjustment unit 12, and causes the image processing unit 14 to calculate the image data Γ (x, y) of the composite processed image. At this time, the image recognition unit 18 can modulate the threshold value of the inclination angle of the unevenness in the image data Γ (x, y) as shown in FIG. 19A by sequentially changing the parameters. be. During this modulation, as shown in FIG. 19B, the value of the image data Γ (x, y) changes rapidly at the moment when the inclination angle of the unevenness and the threshold value match, so the change in this value is captured. This makes it possible for the image recognition unit 18 to identify the inclination angle of the unevenness of each pixel.

As a specific example, in a state where the ratio D of the completely dark portion 31 is sufficiently large based on the parameters in the adjustment unit 12, the threshold value is sufficiently large, and as can be seen from FIG. 11, even a pixel that is imaging a steep unevenness is gentle. The value of the image data Γ (x, y) is 1 regardless of whether the pixel is imaged with unevenness or the pixel without unevenness. When the parameter is changed and the ratio D of the completed dark portion 31 is reduced, the threshold value becomes smaller, and the value of the image data Γ (x, y) is higher than 1 in the pixel that is imaging the steep unevenness. Also becomes smaller. At this time, the value of the image data Γ (x, y) is maintained at 1 in the pixels that image the gentle unevenness and the pixels that image the image without the unevenness.

When the parameter is further changed and the ratio D of the completed dark portion 31 is further reduced, the value of the image data Γ (x, y) becomes smaller than 1 even in the pixel that is capturing the gentle unevenness. At this time, in the pixel that is capturing the steep unevenness, the value of the image data Γ (x, y) becomes smaller than 1 and then close to 0, and the image that captures the image without unevenness is captured. Then, the value of the image data Γ (x, y) still maintains 1.

In this way, the image recognition unit 18 can recognize the parameter value at the time when the value of the image data Γ (x, y) becomes smaller than 1 for each pixel, and by extension, for each pixel, The inclination angle of the unevenness can be recognized. In this example, the threshold value was gradually reduced, but when the threshold value was gradually increased, the value of the image data Γ (x, y) became 1 in the image recognition unit 18. Or, by recognizing the parameter value at the time when the value of the image data Γ (x, y) suddenly increases, the inclination angle of the unevenness can be recognized for each pixel.

Further, the image recognition unit 18 includes a visible / invisible classification unit that classifies visible / invisible using reference data previously stored in an external storage medium or an internal storage medium, and the visible / invisible classification unit is used. The type of unevenness may be identified. Specifically, the reference data is, for example, data that defines a range of tilt angles to be visualized and a range of tilt angles to be invisible. As a specific example, for the purpose of detecting wrinkles on the film surface separately from artificially formed irregularities, reference data is used as an inclination angle for invisible the previously known inclination angle of artificially provided irregularities. Keep in.

As described above, the image recognition unit 18 can generate image data representing the correspondence between each pixel and the inclination angle of the unevenness in each pixel. The inclination angle of the unevenness at this time is not information such as a certain value or more, but information of a specific value or information of the specific value ± Δ according to the resolution (± Δ) of the parameter. The image recognition unit 18 can generate image data in which only the tilt angle to be visualized is left by removing the portion of the pixel having the tilt angle to be invisible based on the reference data for such image data. As a result, it is possible to generate image data in which only wrinkles remain.

Further, the reference data may include information such as the size, area, and distribution of the unevenness in addition to the classification information of the inclination angle. FIG. 20 is a diagram showing a structural example of an object recognized by the image recognition unit in FIG. For example, consider a situation in which, as shown in FIG. 20, it is desired to detect only randomly attached defects 22a and 22b in an object 2 in which a periodic dent 21 is artificially formed on the surface. At this time, if the inclination angle of the recess 21 and the inclination angle of the defect 22a are clearly different as in the defect 22a, it is possible to distinguish between the two only by the classification information of the inclination angle. However, in some cases, there may be a defect 22b having an inclination angle equivalent to the inclination angle of the recess 21, such as the defect 22b. In this case, the defect 22b may be erroneously detected as an artificial dent 21.

Therefore, as an example of a method for distinguishing the dent 21 and the defect 22b in such a case, the area or distribution of the artificially formed dent 21 is defined as an evaluation function on the reference data. For example, the area occupied by the pixel corresponding to the defect 22b and the area occupied by the pixel corresponding to the recess 21 are clearly different. Therefore, the image recognition unit 18 can distinguish between the dent 21 and the defect 22b by using such an evaluation function on the reference data. Further, the image recognition unit 18 is not limited to the physical area, and may make a distinction by using an evaluation function focusing on periodicity or discontinuity, or an evaluation function based on a combination thereof. When the discrimination process is completed, the image recognition unit 18 outputs the discrimination result to the output unit 15.

<< Main effect of embodiment 5 >>
As described above, by using the image pickup apparatus of the fifth embodiment, in addition to the various effects described in the first embodiment and the like, the shape of the unevenness in the entire image can be recognized in detail, and further, the reference data determined in advance. Based on, it is possible to extract unevenness having a specific inclination angle from the entire image.

(Embodiment 6)
In the above-described fifth embodiment, the description has been made mainly on the assumption that a defect existing on the surface of the object 2 is detected, but in the sixth embodiment, a predetermined shape previously formed on the object 2 has been described. The method of recognizing the pattern of is described. The configuration of the image pickup apparatus according to the sixth embodiment is the same as the configuration shown in FIG. In the sixth embodiment, the image recognition unit 18 of FIG. 18 generally indicates the area of the completely dark portion 31 via the adjustment unit 12 to the image processing unit 14 and gives the image processing unit 14 the image data Γ (x) of the composite processed image. , Y) is calculated, and a predetermined pattern previously formed on the object 2 is recognized based on the image data Γ (x, y).

When the completely dark portion 31 is provided, for example, as shown in FIG. 11, the value of the image data Γ (x, y) sharply decreases only in the unevenness having an inclination angle equal to or higher than a certain threshold value. This means that it is possible to optically perform the same processing as the correction processing for increasing the contrast using the tone curve performed by the software. Therefore, the image recognition unit 18 performs pattern recognition by utilizing the optical correction process.

Specifically, the image recognition unit 18 can learn, for example, the image data Γ (x, y) of the composite processed image corresponding to the reference pattern (for example, characters, numbers, etc.) as teacher data in advance. It is provided with a pattern recognition unit that performs pattern recognition. The image recognition unit 18 performs pattern recognition processing using the pattern recognition unit based on the image data Γ (x, y) of the composite processed image transmitted from the image processing unit 14. At this time, the pattern recognition unit may perform pattern recognition processing using the composite processed image itself, or may perform pattern recognition processing after adding another image processing on the software. After the pattern recognition process, the image recognition unit 18 converts the composite processed image into a sequence of the corresponding pattern class and outputs it to the output unit 15 as compressed signal information such as text data and a binary signal.

As a specific example, it is assumed that a composite processed image is generated for printing in a product and collated with a character or number pattern in the image recognition unit 18. At this time, since the inclination angle or the range thereof representing the outline of the pattern of characters and numbers is known in advance, the image recognition unit 18 uses the visible / invisible classification unit described in the fifth embodiment to display characters and characters. It is possible to generate image data in which only the number pattern remains and other extra noise components are removed. The image recognition unit 18 specifies the type of characters and numbers by using the pattern recognition unit for the image data in which only the patterns of characters and numbers remain. At this time, since the extra noise component is removed in advance from the image data to be the target of pattern recognition, it is possible to realize easy pattern recognition, high accuracy, and the like.

<< Main effect of Embodiment 6 >>
As described above, by using the image pickup apparatus of the sixth embodiment, in addition to the various effects described in the first embodiment and the like, it is possible to realize easy pattern recognition, high accuracy, and the like. Further, since only the necessary information (for example, the recognition result of characters and numbers) can be output as compressed data from the composite processed image, the information from the output unit 15 can be significantly reduced. As a result, it is possible to reduce the storage capacity of output information and shorten the time required for transfer.

(Embodiment 7)
<< Outline of imaging device >>
FIG. 21 is a schematic view showing a configuration example of a part of the image pickup apparatus according to the seventh embodiment of the present invention. The image pickup device 1 shown in FIG. 21 is, for example, a periodic pattern generation unit 10, an illumination unit 11 (not shown), an adjustment unit 12, an image pickup unit 13 (not shown), an image processing unit 14, and an output unit similar to those in FIG. In addition to 15, an image recognition unit 18 and an automatic adjustment unit 19 are provided. The image recognition unit 18 recognizes the three-dimensional shape of the object 2 based on the image data Γ (x, y) of the composite processed image from the image processing unit 14. Based on the recognition result of the image recognition unit 18, the automatic adjustment unit 19 instructs the area of the completely dark unit 31 via the adjustment unit 12 so as to visualize only the three-dimensional shape having a predetermined feature.

In the fifth and sixth embodiments, an example is shown in which the image recognition unit 18 recognizes the entire shape of the target unit 2 while changing the parameters to the adjustment unit 12, but it is not always necessary to recognize the entire shape, for example. It is only necessary to fulfill the purpose according to various purposes such as the case of detecting a defect or the case of detecting a pattern. In order to achieve such a purpose, the automatic adjustment unit 19 does not simply change the parameters linearly, but performs various optimizations of the parameters such as, for example, narrowing down the range and changing the parameters in a discrete manner.

As an example, when detecting a pattern, the automatic adjustment unit 19 can narrow down the range of parameters based on the inclination angle of the pattern determined in advance. Further, as another example, the automatic adjustment unit 19 may determine the next parameter by observing the difference in the recognition result of the image recognition unit 18 while changing the parameter by using the two-minute search. .. At this time, for example, when a defect is detected in a range of a certain inclination angle by the image recognition unit 18, the automatic adjustment unit 19 pays attention to the defect and specifies a more detailed inclination angle of the defect. It is also possible to narrow down the parameters.

<< Main effects of Embodiment 7 >>
As described above, by using the image pickup apparatus of the seventh embodiment, it is possible to realize automatic control of the image pickup apparatus according to the purpose in addition to the various effects described in the first embodiment and the like. For example, when applied to a surface defect inspection, it is possible to automate the inspection that has been visually performed by a person through the work of changing the angle and position with respect to the lighting, and the effects of labor saving and quality stabilization can be obtained. Further, by optimizing the parameters, the number of imagings and the like can be reduced, and the time cost such as shortening the inspection time can be reduced.

Each of the above-described embodiments is for explaining the present invention in an easy-to-understand manner, and does not limit the scope of the present invention. Further, it is possible to add / delete / replace a part of the configuration of each embodiment with another configuration. For example, in the image pickup apparatus 1 having the transmission type arrangement shown in the first embodiment or the reflection type arrangement shown in the third embodiment, the illumination unit 11 of the second embodiment, the image processing unit 15 of the fourth embodiment, and the fifth embodiment. , Any one of the image recognition units 18 of 6 and 6 or a combination thereof may be provided. Further, the image pickup apparatus 1 shown in the fifth embodiment or the sixth embodiment may include the automatic adjustment unit 19 shown in the seventh embodiment.

Further, each of the above configurations, functions, processing units, processing means and the like may be realized by hardware by designing a part or all of them by, for example, an integrated circuit. Further, each of the above configurations, functions, and the like may be realized by software by the processor interpreting and executing a program that realizes each function. Information such as programs, tables, and files that realize each function can be placed in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

1 ... Image pickup device, 2 ... Object, 10 ... Periodic pattern generation unit, 11 ... Lighting unit, 12 ... Adjustment unit, 13 ... Imaging unit, 14 ... Image processing unit, 15 ... Output unit, 16 ... Movable stage control unit, 18 ... Image recognition unit, 19 ... Automatic adjustment unit, 30 ... Completely bright part, 31 ... Completely dark part, 110 ... Uniform lighting unit, 111 ... Shading unit, 142 ... First calculation unit, 143 ... Second calculation unit 144 ... Third arithmetic unit.

Claims (13)

A lighting unit that irradiates an object with light having a corresponding luminance distribution for each of a plurality of periodic patterns having different phases.
A periodic pattern generation unit that generates the periodic pattern and controls the illumination unit so as to have the corresponding luminance distribution for each of the plurality of periodic patterns.
An imaging unit that generates a plurality of captured image data by imaging a transmission pattern or a reflection pattern from the object for each of the plurality of periodic patterns.
An image processing unit that calculates image data of a composite processed image that represents the characteristics of the three-dimensional shape of the object using the plurality of captured image data, and an image processing unit.
It is an image pickup device having
The periodic pattern is a pattern in which a completely dark portion having a minimum pixel value is generated in the illumination portion by a finite area.
When calculating the image data of the composite processed image, the image processing unit extracts the minimum pixel value for each pixel of the plurality of captured image data.
Further, the periodic pattern generation unit is provided with an adjusting unit that indicates the area of the completely dark portion in the periodic pattern.
Imaging device. In the image pickup apparatus according to claim 1 ,
The adjusting unit instructs the periodic pattern generation unit of the period of the periodic pattern and the ratio of the completely dark portion in the period.
Imaging device. A lighting unit that irradiates an object with light having a corresponding luminance distribution for each of a plurality of periodic patterns having different phases.
A periodic pattern generation unit that generates the periodic pattern and controls the illumination unit so as to have the corresponding luminance distribution for each of the plurality of periodic patterns.
An imaging unit that generates a plurality of captured image data by imaging a transmission pattern or a reflection pattern from the object for each of the plurality of periodic patterns.
An image processing unit that calculates image data of a composite processed image that represents the characteristics of the three-dimensional shape of the object using the plurality of captured image data, and an image processing unit.
It is an image pickup device having
The periodic pattern is a pattern in which a completely dark portion having a minimum pixel value is generated in the illumination portion by a finite area.
When calculating the image data of the composite processed image, the image processing unit extracts the minimum pixel value for each pixel of the plurality of captured image data.
Further, it has an adjusting unit that determines the area of the completely dark portion in the periodic pattern by instructing the physical arrangement relationship between the lighting unit and the object.
Imaging device. In the image pickup apparatus according to claim 1 or 3 ,
The periodic pattern is a rectangular wavy pattern.
Imaging device. A lighting unit that irradiates an object with light having a corresponding luminance distribution for each of a plurality of periodic patterns having different phases.
A periodic pattern generation unit that generates the periodic pattern and controls the illumination unit so as to have the corresponding luminance distribution for each of the plurality of periodic patterns.
An imaging unit that generates a plurality of captured image data by imaging a transmission pattern or a reflection pattern from the object for each of the plurality of periodic patterns.
An image processing unit that calculates image data of a composite processed image that represents the characteristics of the three-dimensional shape of the object using the plurality of captured image data, and an image processing unit.
It is an image pickup device having
The periodic pattern is a pattern in which a completely dark portion having a minimum pixel value is generated in the illumination portion by a finite area.
When calculating the image data of the composite processed image, the image processing unit extracts the minimum pixel value for each pixel of the plurality of captured image data.
The image processing unit
The minimum pixel value and the maximum pixel value are extracted for each pixel of the plurality of captured image data, and the image data of the minimum value image composed of the minimum pixel value and the maximum pixel value are composed. The first calculation unit that calculates the image data of the maximum value image,
A second image data of the composite processed image is calculated by calculating the ratio of the difference between the pixel values of the maximum value image and the minimum value image to the sum of the pixel values of the maximum value image and the minimum value image for each pixel. And the calculation part of
Imaging device with . In the image pickup apparatus according to claim 5 ,
The second calculation unit calculates the first image data and the second image data as the image data of the composite processed image.
The first image data is data obtained when the area of the completely dark portion in the periodic pattern is the first area.
The second image data is data obtained when the area of the completely dark portion in the periodic pattern is a second area different from the first area.
The image processing unit further has a third calculation unit that calculates image data of another composite processed image using the first image data and the second image data.
Imaging device. In the image pickup apparatus according to claim 6 ,
The third calculation unit calculates the difference between the first image data and the second image data for each pixel.
Imaging device. In the image pickup apparatus according to claim 1 or 3 ,
Further, while changing the area of the completely dark portion via the adjusting unit, the image processing unit is made to calculate the image data of the composite processed image, and the object is three-dimensionally based on the image data of the composite processed image. It has an image recognition unit that recognizes the shape.
Imaging device. In the image pickup apparatus according to claim 8 ,
The image recognition unit recognizes the inclination angle of the unevenness in the object.
Imaging device. In the image pickup apparatus according to claim 1 or 3 ,
Further, the image processing unit is made to calculate the image data of the composite processed image in a state where the area of the completely dark portion is instructed via the adjustment unit, and the object is preliminarily formed based on the image data of the composite processed image. It has an image recognition unit that recognizes a predetermined pattern.
Imaging device. In the image pickup apparatus according to claim 1 or 3 , further
An image recognition unit that recognizes the three-dimensional shape of the object based on the image data of the composite processed image from the image processing unit.
Based on the recognition result of the image recognition unit, an automatic adjustment unit that instructs the area of the completely dark portion via the adjustment unit so as to visualize only a three-dimensional shape having a predetermined feature, and an automatic adjustment unit.
Have,
Imaging device. A uniform lighting unit that emits uniform light,
By providing an opening arranged between the uniform illumination unit and the object and periodically formed and a closed opening provided between the openings, light that passes through the opening and is blocked by the closed opening is provided. A light-shielding portion that generates a periodic pattern and irradiates the object with the periodic pattern.
A movable stage control unit that controls the object to be irradiated with a plurality of periodic patterns having different phases by moving the light-shielding unit in the periodic direction.
An imaging unit that generates a plurality of captured image data by imaging a transmission pattern or a reflection pattern from the object for each of the plurality of periodic patterns.
An image processing unit that calculates image data of a composite processed image that represents the characteristics of the three-dimensional shape of the object using the plurality of captured image data, and an image processing unit.
It is an image pickup device having
Based on the closure, the periodic pattern is provided with a completely dark area having the lowest brightness in a finite area.
When calculating the image data of the composite processed image, the image processing unit extracts the minimum pixel value for each pixel of the plurality of captured image data.
The image processing unit
The minimum pixel value and the maximum pixel value are extracted for each pixel of the plurality of captured image data, and the image data of the minimum value image composed of the minimum pixel value and the maximum pixel value are composed. The first calculation unit that calculates the image data of the maximum value image,
A second image data of the composite processed image is calculated by calculating the ratio of the difference between the pixel values of the maximum value image and the minimum value image to the sum of the pixel values of the maximum value image and the minimum value image for each pixel. And the calculation part of
Imaging device with . A lighting unit that irradiates an object with light having a corresponding luminance distribution for each of a plurality of periodic patterns having different phases.
A periodic pattern generation unit that generates the periodic pattern and controls the illumination unit so as to have the corresponding luminance distribution for each of the plurality of periodic patterns.
An imaging unit that generates a plurality of captured image data by imaging a transmission pattern or a reflection pattern from the object for each of the plurality of periodic patterns.
An image processing unit that calculates image data of a composite processed image that represents the characteristics of the three-dimensional shape of the object using the plurality of captured image data, and an image processing unit.
It is an image pickup device having
The periodic pattern is a pattern in which a completely bright portion having a maximum pixel value is generated in the illumination portion by a finite area.
When calculating the image data of the composite processed image, the image processing unit extracts the maximum pixel value for each pixel of the plurality of captured image data.
Further, the periodic pattern generation unit is provided with an adjusting unit that indicates the area of the completely bright portion in the periodic pattern.
Imaging device.

Images