JP6506914B2 - Three-dimensional image processing apparatus, three-dimensional image processing method, three-dimensional image processing program, computer readable recording medium, and recorded apparatus - Google Patents

Description

  The present invention relates to a three-dimensional image processing apparatus, a three-dimensional image processing method, a three-dimensional image processing program, a computer readable recording medium, and a recorded apparatus.

  At many production sites such as factories, image processing apparatuses have been introduced that automate and speed up inspections that have relied on human vision. The image processing apparatus captures an image of a workpiece flowing on a production line such as a belt conveyor with a camera, and executes measurement processing such as edge detection and area calculation of a predetermined area using the obtained image data. Then, based on the processing result of the measurement processing, inspections such as chipping detection of a workpiece and position detection of an alignment mark are performed, and a determination signal that determines the presence or absence of a chipping or displacement of a workpiece is output. Thus, the image processing apparatus may be used as one of the FA sensors.

  Images targeted for measurement processing by the image processing apparatus used as an FA sensor are mainly luminance images that do not include height information. Therefore, when it comes to the above-mentioned chipping detection of the work, it is good at stably detecting the two-dimensional shape of the chipped part, but it stably stabilizes the three-dimensional shape which hardly appears as a luminance image It is difficult to detect. For example, it is conceivable to detect a shadow caused by the dent of a flaw by devising the type and direction of illumination that illuminates a workpiece at the time of inspection, and indirectly detect a three-dimensional shape, but in a luminance image Clear shadows are not always detected. For example, if the judgment threshold value is biased to the safety side to prevent a false determination of a defective product as a non-defective product when an unclear shadow is detected, for example, a large number of non-defective products are judged as a defective product and the yield is deteriorated. There is a risk of

  Therefore, the height is expressed two-dimensionally by using not only the brightness image having the gray value corresponding to the amount of light received by the camera as the pixel value but also the gray value corresponding to the distance between the camera and the work as the pixel value. An appearance inspection using a distance image (see, for example, Patent Document 1) can be considered.

  An example of the three-dimensional image processing apparatus is shown in a schematic view of FIG. This three-dimensional image processing apparatus is connected to a head unit including an imaging unit such as a light receiving element and the head unit, and a controller that receives image data captured by the head unit and generates a distance image from the acquired image data It consists of parts.

  Here, the principle of triangular distance measurement will be described based on FIG. In the head unit, an angle α between the optical axis of incident light emitted from the light projecting unit 110 and the optical axis of the reflected light incident on the light receiving unit 120 (the optical axis of the light receiving unit 120) is set in advance . Here, when the workpiece is not placed on the workpiece, the incident light emitted from the light projecting unit 110 is reflected by the point O of the placement surface of the workpiece and is incident on the light receiving unit 120. On the other hand, when the work is placed on the work, the incident light emitted from the light projecting unit 110 is reflected by the point A on the surface of the work and becomes reflected light and is incident on the light receiving unit 120. Then, the distance d in the X direction between the point O and the point A is measured, and the height h of the point A on the surface of the work is calculated based on the distance d.

  The three-dimensional shape of the workpiece is measured by calculating the heights of all the points on the surface of the workpiece by applying the measurement principle of triangular distance measurement described above. In the pattern projection method, in order to irradiate incident light to all points on the surface of the workpiece, the light emitting unit 110 emits incident light according to a predetermined structured pattern, receives reflected light reflected on the workpiece surface, and receives light. The three-dimensional shape of the workpiece is efficiently measured based on the plurality of pattern images.

  As such a pattern projection method, a phase shift method, a space coding method, a multi-slit method and the like are known. The projection pattern is changed by three-dimensional measurement processing using a pattern projection method, the imaging is repeated plural times by the head unit, and is sent to the controller unit. The controller unit can perform calculation based on the pattern projection image sent from the head unit, and obtain a distance image having workpiece height information.

  On the other hand, in the existing image processing apparatus, a luminance image mainly having luminance as a pixel value is used. For example, there is a system in which an inspection is performed by image processing by capturing a luminance image in which the appearance of a workpiece is represented by optical shading with a monochrome camera with respect to a workpiece transported on a line. Under such circumstances, if it is attempted to introduce a three-dimensional measuring device and a processing device that processes three-dimensional data (point cloud data) output from the new three-dimensional measuring device, considerable costs are incurred.

  Therefore, if the height information is expressed as a distance image representing the lightness and darkness of the image, the image data itself to be handled becomes equivalent to the existing luminance image, and therefore, even in the equipment of the image processing apparatus using the existing luminance image It will be able to handle images.

  In this case, in the conventional luminance image, an image with a relatively low number of gradations was used. For example, an image in which information per pixel is expressed by eight gradations is often used. On the other hand, for image data having height information, a large number of gradations (for example, 16 gradations) are required to express the height information. For this reason, in order to be able to process distance images with an image processing apparatus that handles existing low-gradation images, it is necessary to convert multi-gradation distance images into relatively low-gradation distance images. . (Hereinafter, an image obtained by converting a multi-gradation distance image into a low-gradation distance image is referred to as a "low-gradation distance image", and a conventional luminance image having luminance information is referred to as a "luminance image".

  Thus, in addition to the conventionally used luminance image, by using a low gradation distance image subjected to gradation conversion, an image such as extraction of a conventional shape in an inspection to determine normality / abnormality of a work Not only the processing but also processing such as measurement of height using height information can be used in the three-dimensional image processing apparatus.

  In this case, it is necessary to perform gradation conversion to convert height information of high gradation (for example, 16 gradations) into the same gradation number (for example, 8 gradations) as that of the existing luminance image. At this time, if the gradation conversion parameter is not set appropriately, height information necessary for the inspection may be lost, and an image suitable for the inspection may not be obtained.

  Therefore, it is conceivable to appropriately set the gradation conversion parameter and determine the reference height for gradation conversion using the height information of the work to be inspected.

  However, when there is variation in the height direction in the inspection target work of the user, it may not be possible to acquire an image suitable for inspection only by performing gradation conversion using the gradation conversion parameter initially specified.

Patent No. 4969478 gazette

  The present invention has been made to solve such conventional problems. The main object of the present invention is to provide a three-dimensional image processing apparatus, a three-dimensional image processing method, and a third-order image processing method that suppress loss of height information and suppress deterioration of accuracy when converting distance images into low gradation distance images. An original image processing program and a computer readable recording medium are provided.

Means for Solving the Problems and Effects of the Invention

In order to achieve the above object, according to a three-dimensional image processing apparatus according to a first aspect of the present invention, a distance image including height information of an inspection object is obtained, and an image based on the distance image A three-dimensional image processing apparatus capable of performing processing, comprising: an imaging unit configured to capture an image of an inspection object; and a distance image generating unit configured to generate a distance image based on the image captured by the imaging unit A display unit for displaying the distance image generated by the distance image generation unit; and a setting unit for receiving an operation from the user and performing various settings in a setting mode for performing necessary settings. In the operation mode in which the three-dimensional image processing is performed by the three-dimensional image processing apparatus after the setting mode, a height of the distance image, the number of gradations lower than the number of gradations of the distance image, in the distance image information It provided the low tone range image by replacing the density value of the image and the gradation conversion means for gradation conversion, the relative low tone range image, and a test execution means for executing a predetermined inspection process , the setting means, the distance image generated by the distance image generating means as a gradation conversion parameters constituting the gradation conversion conditions for gradation conversion, setting a reference plane serving as a reference for performing grayscale conversion And an extraction method selection unit for receiving selection of either static conversion or dynamic conversion, and in the operation mode, the gradation conversion unit is configured to When the objective conversion is selected, the tone conversion unit performs tone conversion on the newly input distance image with reference to the height of the reference plane set in the setting mode, and By extraction method selection means If serial dynamic translation is selected, based on the newly inputted range image, based on the height of the newly set reference plane, to perform the gradation conversion by the gradation conversion means It can be configured. According to the above-described configuration, a high-gradation distance image is not simply converted to a low-gradation distance image, but after the reference plane is set, the gradation conversion is performed on the basis of the height of the reference plane. It is possible to realize a high-accuracy inspection by avoiding the case where height information in the vicinity of the surface is lost in gradation conversion.

Further, according to the three-dimensional image processing apparatus according to the second aspect, the point wherein the reference plane setting means, by specifying an arbitrary point of the distance in the image displayed on the display means, which is the designated It can be configured to set the height information of as a reference plane.

Further, according to the three-dimensional image processing apparatus according to the third aspect, the point wherein the reference plane setting means, by designating a desired point of the distance in the image displayed on the display means, which is the designated It can be configured to set the height information of as a reference plane.

Furthermore, according to the three-dimensional image processing device pertaining to the fourth aspect, the reference surface setting unit is specified by specifying an arbitrary surface in the distance image displayed on the display unit. The height information of the surface can be set as the reference surface.

Furthermore, according to the three-dimensional image processing device pertaining to the fifth aspect, the reference surface setting unit is specified by specifying any three points in the distance image displayed on the display unit. The height information of the surface including the three points can be set as the reference surface.

Furthermore, according to the three-dimensional image processing device pertaining to the sixth aspect, the reference surface setting means calculates a curved surface from the distance image displayed on the display means, and the calculated curved surface is used as a reference surface Can be configured to set as

Furthermore, according to the three-dimensional image processing apparatus according to the seventh aspect, a light projection unit for projecting incident light as structured illumination of a predetermined projection pattern from a direction oblique to the optical axis of the imaging unit The imaging means acquires the reflected light that is projected by the light projecting means and reflected by the inspection object to capture a plurality of pattern projection images, and the distance image generating means performs the imaging The distance image may be generated based on the plurality of pattern projection images captured by the means. According to the above configuration, inspection processing such as height measurement using a distance image using pattern projection can be realized.

Furthermore, according to the three-dimensional image processing apparatus according to the eighth aspect, the image pickup device further includes a light projection unit for projecting light to the inspection object by a light cutting method, The image projected by the means may be captured, and the distance image generating means may be configured to combine the profiles obtained by the light cutting method to generate a distance image.

Furthermore, according to the three-dimensional image processing program according to the ninth aspect, while obtaining a distance image including height information of the inspection object, it is possible to perform image processing based on the distance image. An image processing program, which projects incident light as structured illumination of a predetermined projection pattern from a direction oblique to the optical axis of the imaging means to a computer, and acquires reflected light reflected by an inspection object A range image generation function capable of generating a range image based on a plurality of captured pattern projection images, a function to display the range image generated by the range image generation function, and a setting mode for performing necessary settings In the above, as a setting function for receiving an operation from the user and performing various settings, it is possible to use gradation conversion conditions that constitute gradation conversion conditions when performing gradation conversion on the distance image generated by the distance image generation function. A reference plane setting function for setting a reference plane serving as a reference for performing the gradation conversion as a parameter, an extraction method selection function for receiving selection of either static conversion or dynamic conversion, and after the setting mode In an operation mode for performing three-dimensional image processing, a low gradation distance obtained by replacing height information of the distance image with a gradation value of the image, the number of gradations lower than the number of gradations of the distance image An image tone conversion function for tone conversion to an image and an inspection execution function for performing a predetermined inspection process on the low tone distance image are realized, and in the operation mode, the tone When static conversion is selected by the extraction method selection function, the conversion function performs the gradation on the newly input distance image with reference to the height of the reference plane set in the setting mode. Gradation by conversion function When the dynamic conversion is selected by the extraction method selection function, the floor is selected based on the height of the newly set reference plane based on the newly input distance image. Gradation conversion can be performed by the tonal conversion function. According to the above-described configuration, a high-gradation distance image is not simply converted to a low-gradation distance image, but after the reference plane is set, the gradation conversion is performed on the basis of the height of the reference plane. It is possible to realize a high-accuracy inspection by avoiding the case where height information in the vicinity of the surface is lost in gradation conversion.

Furthermore, the computer-readable recording medium or the stored device according to the tenth aspect stores the three-dimensional image processing program. Recording media include CD-ROM, CD-R, CD-RW, flexible disk, magnetic tape, MO, DVD-ROM, DVD-RAM, DVD-R, DVD + R, DVD-RW, DVD + RW, Blu-ray (registration The media includes magnetic disks such as HD DVD (AOD), optical disks, magneto-optical disks, semiconductor memories, and other media capable of storing programs. The program also includes programs distributed by downloading through a network line such as the Internet, in addition to those stored and distributed in the recording medium. The stored devices include general-purpose or dedicated devices in which the above-described program is implemented in an executable state such as software or firmware. Furthermore, each process or function included in the program may be executed by program software that can be executed by a computer, or the process of each part may be hardware such as a predetermined gate array (FPGA, ASIC) or program software. And a partial hardware module that implements some elements of hardware may be implemented in a mixed form.

It is a figure showing an example of system configuration of a three-dimensional image processing system including an image processing device concerning an embodiment of the invention. It is a figure showing an example of system composition of a three-dimensional image processing system concerning a modification of the present invention. It is a schematic diagram which shows the hardware constitutions of the three-dimensional image processing apparatus which concerns on Embodiment 2 of this invention. FIG. 4A is a schematic view showing a head unit of a three-dimensional image processing apparatus according to a third embodiment of the present invention, and FIG. 4B is a schematic view showing a head unit of the three-dimensional image processing apparatus according to the fourth embodiment. It is a block diagram which shows the three-dimensional image processing apparatus concerning Embodiment 3 of this invention. It is a block diagram which shows the controller part of FIG. It is a flowchart which shows the processing operation of the image processing apparatus which concerns on this embodiment. It is a flowchart which shows the procedure of static conversion at the time of setting. It is an image figure showing the initial screen which added "imaging" with a three-dimensional image processing program. FIG. 11 is an image view showing a state in which image registration is selected on the imaging setting menu. It is an image figure showing an example of a screen of an image registration screen. It is an image figure which shows the example of a screen in process of registration of a distance image. It is an image figure showing an example of a screen under registration of a luminosity picture. It is an image figure showing an imaging setting screen. It is an image figure showing an imaging effective setting screen. It is an image figure which shows a three-dimensional measurement setting screen. It is an image figure which shows the kind of pre-processing which can be selected. It is an image figure showing an option which can be set up in a measurement impossible standard setting column. It is an image figure which shows the example which selected "none" in the measurement impossible standard setting column. It is an image figure which shows the example which selected "low" in the measurement impossible standard setting column. It is an image figure which shows the example which selected "middle" in the measurement impossible reference setting column. It is an image figure which shows the example which selected "high" in the measurement impossible standard setting column. It is an image figure which shows the option which can be set by the equal interval process setting column. It is an image figure which shows the example which made the "display image" selection column "the height image" for the equal interval process ON. It is an image figure which shows the example which made the equal interval process ON and made the "display image" selection column into a "grayscale image." It is an image figure which shows the example which OFF the uniform space | interval process and having displayed the distance image. It is an image figure showing the example which turned off the equal interval processing and displayed the luminance image. It is an image figure which shows the option which can be set by the space code setting column. It is an image figure which shows the state which selected "height image" in the "display image" selection column, and displayed the distance image on the 2nd image display area. It is an image figure which shows the state which selected the "grayscale image" in the "display image" selection column, and displayed the brightness | luminance image on a 2nd image display area. It is an image figure which shows the example which selected OFF by the space code setting column. It shows a state in which "grayscale image" is displayed in the "display image" selection field. It is an image figure which shows the option which can be set by the projector selection setting column. It is an image figure which shows the example which selected "1" in the projector selection setting column. It is an image figure which shows the example which selected "2" in the projector selection setting column. It is an image figure which shows the example which selected "1 + 2" in the projector selection setting column. It is an image figure which shows the option which can be set by the "display image" selection column. It is an image figure showing the example which displayed the pattern projection picture of the 1st projector in the 2nd image display field by choosing "stripe projection-projector 1" in the "display picture" selection column. It is an image figure showing the example which displayed the pattern projection picture of the 2nd projector in the 2nd image display field by choosing "stripe light projection-projector 2" in the "display picture" selection column. It is an image figure which shows the three-dimensional measurement setting screen which set the shutter speed setting column to "1/15." It is an image figure which shows the three-dimensional measurement setting screen which set the shutter speed setting column to "1/30." It is an image figure which shows the three-dimensional measurement setting screen which set the density range setting column to "ordinary (0)." It is an image figure which shows the three-dimensional measurement setting screen which set the density range setting column to "high (1)." Image showing how to add the "Height measurement" processing unit from the state in Figure 9 FIG. 45 is an image view showing an initial screen to which a “height measurement” processing unit has been added through FIG. 44; It is an image figure showing height measurement setting screen. It is an image figure which shows an inspection object area | region setting screen. It is an image figure which shows the state which displayed the drop-down menu of the "measurement area" setting column of FIG. It is an image figure which shows the state which selected "rotation rectangle" in the "measurement area" setting column. It is an image figure showing a measurement field edit screen. It is an image figure which shows the state which selected "circumference" in the "measurement area" setting column of a measurement area edit screen. It is an image figure showing the initial screen which added the 2nd "height measurement" processing unit. It is an image figure showing the state where "a rotation rectangle" was set up as a measurement field. It is an image figure which shows the state which sets "a rotation rectangle" as a measurement area | region. It is an image figure showing the state where "rotation rectangle" was set up as a measurement field. It is an image figure showing the state where it is going to add a "numerical operation" processing unit. It is an image figure showing the initial screen where the "numerical operation" processing unit was added. It is an image figure showing the state where the numerical operation edit screen was displayed. FIG. 59 is an image view showing a state in which an arithmetic expression is input to the numerical operation editing screen of FIG. 58. It is an image figure which shows the initial screen in which the "numerical operation" processing unit was set. It is an image figure showing the initial screen which added the "area" processing unit. It is an image figure showing an area setting screen. It is an image figure showing the field setting screen which sets up the details of rotation rectangle. It is an image figure showing the field setting screen where the rotation rectangle was set up. It is an image figure showing height extraction selection screen. It is an image figure showing GUI of a one-point specification screen. It is an image figure showing GUI of a one-point specification screen. It is an image figure showing GUI of a one-point specification screen. 69A is an image view showing a profile of an input image, and FIG. 69B is an image view showing a profile of a low tone distance image obtained by tone-converting the input image of FIG. 69A. It is an image figure which shows the state which made the gain increase from the state of FIG. It is an image figure which shows the state which reduced the gain from the state of FIG. It is an image figure which shows the state which selected detailed setting in FIG. FIG. 18 is an image diagram showing a GUI of an emphasizing method details setting screen for one-point specification. It is an image figure which shows the state which displayed the drop-down list of the "extraction height" setting column from the state of FIG. 75A is an image diagram showing a profile of an input image, FIG. 75B is an image diagram showing the input image of FIG. 75A with reference to a reference plane, and FIG. 75C is an image diagram showing a profile of a low gradation distance image obtained by gradation converting FIG. It is. 76A is a luminance image, FIG. 76B is a distance image of high gradation, FIG. 76C is a low gradation distance image obtained by gradation conversion of FIG. 76B, FIG. 76D is a low gradation distance image with higher gain than FIG. FIG. 76F is an image view showing a low gradation distance image in which the noise removal is raised more than FIG. 76D, and FIG. 76F shows a low gradation distance image in which “extraction height” is set to a high side in FIG. 76E. FIG. 77A is an image view showing a profile of an input image, and FIG. 77B is an image view showing a profile of a low tone distance image in which the input image in FIG. 77A is tone-converted by setting “extraction height” to the high side. It is an image figure showing the state where the gradation conversion picture was displayed. FIG. 79A is a perspective view showing an example of a work in which the method of setting a reference plane at one-point specification is effective, and FIG. 79B is an image view of a low gradation distance image obtained by gradation conversion of the distance image captured in FIG. It is an image figure showing GUI of a height extraction selection screen. It is an image figure showing GUI of a three-point specification screen. It is an image figure which shows the state which designates the 1st point by the height extraction means from the state of FIG. It is an image figure which shows the state which designates the 2nd point further from the state of FIG. It is an image figure which shows the state which designates the 3rd point further from the state of FIG. It is an image figure showing GUI of a detailed setting screen for three-point specification. 86A is a perspective view showing an example of a work in which the method of setting a reference plane by specifying three points is effective, FIG. 86B is an image view of a low gradation distance image obtained by gradation conversion of the distance image captured in FIG. 86A; FIG. 86D is an image diagram of an image obtained by binarizing FIG. 86B, and FIG. 86D is an image diagram of a binarized image obtained by designating one point when the workpiece in FIG. 86A has an inclination. FIG. 87A is a perspective view showing an example of another work in which the method of setting a reference plane by specifying three points is effective, and FIG. 87B is an image view of a low gradation distance image obtained by gradation conversion of the distance image captured in FIG. 87C is an image view of an image obtained by binarizing FIG. 87B, and FIG. 87D is an image view of a binarized image obtained by designating one point when the work shown in FIG. 87A is inclined. It is an image figure showing GUI of a height dynamic extraction setting screen. It is an image figure which shows the drop down box of a "calculation method" selection column. It is an image figure showing an average height standard setting screen. It is an image figure showing a mask field setting screen. It is an image figure showing a plane standard detailed setting screen. FIG. 93A is a perspective view showing an example of a work in which the method of setting the reference plane based on the average height is effective, and FIG. 93B is an image view of a low gradation distance image obtained by gradation conversion of the distance image captured in FIG. It is an image figure showing a plane standard detailed setting screen. It is an image figure which shows the detail of an invalid pixel designation | designated column on the screen of FIG. It is an image figure which shows a free-form surface reference | standard setting screen. It is an image figure which shows the state which enlarged the numerical value of the "extraction size" designation | designated column in FIG. FIG. 98A is a perspective view showing an example of a work in which the method of setting a reference plane based on a free-form surface is effective, and FIG. 98B is an image view of a low gradation distance image obtained by gradation conversion of the distance image captured in FIG. It is an image figure which shows the state which displayed the extraction area setting dialog from the state of FIG. It is an image figure which shows the state which selected "rectangle" in the extraction area | region selection column of FIG. It is an image figure which shows the state which displayed the extraction area edit dialog from the state of FIG. It is an image figure which shows the state which selected "circular" by the mask area selection column of FIG. It is an image figure which shows the state which displayed the mask area | region edit dialog from the state of FIG. It is an image figure which shows the state which set height extraction by the "area" process unit. It is an image figure showing a filter processing setting screen. It is an image figure which shows a binarization level setting screen. It is an image figure showing the state where filter processing was set up. It is an image figure showing a judgment condition setting screen. It is an image figure showing the state where the judgment condition was set up. It is an image figure showing the initial screen which added the "blob" processing unit. It is an image figure which shows a mode that a filter process is set in a "blob" process unit. It is an image figure which shows a mode that a detection condition is set in a "blob" processing unit. It is an image figure which shows a mode that a judgment condition is set in a "blob" process unit. It is an image figure showing a state where it is going to add a "color inspection" processing unit to an initial screen. It is an image figure which shows a mode that the area | region of a circle | round | yen is set to a "color inspection" process unit. It is an image figure which shows the state which area | region setting of the circle was performed to the "color inspection" process unit. It is an image figure which shows the state which set the density | concentration average to the "color inspection" process unit. It is an image figure showing the initial screen which set up a "color inspection" processing unit. 15 is a flowchart showing a flow of processing at the time of operation of the head unit of the three-dimensional image processing apparatus according to Embodiment 3. FIG. 18 is a flowchart showing a flow of processing at the time of operation of the head unit of the three-dimensional image processing apparatus according to Embodiment 4. FIG. 3 is a flowchart showing a gradation conversion method according to Embodiment 1; FIG. 16 is a flowchart showing a flow of processing at the time of operation of the controller unit of the three-dimensional image processing apparatus according to Embodiment 3. FIG. It is an image figure which shows the initial screen of a three-dimensional image processing program. It is an image figure which shows the state which set the search object area | region on the brightness | luminance image. It is an image figure showing the state where two or more inspection object fields were set up on a distance picture. It is an image figure which shows the state which expanded the distance image of FIG. It is an image figure showing the state where height measurement is performed with a three-dimensional image processing program. FIG. 5 is a data flow diagram for generating a distance image by combining a phase shift method and a space coding method. FIG. 6 is a data flow diagram for generating a distance image only by a phase shift method without using a space coding method. It is a data flow figure showing the procedure of turning off XY equal pitching and acquiring a Z picture. It is a figure which shows the example which outputs point cloud data. It is a flowchart which shows the procedure of static conversion at the time of operation | use. It is a flowchart which shows the procedure of the dynamic conversion at the time of operation | use. It is a perspective view which shows the workpiece | work to be examined. It is a schematic diagram which shows the method of preparing several gradation conversion parameter sets beforehand. It is a flowchart which shows the procedure of the method of FIG. It is a perspective view which shows the example of the workpiece | work which hardly changes in height. FIG. 138A is a perspective view showing an example of a work for which a change in height is to be detected, and FIG. 138B is a conceptual view of a low gradation distance image obtained by gradation conversion of the distance image captured in FIG. 138A. It is a perspective view which shows the example of a workpiece | work in which designation | designated of the reference plane by dynamic conversion (average height reference | standard) is effective. It is a perspective view showing an example of a work in which specification of a reference plane by dynamic conversion (plane standard) is effective. It is a perspective view which shows the example of the workpiece | work in which designation | designated of the reference plane by dynamic conversion (free-form surface reference | standard) is effective. It is a flowchart which shows the procedure which repeats adjustment of a gradation conversion parameter until a suitable image is obtained. It is a flowchart which shows the procedure which abbreviate | omitted determination of whether the image is appropriate in FIG. It is a flowchart which shows the concrete procedure of gradation conversion parameter adjustment. 145A shows the appearance of the work, FIG. 145B shows an image of the distance image obtained from the work shown in FIG. 145A, and FIG. 145C shows an image showing the inspection target area set for the height inspection process for the distance image of FIG. Fig. 145D is an image showing a state where an inspection target area is set for image inspection processing on the distance image in Fig. 145B, and Fig. 145E is a low gradation distance image obtained by performing gradation conversion on the distance image in Fig. 145D. It is a figure which each shows the image of. It is a flowchart which shows the procedure at the time of setting. It is an image figure showing the screen which sets up an "area" processing unit to the work of Drawing 145A. It is an image figure which shows the screen which sets the conditions of height extraction in FIG. It is a flowchart which shows the procedure at the time of driving | operation. It is a flowchart which shows the procedure in the case of performing gradation conversion by the test | inspection process of FIG. It is a flowchart which shows the procedure in the case of not performing gradation conversion by the test | inspection process of FIG. It is a flowchart which shows the procedure which sets test | inspection process conditions. It is an image figure showing an image setting screen. It is an image figure which shows the state which called the image variable selection screen which can select a brightness | luminance image or a distance image. It is an image figure showing the state where the image variable selection screen which can select only a distance image is called. It is a flow chart which shows a procedure which chooses inspection processing and makes inspection processing conditions, after making an image select. It is a schematic diagram which shows the state which acquired the brightness | luminance image and the distance image. FIG. 160 is a schematic view showing an inspection processing tool that can be set when a luminance image is selected in FIG. 157. FIG. 160 is a schematic view showing an inspection processing tool that can be set when the distance image is selected in FIG. 157. It is a schematic diagram which shows a mode that a distance image is imaged by a triangular ranging system. It is a flowchart which shows the procedure of fitting the height information distributed in a free-form surface object area | region. It is an image figure which shows the distance image of the test subject of uniform shape in the vertical direction. It is an image figure which shows the low gradation distance image which extracted and converted the gradation to XY direction from the distance image of FIG. It is an image figure which shows the other example of a free-form surface reference | standard setting screen. FIG. 164 is an image view showing a detailed setting screen of a free curved surface standard setting screen; It is an image figure which shows the low gradation distance image which extracted and converted the gradation to the Y direction from the distance image of FIG. It is an image figure which shows the distance image of the other to-be-tested object of uniform shape in the vertical direction. FIG. 167 is an image view showing a low gradation distance image extracted and converted in gradation from the distance image of FIG. 167 in the X and Y directions. FIG. 167 is an image view showing a low gradation distance image extracted and converted in gradation from the distance image of FIG. 167 in the Y direction.

  Hereinafter, embodiments of the present invention will be described based on the drawings. However, the embodiments described below are a three-dimensional image processing apparatus, a three-dimensional image processing method, a three-dimensional image processing program, a computer readable recording medium, and a recorded device for embodying the technical concept of the present invention. The present invention does not specify the three-dimensional image processing apparatus, the three-dimensional image processing method, the three-dimensional image processing program, the computer readable recording medium, and the recorded apparatus as follows. Further, the present specification does not in any way specify the members described in the claims to the members of the embodiment. In particular, the dimensions, materials, shapes, relative arrangements, and the like of the components described in the embodiments are not intended to limit the scope of the present invention to the scope of the present invention unless otherwise specified. It is just an example. Note that the size, positional relationship, and the like of the members shown in each drawing may be exaggerated for the sake of clarity. Further, in the following description, the same names and reference numerals indicate the same or the same members, and the detailed description will be appropriately omitted. Furthermore, each element constituting the present invention may be configured such that a plurality of elements are constituted by the same member and one member is used in common as a plurality of elements, or conversely, the function of one member is realized by a plurality of members It can be shared and realized.

Further, in the present specification, “distance image (height image)” is used in the sense of an image including height information, for example, a three-dimensional image in which an optical luminance image is attached as texture information The composite image is also used in the meaning included in the distance image. Further, in the present specification, the display form of the distance image is not limited to that displayed in a two-dimensional form, but also includes that displayed in a three-dimensional form.
Embodiment 1

  The configuration of the three-dimensional image processing apparatus according to the first embodiment of the present invention is shown in FIG. The three-dimensional image processing apparatus 100 includes a head unit 1 and a controller unit 2. The head unit 1 includes a light projecting unit 20 that illuminates the inspection target (work) W, an imaging unit 10 that captures an image of the work W, and a head communication unit 36 for connecting to the controller unit 2.

  On the other hand, the controller unit 2 executes measurement processing such as edge detection and area calculation based on the captured image. The controller unit 2 can be detachably connected with display means 4 such as a liquid crystal panel, input means 3 such as a console for the user to perform various operations on the display means 4, PLC (Programmable Logic Controller), and the like.

  The above three-dimensional image processing apparatus 100 projects measurement light onto the work W by the light projecting means 20 of the head unit 1 and the reflected light reflected by the measurement light incident on the work W is patterned by the imaging means 10 Capture as a projection image. Further, a distance image is generated based on the pattern projection image, and this distance image is converted into a low gradation distance image in which the height information of each pixel is replaced with luminance. The controller unit 2 executes measurement processing such as edge detection and area calculation based on the converted low gradation distance image.

The workpiece W to be inspected is, for example, an article sequentially transported on the line, and is moving or stationary. The moving workpieces include those that rotate as well as parallel displacements by a conveyor or the like.
(Lighting means 20)

  The light projecting means 20 is used as illumination to illuminate the work W in order to generate a distance image. Therefore, the light projection means 20 is, for example, a light projector for projecting a linear laser beam to a work according to a light cutting method or a pattern projection method for acquiring a distance image, a sinusoidal stripe pattern on the work Can be a pattern projector or the like for projecting the In addition to the light emitting means, a general illumination device for bright field illumination or dark field illumination may be separately provided. Alternatively, it is possible to give the light emitting means 20 a function as a general lighting device.

  The controller unit 2 executes image processing using the distance image data acquired from the head unit 1 and outputs a determination signal as a signal indicating the determination result such as the quality of the work to a control device such as the PLC 70 connected externally. Do.

  The imaging unit 10 captures an image of a work based on a control signal input from the PLC 70, for example, an imaging trigger signal that defines a timing of capturing image data from the imaging unit 10.

  The display means 4 is a display device for displaying image data obtained by imaging a workpiece and the result of measurement processing using the image data. In general, the user can confirm the operation state of the controller unit 2 by visually recognizing the display means 4. The input unit 3 is an input device for moving the focus position on the display unit 4 or selecting a menu item. In addition, when using a touch panel for the display means 4, it can be used as a display means and an input means.

The controller unit 2 can also connect a personal computer PC for generating a control program of the controller unit 2. In addition, a three-dimensional image processing program for performing settings relating to three-dimensional image processing can be installed in the personal computer PC, and various settings for processing performed by the controller unit 2 can be performed. Alternatively, a processing order program that defines the processing order of image processing can be generated by software operating on the personal computer PC. The controller unit 2 sequentially executes each image processing in accordance with the processing order. The personal computer PC and the controller unit 2 are connected via a communication network, and the processing order program generated on the personal computer PC is, for example, a controller unit together with layout information and the like defining the display mode of the display means Transferred to 2 Also, conversely, the processing order program, layout information and the like can be fetched from the controller unit 2 and edited on the personal computer PC. The processing order program may be generated not only by the personal computer PC but also by the controller unit 2.
(Modification)

In the above example, dedicated hardware is constructed as the controller unit 2, but the present invention is not limited to this configuration. For example, as a 3D image processing apparatus 100 'according to the modification shown in FIG. 2, a general-purpose personal computer or workstation etc. with a dedicated inspection program or 3D image processing program installed functions as a controller unit 2' Can be connected to the head unit 1 and used. This three-dimensional image processing apparatus performs necessary settings such as image processing using a three-dimensional image processing program, and then performs image processing on a low gradation distance image according to the pattern projection image captured by the head unit 1 Perform an inspection.
(Head side communication means 36)

Further, in accordance with this, an interface can be provided on the head unit 1 side so that it can be connected to either the dedicated controller unit 2 or the personal computer functioning as the controller unit 2 as the head side communication means 36. For example, the head unit 1 is provided with a controller connection interface 36A for connecting to the controller unit 2 as shown in FIG. 1 as the head side communication means 36, or to be connected to a personal computer as shown in FIG. Interface 36B for PC connection is provided. Further, by making such an interface unitary exchangeable, the other configuration of the head unit can be made common to a certain extent, and the common head unit can be connected to either the controller unit or the personal computer. Alternatively, one head-side communication unit having a dedicated controller unit 2 and an interface connectable to any of the personal computer may be provided. As such an interface, existing communication standards, such as Ethernet (product name), USB, RS-232C, etc. can be used. Also, the communication system may be a dedicated communication system, not necessarily based on a standardized or general communication system.
(PC connection mode)

  Furthermore, the three-dimensional image processing program can also be provided with a PC connection mode for setting when using a personal computer as the controller unit 2 'connected to the head unit 1. That is, by changing the settable items and setting contents in accordance with whether the controller unit is a dedicated hardware or a personal computer, appropriate settings regarding three-dimensional image processing can be made in any case. It becomes possible. Furthermore, the operation check application of the head unit 1 and a viewer program having a simple measurement function can be installed on a personal computer functioning as the controller unit 2 'to check the operation and function of the connected head unit. You may do so.

  The “distance image” obtained by using the image pickup means 10 and the light projection means 20 shown in FIG. 1 corresponds to the gray value of each pixel according to the distance from the image pickup means 10 for picking up the workpiece W to the workpiece W Refers to an image that changes. In other words, it can be said that it is an image whose gray level is determined based on the distance from the imaging means 10 to the work W, it can also be said as a multi-valued image having gray levels corresponding to the distance to the work W It can be said that the image is a multi-valued image having gray values according to the height. Furthermore, it can also be said that it is a multi-valued image in which the distance from the imaging unit 10 is converted to a gray value for each pixel of the luminance image.

  There are two major methods for generating a distance image, and one is a passive method (passive measurement method) that generates a distance image using an image captured under illumination conditions for obtaining a normal image. The other is an active method (active measurement method) in which a distance image is generated by actively emitting light for measurement in the height direction. A representative method of the passive method is stereo measurement. This is because a distance image can be generated only by preparing two imaging units 10 and arranging these two cameras in a predetermined positional relationship, so a general image processing system for generating a luminance image is used. The distance image can be generated and the system construction cost can be suppressed. However, in stereo measurement, it is necessary to determine which point in the image obtained by one camera corresponds to which point in the image obtained by the other camera, so-called decision processing of corresponding points Has the problem of taking time. In addition, since the measurement position is only the corresponding point and not all the pixels, this point is not suitable for speeding up the appearance inspection.

  On the other hand, representative techniques of the active method are light cutting and pattern projection. The light cutting method is a method of optically measuring the shape, roughness, etc. of the surface, and a thin slit image is projected at an angle of about 45 ° to the surface of the inspection object, and the image is reflected in the specular reflection direction There is known a method of observing from the side or a method of irradiating a test object with a thin slit-like light bundle so as to cut it and observing the shape of a cutting line generated on the surface from the side. Moreover, although the profile (cut surface) of only one line can be obtained by the light cutting method, the distance image can be obtained by continuously changing the position where the light cutting is performed and combining the profiles obtained at each position. It can also be configured.

  Here, in the above-described stereo measurement method, one of the cameras is replaced with a light projector, and a line-like laser beam is projected to the work, and the line light corresponding to the shape of the object surface is The three-dimensional shape of the work is restored from the degree of distortion of the image. The photo-cleavage method enables stable measurement because determination of corresponding points is unnecessary. However, since only one line can be measured in one measurement, it is necessary to scan an object or a camera to obtain measurement values of all pixels. On the other hand, in the pattern projection method, the three-dimensional shape of the workpiece is restored by imaging a plurality of images while shifting the shape, phase or the like of the predetermined pattern projected onto the workpiece and analyzing the plurality of images captured It is There are several types of pattern projection methods. A plurality of (at least three) images are captured by shifting the phase of the sinusoidal stripe pattern, and the phase of the sine wave is determined for each pixel from the plurality of images. A three-dimensional shape is restored using a phase shift method that finds three-dimensional coordinates on the workpiece surface using the determined phase, or a kind of spatial frequency undulation that occurs when two regular patterns are combined. Moiré topography method, the pattern itself to be projected on the work is made different for every photography, for example, the stripe width becomes narrow with screen half, 1/4, 1/8, etc., with black and white duty ratio 50% A space coding method in which a pattern is sequentially projected, a pattern projection image is photographed in each pattern, and an absolute phase of the height of the workpiece is obtained, and a plurality of thin line pattern illuminations (multislits) are projected on the workpiece Moving the pattern with a narrow pitch than the slit cycle, multi-slit method of performing a plurality of times imaging is typical.

  The three-dimensional image processing apparatus 100 according to the present embodiment generates a distance image by combining the above-described phase shift method and space coding method. Thereby, the distance image can be generated without relatively moving the work or the head. The present invention is not limited to the generation of the distance image by the phase shift method and the space coding method, and the distance image may be generated by other methods. Also, any method other than the above-described method may be adopted, for example, optical radar method (time-of-flight), in-focus method, confocal method, white light interferometry, etc., to generate a distance image. Absent.

The arrangement layout of the imaging means 10 and the light emitting means 20 shown in FIG. 1 is such that the light emitting means 20 can project light from an oblique direction to the work W and receive reflected light from the work W in a substantially perpendicular direction. Of the image pickup means 10 in the vertical posture. As described above, it is possible to capture a pattern projection image capturing a shadow resulting from the unevenness of the surface shape of the workpiece W by inclining the light projection direction and the imaging direction without matching them.
Second Embodiment

However, the present invention is not limited to this arrangement example. For example, as in the three-dimensional image processing apparatus 200 according to the second embodiment shown in FIG. Alternatively, the projection unit 20 may be arranged in the vertical posture. Also by the head unit 1B of such arrangement, it is possible to pick up a pattern projection image in which the shadow of the work W is captured by similarly tilting the light projection direction and the imaging direction.
Third Embodiment

Furthermore, one or both of the light emitting means and the imaging means can be arranged in plural. For example, as in the three-dimensional image processing apparatus 300 shown in FIG. 4A as the third embodiment, while holding the imaging unit 10 in a vertical posture with respect to the work W, two light projection units 20 centering on the imaging unit 10 It can also be arranged on both sides and configured as a head unit 1C that emits light from the left and right respectively. As described above, by imaging the pattern projection images different in the direction of light projection, there is a state where the pattern projection image can not be partially imaged, such as the shadow pattern is hidden by the work W itself when light projection from one direction It is possible to reduce the situation where height measurement becomes inaccurate or impossible. In particular, by disposing the light projection means 20 so as to emit light from a direction (for example, left and right or front and back) facing the workpiece, the possibility of being blocked by the workpiece itself and being unable to image can be significantly reduced.
Embodiment 4

  Further, in the above example, the configuration in which one imaging unit is one and two projection units is described, but conversely, two imaging units and one projection unit may be used. Such an example is shown in FIG. 4B as a three-dimensional image processing apparatus 400 according to the fourth embodiment. In the head portion 1D shown in this example, the light projecting means 20 is held in a vertical posture with respect to the work W, and the imaging means 10 is disposed in an inclined posture with respect to the work W on the left and right in the figure. Even in this configuration, since the workpiece W can be imaged from different inclination angles, it is possible to suppress a situation in which the pattern projection image is partially difficult to be imaged similarly to the third embodiment. Further, according to this method, it is possible to simultaneously capture two pattern projection images by one light projection, so that an advantage that processing time can be shortened can be obtained.

  On the other hand, even if the same work is imaged from different angles by two imaging means, since the site being imaged, the field of view, etc. are different, it is necessary to match the position of each pixel, which may cause errors. is there. On the other hand, according to the third embodiment described above, by making the imaging means common, images of the same field of view can be taken regardless of which of the light emitting means emits the measurement light. It is possible to eliminate the need for various integration tasks, and to avoid the occurrence of errors associated with integration tasks, thus providing the advantage of simplifying processing.

In the above example, the example in which the imaging unit 10 and the light projection unit 20 are integrally configured in each head unit has been described, but the present invention is not limited to this configuration. For example, the imaging unit 10 and the light projection unit 20 can be a head unit configured by separate members. Moreover, it is also possible to provide three or more imaging means and light projection means.
(Block Diagram)

Next, FIG. 5 is a block diagram showing the configuration of a three-dimensional image processing apparatus 300 according to Embodiment 3 of the present invention. As shown in FIG. 5, the three-dimensional image processing apparatus 300 includes a head unit 1 and a controller unit 2.
(Head part 1)

The head unit 1 includes a light projecting unit 20, an imaging unit 10, a head side control unit 30, a head side operation unit 31, a storage unit 38, a head side communication unit 36, and the like. The light projector 20 includes a measurement light source 21, a pattern generator 22, and a plurality of lenses 23, 24, 25. The imaging means 10 includes a camera and a plurality of lenses (not shown).
(Lighting means 20)

  The light projection means 20 is a member for projecting incident light as structured illumination of a predetermined projection pattern from an oblique direction with respect to the optical axis of the imaging means. The projector 20 can be used as the light projector 20, and includes a lens as an optical member, a pattern generator 22 and the like. The light projecting means 20 is disposed obliquely above the position of the stationary or moving workpiece. The head unit 1 can also include a plurality of light projecting means 20. In the example of FIG. 5, the head unit 1 includes two light projecting means 20. Here, the first projector 20A (right side in FIG. 5) capable of emitting measurement illumination light to the work from the first direction and the measurement for the work from the second direction different from the first direction The second projectors 20B (left side in FIG. 5) capable of emitting illumination light are respectively disposed. The first projector 20A and the second projector 20B are disposed symmetrically with respect to the optical axis of the imaging unit 10. The measurement light is alternately projected to the work from the first projector 20A and the second projector 20B, and the pattern of each reflected light is imaged by the imaging unit 10.

  For example, a halogen lamp that emits white light or a white LED (light emitting diode) that emits white light can be used as the measurement light source 21 of each of the first projector 20A and the second projector 20B. The measurement light emitted from the measurement light source 21 is appropriately condensed by a lens, and then enters the pattern generation unit 22.

  Furthermore, in addition to the light projecting means for emitting measurement light for acquiring a pattern projection image for generating a distance image, an observation illumination light source for capturing a normal optical image (luminance image) can be provided. . As the observation illumination light source, in addition to the LED, a semiconductor laser (LD), a halogen light, an HID or the like can be used. In particular, when an element capable of imaging in color is used as the imaging element, a white light source can be used as the observation illumination light source.

The measurement light emitted from the measurement light source 21 is appropriately condensed by the lens 113, and then enters the pattern generation unit 112. The pattern generation unit 22 can realize illumination of an arbitrary pattern. For example, the pattern can be inverted according to the color of the work or background, such as white for black, black for white, etc., so that an appropriate pattern that is easy to view or measure can be expressed. For example, a DMD (digital micro mirror device) can be used as such a pattern generation unit 22. The DMD can turn on / off a minute mirror for each pixel to express an arbitrary pattern. Thereby, it is possible to easily irradiate a pattern in which white and black are inverted. By using a DMD for the pattern generation unit 22, an arbitrary pattern can be easily generated, and the preparation of a mechanical pattern mask and the replacement operation thereof can be omitted, so that there is an advantage that the apparatus can be miniaturized and quick measurement can be performed. . In addition, since the pattern generation unit 112 using the DMD can be used in the same manner as normal illumination by irradiation of a full illumination pattern in which all pixels are turned ON, it can also be used for capturing a luminance image. The pattern generation unit 22 can also be an LCD (liquid crystal display), LCOS (Liquid Crystal on Silicon: reflective liquid crystal element) or a mask. The measurement light incident on the pattern generation unit 22 is converted into a preset pattern and a preset intensity (brightness) and emitted. The measurement light emitted by the pattern generation unit 22 is converted by the plurality of lenses into light having a diameter larger than the field of view that can be observed and measured by the imaging unit 10, and then the work is irradiated.
(Imaging means 10)

  The imaging unit 10 includes a camera for acquiring reflected light which is projected by the light projecting unit 20 and reflected by the work WK and captures a plurality of pattern projection images. For such a camera, a CCD, a CMOS or the like can be used. In this example, a monochrome CCD camera capable of obtaining high resolution is used. It goes without saying that it is also possible to use a camera capable of color imaging. In addition to the pattern projection image, the imaging unit can also capture a normal luminance image.

  The head-side control unit 30 is a member for controlling the first projector 20A and the second projector 20B, which are the imaging unit 10 and the light projecting unit 20. For example, the head unit side control unit 30 creates a light projection pattern for the projection unit 20 to project measurement light onto the workpiece to obtain a pattern projection image. As a result, a projection pattern for phase shift is projected from the light projection means 20 by the imaging means 10 to pick up a phase shift image, and a projection pattern for space coding is projected from the light projection means 20 to obtain a space code image. Take an image. As described above, the head side control unit 30 functions as a light projection control unit for controlling the light projection unit so that the imaging unit 10 picks up the phase shift image and the space code image.

  The head side operation unit 31 includes a filter processing unit 34 and a distance image generation unit 32. The distance image generation unit 32 generates a distance image based on the plurality of pattern projection images captured by the imaging unit 10.

  The head-side storage unit 38 is a member for holding various settings, images, and the like, and can use storage elements such as a semiconductor memory and a hard disk. For example, it includes a luminance image storage unit 38 b for holding a pattern projection image captured by the imaging unit 10 and a distance image storage unit 38 a for holding a distance image generated by the distance image generation unit 32.

The head side communication means 36 is a member for communicating with the controller unit 2. Here, the controller side communication means 42 of the controller unit 2 is connected to perform data communication. For example, the distance image generated by the distance image generation unit 32 is sent to the controller unit 2.
(Distance image generating means 32)

  The distance image generation unit 32 is a unit that generates a distance image in which the gray value of each pixel changes in accordance with the distance from the imaging unit 10 that images the workpiece WK to the workpiece WK. For example, in the case of generating a distance image by the phase shift method, the head side control unit 30 controls the light projecting means 20 so that the sinusoidal stripe pattern is projected on the workpiece with phase shift, The head-side control unit 30 controls the imaging unit 10 so as to capture a plurality of images in which the phase of the wave stripe pattern pattern is shifted. And the head side control part 30 calculates | requires the phase of a sine wave for every pixel from several sheets of image, and produces | generates a distance image using the calculated | required phase.

  When a space image method is used to generate a distance image, the space irradiated with light is divided into a large number of fan-shaped small spaces of a large number of cross sections, and a series of space code numbers are given to the small spaces. For this reason, even if the height of the work is high, in other words, even if the height difference is large, the height can be calculated from the space code number as long as it is in the space where the light is irradiated. Therefore, the shape can be measured over the entire height of the workpiece.

  By thus generating a distance image on the head unit side and sending it to the controller unit side, the amount of data to be sent from the head unit to the controller unit side can be reduced, and processing delays that may occur due to a large amount of data transfer It can be avoided.

In the present embodiment, the generation process of the distance image is performed on the head unit 1 side, but for example, the generation process of the distance image may be performed on the controller unit 2 side. Further, gradation conversion from the distance image to the low gradation distance image may be performed by the head unit side in addition to the controller unit. In this case, the head side operation unit 31 implements the function of the gradation conversion unit.
(Controller part 2)

The controller unit 2 further includes a controller communication unit 42, a controller control unit, a controller operation unit, a controller storage unit, a test execution unit 50, and a controller setting unit 41. The controller side communication unit 42 is connected to the head side communication unit 36 of the head unit 1 to perform data communication. The controller-side control unit is a member for controlling each member. The controller side operation unit realizes the function of the image processing unit 60. The image processing unit 60 implements the functions of the image search unit 64, the gradation conversion unit 46, and the like.
(Gradation conversion means)

  The tone conversion means 46 tone converts the high tone distance image into a low tone low tone distance image based on the distance image (detailed procedures will be described later). In this way, the distance image having the height information generated by the head unit can be represented as a two-dimensional gray-scale image that can be handled by existing equipment, which can contribute to measurement processing and inspection processing. In addition, there is also an advantage that the load can be distributed by sharing the distance image generation processing and the gradation conversion processing between the head unit and the controller unit. In addition to the generation of the distance image on the side of the head unit, the generation of a low gradation distance image may also be performed. Such processing can be performed by the head side operation unit. As a result, the load on the controller unit side can be further reduced to enable efficient operation.

  Furthermore, the tone conversion means does not perform tone conversion on the entire distance image, but preferably selects only necessary portions to perform tone conversion. Specifically, only the portion corresponding to the inspection target area set in advance by the inspection target area setting unit (details will be described later) is subjected to gradation conversion. By doing this, the processing required to convert a multi-gradation distance image into a low-gradation distance image can be limited to the inspection target area only, so that the load required for gradation conversion can be reduced. This also contributes to the shortening of the processing time. That is, by shortening the processing time, it can be suitably used even in an application with a limited processing time such as inspection of FA application, and real time processing is realized.

  The controller-side storage unit is a member for holding various settings and images, and a semiconductor storage element, a hard disk or the like can be used.

  The controller-side setting unit 41 is a member for performing various settings for the controller unit, receives an operation from the user via the input unit 3 such as a console connected to the controller unit, and sends necessary conditions to the controller side. To direct. For example, the functions of the gradation conversion condition setting unit 43, the reference surface setting unit 44, the space coding switching unit 45, the interval equalizing processing setting unit 47, the light projection switching unit 48, the shutter speed setting unit 49, etc. are realized.

  The reference plane setting means 44 is used as a tone conversion parameter constituting a tone conversion condition when performing tone conversion to convert the distance image received by the controller side communication means 42 into a two-dimensional low tone distance image. A reference plane to perform this gradation conversion is set based on the distance image. Based on the reference plane set by the reference plane setting means 44, the gradation conversion means 46 converts the distance image into height information of the number of gradations lower than the number of gradations of this distance image as the gray value of the image. The gradation conversion is performed to the replaced low gradation distance image.

The inspection execution unit 50 performs predetermined inspection processing on the low gradation distance image subjected to gradation conversion by the gradation conversion unit 46.
(Hardware configuration)

  Next, a hardware configuration example of the controller unit 2 is shown in a block diagram of FIG. The controller unit 2 shown in this figure has a main control unit 51 that performs numerical calculation and information processing based on various programs and controls each component of hardware. The main control unit 51 stores, for example, a CPU as a central processing unit, a work memory such as a RAM that functions as a work area when the main control unit 51 executes various programs, and a start program, an initialization program, etc. And a program memory such as a flash ROM or an EEPROM.

  In addition, the controller unit 2 is connected to the head unit 1 including the imaging unit 10, the light projection unit 20, etc., and controls the light projection unit 20 to project the sine wave stripe pattern on the workpiece with a phase shift. A controller-side connection unit 52 for capturing image data obtained by imaging with the imaging unit 10, an operation input unit 53 to which an operation signal from the input unit 3 is input, and a display unit 4 such as a liquid crystal panel And a communication control unit 55 communicably connected to an external PLC 70 or a personal computer PC, etc., a RAM 56 for holding temporary data, and settings Controller-side storage means 57 for storing contents and data set by a three-dimensional image processing program installed in the personal computer PC are held. Image processing unit 60 including an auxiliary storage means 58 for calculation, an arithmetic DSP for executing measurement processing such as edge detection and area calculation, and a predetermined inspection based on the processing result in the image processing unit 60 etc. An output unit 59 or the like for outputting the result of the operation is provided. These pieces of hardware are communicably connected via an electrical communication path (wiring) such as a bus.

  In the program memory in the main control unit 51, control for controlling each unit of the controller side connection unit 52, the operation input unit 53, the display control unit 54, the communication unit 55, and the image processing unit 60 by a command of the CPU or the like The program is stored. Further, the above-described processing order program, that is, the processing order program generated by the personal computer PC and transferred from the personal computer PC is stored in the program memory.

  The communication unit 55 functions as an interface (I / F) that receives an imaging trigger signal from the PLC 70 when there is a trigger input by a sensor (such as a photoelectric sensor) connected to the external PLC 70. It also functions as an interface (I / F) that receives the three-dimensional image processing program transferred from the personal computer PC, layout information that defines the display mode of the display unit 4, and the like.

  When the CPU of the main control unit 51 receives an imaging trigger signal from the PLC 70 via the communication unit 55, the CPU transmits an imaging command (command) to the controller side connection unit 52. Further, based on the processing order program, a command instructing image processing to be executed is transmitted to the image processing unit 60. As a device for generating an imaging trigger signal, not the PLC 70 but a sensor for trigger input such as a photoelectric sensor may be directly connected to the communication unit 55.

  The operation input unit 53 functions as an interface (I / F) that receives an operation signal from the input unit 3 based on the user's operation. The display unit 4 displays the operation content of the user using the input unit 3. For example, in the case of using a console as the input means 3, components such as a cross key for moving the cursor displayed on the display means 4 up, down, left and right, an enter button, or a cancel button can be arranged. By operating each of these parts, the user can create a flowchart defining the processing order of image processing on the display means 4, edit parameter values of each image processing, and set a reference area. , And can edit the reference registration image.

  The controller side connection unit 52 captures image data. Specifically, for example, when an imaging command of the imaging unit 10 is received from the CPU, an image data capture signal is transmitted to the imaging unit 10. Then, after imaging is performed by the imaging unit 10, image data obtained by imaging is captured. The captured image data is temporarily buffered (cached) and substituted into an image variable prepared in advance. Note that “image variable” refers to a variable that is a reference destination of measurement processing and image display by being allocated as an input image of a corresponding image processing unit, unlike a normal variable that handles numerical values.

  The image processing unit 60 executes measurement processing on image data. Specifically, the controller-side connection unit 52 first reads the image data from the frame buffer while referring to the above-described image variable, and performs internal transfer to the memory in the image processing unit 60. Then, the image processing unit 60 reads the image data stored in the memory and executes the measurement process. The image processing unit 60 further includes a gradation conversion unit 46, an abnormal point highlight unit 62, an image search unit 64, and the like.

  The display control unit 54 transmits a control signal for causing the display unit 4 to display a predetermined image (video) based on the display command (display command) sent from the CPU. For example, in order to display image data before or after measurement processing, a control signal is transmitted to display means 4. The display control unit 54 also transmits a control signal for causing the display unit 4 to display the content of the user's operation using the input unit 3.

The head unit 1 and the controller unit 2 configured by hardware as described above are configured such that each means and function of FIG. 5 can be realized as software by various programs and the like. In this example, a three-dimensional image processing program is installed in the computer shown in FIG. 1, and the setting necessary for three-dimensional image processing is performed.
(Gradation conversion)

  The above three-dimensional image processing apparatus acquires a distance image of a workpiece, performs image processing on the distance image, and performs an inspection on the result. The three-dimensional image processing apparatus according to the present embodiment uses the existing hardware to add information such as an area, an edge, etc., in addition to the height inspection process that uses the height information that is the pixel value of the distance image as it is. Two types of inspections can be performed in image inspection processing that performs calculations using. Here, in order to maintain the accuracy of the height inspection process, it is necessary to generate a multi-tone distance image. On the other hand, existing hardware can not perform image inspection processing on such multi-gradation distance images. Therefore, in order to perform image inspection processing using existing hardware, gradation conversion is performed on a multi-gradation distance image to generate a low gradation distance image.

  However, if the height information of the multi-gradation distance image is converted as it is into the low-gradation distance image, there is a problem that the accuracy of the height information is lost. Many common images used in FA applications and the like are monochrome images in which gray-scale values of each pixel are expressed by eight gradations. On the other hand, as the distance image, a high gradation image such as a 16 gradation image is used. For this reason, when performing gradation conversion of a multi-gradation distance image to a low-gradation distance image, the height information is considerably lost, which affects the inspection accuracy. However, in order to raise the number of gradations of the image handled in the existing image processing in order to improve the accuracy, the introduction cost rises, the processing load becomes high, and the hurdle to use becomes high.

Therefore, in the case of such gradation conversion, it is necessary to set conditions for the gradation conversion such that necessary height information is maintained. The method and procedure will be described in detail below.
(Height inspection or image inspection)

  First, the processing operation of performing the height inspection process using the three-dimensional image processing apparatus will be described based on the flowchart of FIG. 7. The three-dimensional image processing apparatus uses a height inspection processing tool that performs height inspection on a distance image as a tool for performing calculation processing, and various image inspection processing that performs image inspection on an existing luminance image. Equipped with tools. Here, the height inspection process will be described.

  First, a distance image is generated (step S71). Specifically, the distance image generation unit 32 generates a distance image using the imaging unit 10 and the light projection unit 20. Next, a desired calculation process is selected (step S72). Here, select the tools required for the calculation process.

  When the image inspection processing tool is selected, the process proceeds to step S73, and the tone conversion process is performed on the high tone distance image obtained in step S71 to convert it into a low tone distance image. As a result, the inspection processing tool provided in the existing image processing apparatus can handle low gradation distance images. Note that the gradation conversion processing is not performed on the entire area of the high gradation distance image, but is preferably performed only in the inspection target area set for the image inspection processing.

On the other hand, when the height inspection tool is selected, the process proceeds to step S74 without performing gradation conversion in order to use the height information of the multi-gradation distance image as it is.

Further, the inspection execution means 50 performs various kinds of calculation processing (step S74), and then, based on the calculation result, determines whether or not the workpiece is a non-defective product (step S75). When the inspection execution means 50 determines that the work is non-defective (step S75: YES), the determination signal output means 160 outputs an OK signal as a determination signal to the PLC 70 (step S76), and the inspection execution means 50 If it is determined that the work is not a non-defective product, that is, a defective product (step S75: NO), an NG signal is output to the PLC 70 as a determination signal (step S77).
(Setting mode)

Next, prior to the execution of such height inspection and image inspection processing, an example of a procedure in a setting mode for performing various settings on the three-dimensional image processing apparatus will be described based on the flowchart of FIG. First, in step S81, an image for setting (image for setting) is selected. Here, it is possible to input an image to be subjected to inspection processing in advance and to call an image stored as a registered image, or to acquire a new input image and perform setting on this. Here, the input image obtained by imaging a work is registered as a registered image, as an alternative showing the input image sequentially input at the time of operation. Alternatively, a registered image registered in advance may be recalled. Next, in step S82, the gradation conversion method is selected. Here, the user is prompted to select either static conversion or dynamic conversion. Next, in step S83, tone conversion parameters are adjusted. Here, when static conversion is selected in step S82, tone conversion parameters are adjusted with respect to the image acquired in step S81. The adjustment method of the gradation conversion parameter will be described later. In addition, the procedure demonstrated above is an example, and can also be made into a different order. For example, acquisition of an image may be performed after selection of the gradation conversion method.
(Details of setting procedure)

Next, details of the setting procedure will be described. In the three-dimensional image processing apparatus, prior to the operation mode, necessary settings are made in advance in the setting mode. Various setting means for performing such setting can be provided, for example, on the controller unit 2 side. For example, in the example of FIG. 1, a console which is one form of the input unit 3 connected to the controller unit 2 can be used. Also, instead of or in addition to this, the function of the setting means can be realized by the three-dimensional image processing program installed in the personal computer connected to the controller unit 2 as described above. Hereinafter, here, the user interface (GUI) screen of the 3D image processing program shown in FIG. 9 to FIG. It explains based on. In these examples of GUI, the distance image is displayed as “height image” and the luminance image is displayed as “grayscale image”.
(Registration process of distance image and luminance image)

First, the distance image and the luminance image are registered. Here, the “imaging” processing unit 263 is set from the initial screen 260 of the 3D image processing program shown in FIG. Specifically, the button of the button 263 of the “imaging” processing unit is pressed. This switches to the imaging setting menu 269 in FIG.
(Three-dimensional image processing program)

  In the GUI screen example of FIG. 10, a first image display area 111 for displaying an image is provided on the right side of the screen, and a setting item button area 112 where a plurality of buttons representing a plurality of setting items are arranged is provided on the left. In the setting item button area 112, an "image registration" button 113, an "imaging setting" button 284, a "camera setting" button, a "trigger setting" button, a "flash setting" button, a "lighting volume" button, a "lighting expansion unit" Button, "save" button, etc. are provided. The user can select a desired setting item button from the setting item button area 112 and can set necessary setting items.

  When an “image registration” button 113 provided in the setting item button area 112 in the imaging setting menu 269 in FIG. 10 is pressed, the screen is switched to the image registration screen 270 in FIG. From this screen, various settings such as registration target, camera selection, registration destination, etc. can be performed. Here, after adjustment to a desired image by various buttons and the like provided in the operation area, registration, that is, storage of image data is performed. Here, the distance image is displayed in the second image display area 121, and the image variable assigned to the image is displayed in the operation area 122. In the example of FIG. 11, "Camera 1" is selected in the "Camera selection" column 271 for selecting an imaging means, and the image variable "& Cam 1 Img" is "Camera selection" as the distance image captured by this "Camera 1". Is displayed below the column 271.

When setting is completed, pressing the “Register” button 272 provided at the lower part of the operation area starts registration of the distance image currently displayed in the second image display area 121 as shown in FIG. 12, and the progress is graphically displayed. Is displayed. In addition, as shown in FIG. 13, registration of a luminance image is also performed. In this example, the distance image is first stored in the distance image storage unit 38a, and then the luminance image is stored in the luminance image storage unit 38b. Further, the image variable “& Cam1Img” of the distance image and the image variable “& Cam1GrayImg” of the luminance image are also recorded. These image variables can be used as an index when calling up a registered image, since a unique variable is assigned to each image. However, this example is an example, and the registration order of each image may be reversed or registered simultaneously. Thus, the user can save labor of registration of each image by simultaneously storing the distance image and the luminance image as the registered image. However, it is also possible to separately register the distance image and the luminance image as the registration image.
(Phase shift method)

Here, a phase shift method will be described as one of methods for measuring the displacement and three-dimensional shape of a workpiece without contact. The phase shift method is also called grating pattern projection method, fringe scanning method or the like. In this method, a light beam having a lattice pattern in which the illuminance distribution is sinusoidally varied is projected onto a work. Moreover, the projection is performed with three or more lattice patterns having different phases of sine waves, and each brightness value of the height measurement point is imaged for each pattern from an angle different from the projection direction of the light beam, Calculate the phase value of Depending on the height of the measurement point, the phase of the grating pattern is projected onto the measurement point, and the light beam having a phase different from that observed by the light beam reflected at the reference position is observed. Therefore, the phase of the ray at the measurement point is calculated, and the height of the measurement point (that is, the object) is measured by substituting the geometric relationship of the optical device using the principle of triangulation, and the three-dimensional shape is determined. It is a method. According to the phase shift method, the height of the work can be measured with high resolution by reducing the grating pattern period, but the height range that can be measured is a low height within 2π in phase shift amount. It can measure only the thing (small thing of the height difference).
(Space coding method)

  Therefore, the space coding method is also used. According to the space coding method, a space to be irradiated with light is divided into a large number of substantially fan-shaped small spaces in cross section, and a series of space code numbers are assigned to the small spaces. For this reason, even if the height of the workpiece is high, that is, even if the height difference is large, the height can be calculated from the space code number as long as it is in the space where the light is irradiated. Therefore, the shape can be measured over the entire height of the workpiece. As described above, according to the space coding method, it is possible to measure the shape over the entire work even for a workpiece having a wide range (dynamic range) of the allowable height and a high height.

  Next, a setting example for capturing a distance image by the imaging unit will be described based on FIGS. When the “imaging setting” button 284 is pressed from the imaging setting menu 269 of FIG. 10 prior to the registration of the image described above, the imaging setting screen 280 of FIG. 14 is displayed. In the imaging setting screen 280. Basic settings for imaging can be performed. For example, when the "image pickup valid setting" button is pressed, the image pickup valid setting screen of FIG. For example, in place of or in addition to a monochrome CCD camera or a color CCD camera that picks up a normal luminance image as an imaging unit, a camera capable of acquiring height information is connected to perform three-dimensional image processing of the distance image It can be incorporated into the device. In addition, when connecting a plurality of imaging units to the three-dimensional image processing apparatus, it is possible to select one or more imaging units to be used.

Further, when the “detail setting” button 282 provided in the operation area is pressed from the imaging setting screen 280 in FIG. 14, a three-dimensional measurement setting screen 290 shown in FIG. 16 is obtained. In FIG. 16, different works are displayed for the convenience of description. The three-dimensional measurement setting screen 290 shown in FIG. 16 includes a "display with continuous update" field 292 corresponding to real time updating means, a shutter speed setting field 294 corresponding to shutter speed setting means 49, gray range setting field 296, pre-processing setting field 310 It is provided with a disabling standard setting field 312, a uniform interval process setting field 314, a space code setting field 316, a projector selection setting field 318, a "display image" selection field 322 and the like.
(Real time update means)

  Here, real-time updating means is provided for updating the image being displayed on the second image display area 121 to the setting after the change when the setting is changed in the operation area. The real time updating means can switch ON / OFF. In the screen example of FIG. 16, the real-time update function can be operated by setting “display in continuous update” column 292 provided in the upper stage of the operation area as one form of the real-time update unit to ON.

In the example of FIG. 16, the items that can be set in the operation area include shutter speed, gray range, pre-processing, non-measurable reference, equal interval processing, space code, projector selection, display image and the like. These will be sequentially described below.
(Shutter speed setting means 49)

A shutter speed setting field 294 is provided in the example of FIG. 16 as an aspect of the shutter speed setting unit 49 for adjusting the shutter speed at the time of imaging by the imaging unit. The shutter speed can be designated by the user from the shutter speed setting field 294. Here, the shutter speed set in advance from the drop-down box, for example 1/15, 1/30, 1/60, 1/120, 1/240, 1/500, 1/1000,, 1/20000 Choose The number of seconds corresponding to the selected numerical value is displayed in the numerical value display field 295 on the right. Also, any shutter speed can be directly specified by a numerical value. For example, when “input numerical value” is selected as the option of the drop-down box, the graying out of the numerical value display column 295 is canceled and it becomes possible to directly input a numerical value. Thus, based on the numerical value designated in the shutter speed setting field 294, the exposure time of the camera (image pickup element) which is the image pickup means is adjusted. Note that when the shutter speed is adjusted, it is easier to perform the confirmation operation if the gray image of the luminance image is displayed in the second image display area 121 rather than the distance image. Furthermore, the image whose shutter speed has been changed in the shutter speed setting field 294 is promptly reflected in the second image display area 121 by the above real time update function, so that the user can visually confirm whether the current setting is appropriate. It can be confirmed and adjustment can be easily performed.
(Light / dark range setting field 296)

In the density range setting field 296, the dynamic range of the luminance image which is a density image is adjusted. Here, the dynamic range is increased or decreased by selecting any one of “low (−1)”, “normal (0)”, and “high (1)” from the drop-down box.
(Pre-processing setting column 310)

In the pre-processing setting field 310, common filtering processing performed before the distance image is generated in the head unit is defined. As common filter processing, for example, filters of averaging filters, median filters, Gaussian filters, etc. can be considered. Here, as filter processing for the pattern projection image, in the example of FIG. 17, any one of “none”, “median”, “Gaussian” and “average” is selected by the drop-down box. The filter processing can be performed on the distance image obtained on the head unit 1 side as well as the pre-processing performed before generating the distance image.
(Unmeasurable standard setting column 312)

  In the measurement impossible reference setting column 312, the level at which the noise component is cut is set. That is, the height measurement is not performed by the amount set in the measurement impossible standard setting column 312. In the measurement of three-dimensional height information using a pattern projection image, accurate height information can not be measured without a certain amount of light. On the other hand, when multiple reflection occurs, it is necessary to reduce the light amount because it is too bright. As described above, the noise component cut amount is selected according to the captured pattern projection image. Specifically, in order to calculate height information of each pixel, a threshold value to be regarded as invalid data due to noise is determined.

  Here, as shown in FIG. 18, “High”, “Medium”, “Low”, or “None” is selected from the drop-down box. If “none” is selected, noise component cutting is not performed, and height measurement is performed on all pixels. For example, in the example shown in FIG. 19, “None” is selected in the measurement impossibility criteria setting column 312, and height data is calculated at all points including noise data. Although it is difficult to distinguish from this screen, an incorrect height is measured by noise data at a corner portion of the work or the like.

  On the other hand, in the example shown in FIG. 20, “Low” is selected in the measurement impossible standard setting column 312, and the point where the height information based on the noise data is incorrect decreases. Furthermore, in the example shown in FIG. 21, “Medium” is selected in the measurement impossibility criteria setting column 312, and the point of height information is further reduced.

On the other hand, it can also be confirmed from FIG. 21 that the black points indicating that measurement is impossible increase particularly in the lower left region of the workpiece, and the positions where it is considered as noise and height measurement can not be increased. Furthermore, the example shown in FIG. 22 shows a state in which “high” is selected in the measurement impossibility criteria setting column 312, and as a result of excessive removal of noise components, data that is originally intended to be left is lost. You can confirm that. As described above, when the setting of the unmeasurable reference indicating the noise removal threshold is too low, the height is calculated based on the noise. On the other hand, if it is too high, the part that you want to leave will be regarded as invalid. Therefore, the user adjusts the setting of the immeasurable reference by using the above-mentioned real time update function, confirms the adjusted image in the second image display area, and directly refers to the result of the setting by the image, It can be adjusted to an appropriate value.
(Equal interval processing setting column 314)

  In the uniform interval process setting field 314, the error due to the angle of view is corrected. The uniform interval process setting section 314 functions as an interval equalization process setting unit 47. In the uniform interval process setting field 314, ON and OFF can be selected as shown in FIG. By setting the uniform interval process to ON, distance images arranged at equal pitches in the xy direction are acquired. Here, the second image display area 121 displays an equal-pitch image in which the positions in the X and Y directions are equally spaced regardless of the height (Z coordinate). For example, in an application where inspection of dimensions in the XY plane is to be performed, it is necessary to turn on even spacing processing. In addition, the part which is corrected and loses data is treated as invalid. FIG. 24 and FIG. 25 show a state in which the uniform interval process is ON. In the example of FIG. 24, the example which made the "display image" selection column 322 be displayed on the 2nd image display area 121 as a "height image", ie, a distance image, FIG. 25 selects a "grayscale image" and selects a luminance image. The example displayed is shown respectively.

On the other hand, when the uniform interval processing is turned off, an image (Z image) as viewed with eyes becomes as shown in FIG. 26, and distortion occurs in the X and Y directions as it proceeds to the edge of the screen. However, since the uniform interval processing is not performed, the time required to display the image can be shortened. FIG. 27 shows an example in which “brightness image” is selected in the display image column and the luminance image is displayed in the second image display area 121.
(Space code setting field 316)

  In the space code setting field 316, the use of the space coding method is selected. That is, the space code setting field 316 functions as the space coding switching means 45. In this three-dimensional image processing apparatus, the phase shift method is essential for the generation of the distance image, and in addition to the phase shift method, the presence or absence of application of the space coding method can be selected in the space code setting field 316. In the space code setting field 316, ON and OFF can be selected as shown in FIG. When the space code setting field 316 is turned on, height measurement is performed by a combination of the space coding method and the phase shift method. This example is shown in FIG. 29 and FIG. In these figures, FIG. 29 shows a state in which a distance image is selected as an image to be displayed in the second image display area 121. Specifically, the “height image” is selected in the “display image” selection field 322. On the other hand, FIG. 30 shows a state in which a luminance image is displayed in the second image display area 121, and "grayscale image" is selected in the "display image" selection column 322. By using the space coding method in addition to the phase shift method, an appropriate distance image can be obtained. Specifically, the spatial coding method can correct the phase jump (phase unwrapping) by the phase shift method, so that it is possible to measure with high resolution while securing a wide dynamic range of the height. However, the imaging time is approximately doubled as compared with the case of OFF.

  On the other hand, as shown in FIGS. 31 and 32, when OFF is selected in the space code setting field 316, height measurement is performed only by the phase shift method. In this case, since the measurement dynamic range of the height is narrowed, in the case of a work having a large difference in height, the height can not be measured correctly if the phase is shifted by one or more cycles. On the contrary, in the case of a work having a small change in height, since imaging and combining of a fringe image by the space coding method are not performed, the processing can be speeded up by that amount, and the imaging time can be halved. For example, when measuring a workpiece with little difference in height direction, it is not necessary to take a large dynamic range, so shortening the processing time while maintaining high precision height measurement performance even with the phase shift method alone. Can. In this case, since the measurement dynamic range of the height is narrowed, in the case of a work having a large difference in height, the height can not be measured correctly if the phase is shifted by one or more cycles. On the other hand, in the case of a work having a small change in height, turning off the space code has the advantage that the imaging time can be halved.

  The example of FIG. 31 shows a state in which the “height image”, ie, the distance image is displayed in the “display image” selection column 322, while in the example of FIG. "" Is displayed.

Although the phase shift method is essential in this example, ON / OFF of the phase shift method may be selectable.
(Projector selection setting field 318)

  The projector selection setting field 318 functions as a projection switching unit 48 that switches ON / OFF of the first projector and the second projector. Here, in the projector selection setting field 318, the projector (projector) to be used is selected from the two projectors, the first projector and the second projector. In the example of the projector selection setting column 318, as shown in FIG. 33, from the drop down box, “1” (first projector), “2” (second projector), “1 + 2” (first projector and second Select one of the projectors).

  When "1" or "2" is selected in the projector selection setting column 318, that is, in the case of a single light projection which is a light projection from either one of the first projector or the second projector, a portion to be shaded by the light Height measurement is not performed. In the example of FIG. 34, an example in which “1” is selected in the projector selection setting field 318 and an example in which “2” is selected in the example of FIG. 35 are shown. In each screen, the data of the part to be a shadow is displayed in black, and it can be confirmed from each screen that there is an area where measurement of height can not be performed on the work. Further, as is apparent from these figures, it can be seen that the area where measurement can not be made differs depending on the light projection means. In other words, even if it becomes an unmeasurable area by one of the light projecting means, the other light projecting means can emit light, and thus there are many areas where height measurement is also possible. Therefore, by combining these, the unmeasurable area can be reduced. In particular, the first projector and the second projector arrange the work so as to sandwich the work from both sides, so that the first light projection from the first projector and the second light projection from the second projector face each other. Since the workpiece is irradiated, even in a region that is a shadow in any one light projection, it is possible to reduce the risk of being a shadow by the light projection from the other in the opposite direction.

Specifically, as shown in FIG. 36, when "1 + 2" is selected in the projector selection setting field 318, switching is made to both light projection to be made to emit light from both the first projector and the second projector. In this state, even if it is a portion which is a shadow by the light projection from one of the projectors, if the other projector can project the light, it can be interpolated. However, in this case, an imaging time of about twice as long as single projection is required. The user selects which light projection to use according to the degree of unevenness of the workpiece to be inspected, the allowable imaging time, and the like.
("Display image" selection column 322)

In the “display image” selection field 322, an image to be displayed in the second image display area 121 is selected. For example, by selecting the display target according to the application of the inspection, the appropriateness of each setting can be visually confirmed from the actually displayed image. In particular, by setting the above-mentioned real-time updating means to ON, changes in the setting can be sequentially updated and compared before and after the change, so the setting can be adjusted based on the image to be an image intended according to the application. . In addition, even a beginner who is not familiar with the meaning of each setting parameter has an advantage of being able to set while looking at the image. In this example, as shown in FIG. 37, from the “display image” selection column 322, “height image”, “grayscale image”, “overexposure and overexposure image”, “stripe light projector 1”, “stripe light” Select either "Projector-Projector 2". The “height image” is a distance image, and displays an image in which colors are colored in contour lines for each height and colored. The "grayscale image" is a luminance image. In this example, an image obtained by combining a plurality of pattern projection images captured based on the phase shift method is used as a luminance image. However, it is also possible to illuminate the work and to capture an optical image by the imaging means and use it as a luminance image.
(Abnormal point highlight means 62)

  The three-dimensional image processing apparatus further includes an abnormal point highlight means 62 as shown in FIG. For example, the “overexposure and underexposure image” selectable in the “display image” selection column 322 described above partially includes saturated and overexposed pixels, underexposure light amounts of underexposed pixels, and the like with respect to a luminance image. It is a colored image. In this way, by highlighting the portion in the image where accurate values are not obtained or the measurement accuracy seems to be unreliable by coloring processing, the portion with low measurement accuracy can be visually identified for the user. To make it easy to check whether an image suitable for the desired inspection application is obtained. In this example, the overexposure pixels are colored yellow and the overexposed pixels are colored blue. As a result, the user can visually check how the white-crushed area is distributed in the image, using the color as a clue. In the lower part of the second image display area 121, the number of overexposure pixels and underexposure pixels is counted and displayed. While referring to this, the user adjusts each setting item so that the number of pixels approaches 0.

  The color and mode to be colored are not limited to this, and various known modes such as displaying in other colors or blinking may be used as appropriate. Also, by changing the color of the overexposed pixels and the overexposed pixels, it is possible to notify the user of the reason why the measurement reliability is low, and it is also easy to take measures. However, the same color or highlight may be applied to the overexposure pixels and the underexposure pixels.

  The “stripe projection-projector 1” is a pattern projection image represented by shading obtained by pattern projection only with the first projector. Also, "stripe light projection-projector 2" is a pattern projection image obtained only by the second projector. 38 shows an example in which the pattern projection image of the first projector is displayed in the second image display area 121 by selecting “stripe light projector 1” in the “display image” selection column 322, as shown in FIG. The example which made the 2nd image display area 121 display the pattern projection image of a 2nd projector by selecting "stripe light projection-projector 2" is shown, respectively. Identifying the cause when the height of the work can not be measured by confirming the fringe image from this screen, for example, because the material of the work is translucent, to obtain a pattern projection, such as the projected light is embedded It can be confirmed from the original pattern projection image before generating the distance image that the work is difficult.

  The user refers to the image displayed in the second image display area 121, confirms whether the shutter speed and the gray range are appropriate, and adjusts the values to appropriate values. Specifically, in a state where the overexposure and underexposure images are displayed in the second image display area 121, the adjustment is performed while confirming that the overexposure and underexposure pixels are reduced. For example, by adjusting the shutter speed in the shutter speed setting field 294, it is made possible to eliminate black-flooded pixels, that is, a portion where the light amount is insufficient and too dark. In addition, by adjusting the density range, it is possible to eliminate overexposure pixels, that is, parts that are too bright. In the example of FIG. 16, the overexposure and underexposure images are too dark and therefore there are many underexposure pixels. Therefore, the shutter speed is adjusted.

  For example, when the shutter speed setting field 294 is set to "1/120" and the state shown in FIG. 16 is switched to "1/15" as shown in FIG. It is understood that it has become. However, the number of overexposure pixels has increased conversely. Therefore, as shown in FIG. 41, when the shutter speed is switched to "1/30", it is understood that the number of overexposure pixels is also reduced while the number of blackout pixels is 0.

  Furthermore, as shown in FIG. 42, when the lightness range setting column 296 is set to “low (-1)”, when switching to “normal (0)” and raising by one step as shown in FIG. Can be seen. Further, as shown in FIG. 43, when switching to “high (1)”, it can be understood that overexposure pixels become 0. As a result, the number of overexposure pixels and the number of overexposure pixels both become 0, and the adjustment of the shutter speed and the density range is completed.

As described above, after setting desired imaging conditions, as shown in FIGS. 12 to 13 described above, the distance image and the luminance image are registered, and the image registration operation is completed.
(Height measurement setting procedure)

  Next, a procedure for setting an area in which height measurement is performed on an input image of a workpiece to be inspected which is sequentially input during operation will be described based on FIGS. Here, a typical workpiece representing a plurality of workpieces to be input is registered in advance as a registered image according to the above-described procedure, and an area for height measurement is designated for the registered image.

Specifically, as shown in FIGS. 44 to 45, “height measurement” processing unit 266 is added from the initial screen 260 of FIG. In the example of FIG. 44, “Add” is selected from the first submenu 370 displayed by right clicking or the like in the lower part of the “imaging” processing unit 263 in the flow display area 261, and “measurement” in the second Among the inspection processes listed in the “Measurement” menu 373 displayed by selecting “,” a “height measurement” processing unit 266 for performing “height measurement” is added. As a result, as shown in FIG. 45, a “height measurement” processing unit 266 is newly added below the “imaging” processing unit 263 in the flow display area 261. Thus, the “measurement” menu 373 functions as an inspection process selection unit for selecting an inspection process to be executed by the inspection execution unit.
(Inspection area setting screen 120)

  Next, a procedure for setting an area as the setting to be performed by the “height measurement” processing unit 266 will be described based on FIGS. First, when the editing screen of the “height measurement” processing unit is called from the screen of FIG. 45, the screen shifts to a height measurement setting screen 460 shown in FIG. Also in the GUI screen example of FIG. 46, as in FIG. 10, a first image display area 111 for displaying an image is provided on the right side of the screen, and a setting item button area 112 where a plurality of setting item buttons are arranged is provided on the left. The setting item button area 112 includes an "image registration" button 113, an "image setting" button 114, an "area setting" button 115, a "preprocessing" button 117, a "detection condition" button 118, and a "detail setting" button 119, A “determination condition” button, a “display setting” button, a “save” button, and the like are provided. From this screen, when the user presses the “area setting” button 115 corresponding to the examination target area setting unit, the screen changes to the examination target area setting screen 120 shown in FIG. In the inspection target area setting screen 120, an area to be inspected can be designated. In the example of FIG. 47, the second image display area 121 is provided on the left of the screen, and the operation area 122 for performing various operations is disposed on the right of the screen. In the upper stage of the operation area 122, a “display image” selection field 124 for selecting an image displayed in the second image display area 121 is provided. In the example of FIG. 47, the registered image is selected in the “display image” selection field 124. Further below that, a “measurement area” setting field 126 is provided as an inspection target area setting means for specifying an area to execute an inspection.

  In the “measurement area” setting column 126, a predetermined area can be selected. Here, when the “measurement area” setting field 126 is selected, a drop down box is displayed as shown in FIG. 48, and a desired measurement area shape can be selected. In this example, “None”, “Rectangle”, “Rotating rectangle”, “Circle”, “Ellipse”, “Circle”, “Circle”, “Polygon”, “No”, “Rectangle”, “Circle”, “Circle”, "Composition area" etc. are displayed. If “none” is selected, the entire image displayed in the second image display area 121 is used as the examination target area.

  Further, in accordance with the shape selected in the “measurement area” setting field 126, setting of detailed dimensions and the like becomes possible. In the example of FIG. 48, an eraser is used for the work, and in the “measurement area” setting column 126, as shown in FIG. 49, an example in which “rotation rectangle” is selected is shown. When the “edit” button 128 is pressed in this state, a measurement area edit screen 130 shown in FIG. 50 is displayed. In the example of FIG. 50, in the second image display area 121, a rotation rectangle is displayed superimposed on the workpiece. Here, a rectangular measurement area is drawn on the portion of the case of the eraser, and is displayed superimposed on the distance image. Also, the basic vector of the rotation rectangle is displayed as an arrow in the frame shape of the rotation rectangle, and the width and height of the rotation rectangle, the XY coordinates of the center, the inclination angle of the basic vector, etc. Is displayed. The user can adjust the shape, position, and the like of the rotation rectangle as desired by directly inputting numerical values from the measurement area editing screen 130 or operating the handle displayed in the rotation rectangle with a mouse or the like.

The items that can be set in the measurement area editing screen 130 change according to the shape selected in the “measurement area” setting field 126. For example, when “circumference” is selected, as shown in FIG. 51, it is possible to set parameters related to the circumference, such as designation of the dimensions of the outer diameter and the inner diameter of the circumference. Furthermore, it is also possible to specify a mask area from the screen of FIG. As the mask area, a circular shape, a donut shape, a rectangular shape or another polygonal shape, a free curve, or the like can be designated. In this way, according to the shape of the workpiece to be inspected, the inspection target area can be appropriately set, and an area unnecessary for inspection such as a perforated portion or a background can be excluded to improve processing efficiency. .
(Second measurement display area)

  When the measurement area is set in this manner, the measurement area that has already been set is displayed superimposed on the workpiece as shown in FIG. Subsequently, when specifying another measurement area, the same operation is repeated. That is, as shown in FIG. 52, another second “height measurement” processing unit 266B is added to the lower part of the “height measurement” processing unit 266 in the flow display area 261. Then, as shown in FIGS. 53 and 54, a rotation rectangle is set as a new measurement area on an area not covered with the case of the eraser, which is a work here. As a result, as shown in FIG. 55, the newly set second measurement area is displayed superimposed on the workpiece.

In the above example, one height measurement process is performed by one “height measurement” processing unit. That is, in order to perform a plurality of height measurement processes, it is necessary to add a plurality of "height measurement" processing units. However, it goes without saying that a plurality of height measurement processes can be performed in one "height measurement" processing unit.
(Measurement process)

When the setting of the measurement area is completed in this way, processing for actually performing measurement is added next. Here, as shown in FIG. 56, a “numerical operation” processing unit that performs “operation” is added to the lower part of the second “height measurement” processing unit 266B in the flow display area 261. As the contents of the operation executed by the “numeric operation” processing unit, numerical operation, image operation, calibration, image connection, etc. can be selected. Here, as shown in FIG. 57, a "numerical operation" processing unit in which numerical operation is selected is added.
("Mathematical unit" processing unit)

  In the "numerical operation" processing unit, specific arithmetic expressions can be input. For example, as shown in FIG. 58, a numerical operation editing screen on which a mathematical expression can be directly input is displayed, and the user defines an arithmetic expression. Here, a calculator-like input pad is prepared, and editing buttons for copying, cutting, pasting, and the like are also prepared to facilitate creation of an arithmetic expression. The user inputs a desired arithmetic expression from this screen. An example of the input arithmetic expression is shown in FIG.

When the contents of the numerical calculation process are defined in this manner, an arithmetic expression is displayed on the third image display area 262 on the initial screen 260 as shown in FIG.
("Area" processing unit)

Furthermore, in the example of FIG. 61, the "area" processing unit is added under the "numerical operation" processing unit. In the "area" processing unit, conditions for actually performing pass / fail determination and the like are defined. Specifically, in order to appropriately change the gradation conversion condition for converting the distance image into a low gradation distance image according to the registered image and the input image, the gradation conversion parameter (details will be described later) An area for acquiring reference information, a condition for extracting a height from this area, a condition for performing a filter process when generating a distance image, and the like are set. That is, in the “area” processing unit, area setting, height extraction, pre-processing, determination and the like are set. First, the procedure for setting the area is the same as that of the registered image described above. That is, when the "area setting" button 115 arranged in the setting item button area 112 is pressed from the area setting screen 620 as shown in FIG. 62, the area setting screen shown in FIG. Here again, a rotation rectangle is selected, and further detailed coordinates and the like are designated as necessary. In this way, the area in the "area" processing unit is determined, and the rotation rectangle is superimposed and displayed on the work in the second image display area as shown in FIG.
(Height extraction setting screen)

  Next, set the height extraction. The setting of height extraction is to set tone conversion parameters at the time of tone conversion. That is, when the “height extraction” button 116 is pressed in the setting item button area 112 of FIG. 62, the screen shifts to the height extraction selection screen 140 shown in FIG. 65, and a display image, an extraction method, etc. can be selected. Also in the height extraction selection screen 140, the second image display area 121 is provided on the left of the screen and the operation area 122 for performing various operations is disposed on the right of the screen, as in FIG. In the upper stage of the operation area 122, a “display image” selection field 124 for selecting an image displayed in the second image display area 121 is provided. In the example of FIG. 65, the registered image is selected in the “display image” selection field 124. Further below that, extraction method selection means 142 for selecting an extraction method of the height extraction function is provided.

Here, the “height extraction” button 116 functions as a tone conversion condition setting unit 43 that sets a tone conversion parameter for performing tone conversion of the distance image by the tone conversion unit. In particular, the tone conversion condition setting unit 43 is displayed when the inspection process selection unit selects a process that does not require image height information. On the contrary, when the process requiring the height information of the image is selected by the inspection process selection means, the gradation conversion condition setting means is not displayed. Specifically, when the “height measurement” processing unit 266 is selected as the inspection processing tool, the “height extraction” button is not displayed in the flow display area 261. For other inspection processing tools, such as “area” processing unit or “blob” processing unit 267, “color inspection” processing unit 267B, “Shapetrax2” processing unit 264, “position correction” processing unit 265, etc. An “extract” button 116 is displayed, and the gradation conversion condition can be set. By doing this, the tone conversion condition setting means 43 is displayed when tone conversion is necessary, and the user is prompted to make necessary settings, while when the tone conversion is not necessary, the tone conversion By making the means for setting the conditions itself non-display, it is possible to prevent the user from being confused by unnecessary settings, and a user-friendly environment is realized.
(Extraction method selection means 142)

The extraction method selection unit 142 specifies a gradation conversion method. For example, the user is made to select either static conversion or dynamic conversion. In the example of FIG. 65, either “one-point designation” or “three-point designation (plane)” corresponding to static conversion or “real-time extraction” corresponding to dynamic conversion is selected from the drop-down box as an option in advance. Let
(One point specification screen 150)

  When "one point designation" is selected from the extraction method selection means 142 on the screen of FIG. 65, the screen shifts to a one point designation screen 150 of FIG. In addition, in FIG. 66-FIG. 96, the example using 50 yen ball as a workpiece | work is shown for description. In the one-point specification screen 150 in FIG. 66, the height of the part specified on the second image display area 121 is set as the reference height (reference height). In the example of FIG. 66, when the “extract” button 144 provided in the operation area 122 at the right of the screen is selected, the screen shown in FIG. 67 is displayed, and an arbitrary position on the second image display area 121 at the left of the screen can be specified. It will be. Here, the height extraction unit is used to designate a position at the center of the height in the work displayed in the second image display area 121. In the example of FIG. 67, the height extraction means is constituted by an “extract” button 144 displaying a dropper-like icon SI. When the “extract” button 144 is pressed, a point on the second image display area 121 is displayed. Pointer 146 is displayed. The position designated by the pointer 146 is registered as an intermediate height of the distance range.

  Further, a range for obtaining the height around the point designated by the pointer 146 can be designated by the “extraction area” designation column 145. In the “extraction area” designation column 145, one side of the area for which the average height is to be obtained is designated by the number of pixels. In the example of FIG. 67, “16” is specified in the “extraction area” specification column 145, and the average height within the area of 16 pixels × 16 pixels centered on the point specified by the pointer 146 is extracted , And the height extracted by the pointer 146. The size of the area designated by the pointer 146 on the second image display area 121 can be changed in conjunction with the numerical value designated in the “extraction area” designation column 145.

Further, height information of the designated part is displayed as a numerical value in the “Z height” display field 152 (in the example of FIG. 68, “1.253” is displayed in the “Z height” display field 152). For example, when the distance range is expressed by 2 ⁸ = 256 gradations (0 to 255), a height (density value / mm; details will be described later) is 128 with the height specified by the height extraction means as its center It is set to be With this configuration, the user directly designates on the screen the height to be inspected, so that gradation conversion is performed to a low gradation distance image within a range centered on the specified height. It is possible to avoid the situation where the information is lost.
(Simple display function)

As described above, when the distance range and the span are determined as tone conversion parameters necessary for tone conversion, it becomes possible to tone convert a high tone distance image into a low tone distance image. Also, as shown in FIG. 68, a low gradation distance image having undergone gradation conversion under the gradation conversion conditions currently set in the current operation area 122 is simply displayed on the second image display area 121. . In addition, when the tone conversion condition is changed in the operation area 122, the simple display of the low tone distance image after the tone conversion in the second image display area 121 is accordingly made according to the tone conversion condition after the change. Will be updated. As a result, the user can visually confirm the change in the gradation conversion condition visually and promptly, and can easily perform the adjustment operation by trial and error. As described above, the image displayed in the second image display area 121 is a mode in which the distance image before gradation conversion is displayed, a mode in which the low gradation distance image after gradation conversion is displayed, and a normal luminance image You can change it by switching the mode.
(Gain adjustment means)

  Furthermore, the user can perform gain adjustment, which is one of the gradation conversion parameters, using the gain adjustment unit. In the example of FIG. 68, an emphasizing method setting field 154 is provided in the middle stage of the operation area 122, and a gain adjustment field 156 is disposed as a gain adjusting means here. The current gain is displayed numerically in the gain adjustment field 156. Here, the gain [gradation / mm] is a parameter corresponding to a span at the time of gradation conversion. For example, when converting a distance image of 16 gradations to 8 gradations, it is set as how many gradations out of 8 gradations are to be converted per 1 mm. If the gain value is increased, gradation conversion with clear contrast is obtained. For example, when the gain value is set to 100 [gradation / mm], gradation conversion is performed so as to be 0.010 mm per gradation. Also, assuming that the height information of the distance image before conversion has a resolution of 0.00025 mm per gradation, the difference between the obtained reference surface and the input height data is N gradations before conversion. After conversion, it can be calculated as N [tone] × 0.00025 [mm / tone] × 100 [tone / mm] = N × 0.025 tone.

  Here, the reference plane is a plane obtained by a method such as one-point specification, an average height reference described later, three-point specification, a plane reference, a free-form surface reference, etc. For example, as shown in FIG. 69A, when the cross-sectional profile of the distance image (input image) before conversion of 16 gradations has a shape shown by a solid line, the reference plane is indicated by a dashed line. A profile of a low tone distance image obtained by tone converting such an input image from 16 tones to 8 tones is as shown in FIG. 69B, and the gain (conversion coefficient) as it is with respect to the difference from the reference plane It will be in a state where it

Further, the height per gray level (reciprocal number of gain value) can be automatically calculated according to the above-described gain value, and can be displayed together. In the example of FIG. 68, 250 [gradation / mm] is displayed as the gain value, and 0.0040 mm is displayed as the height per gradation. The user can adjust the gain value by changing the gain value. For example, when the gain value is increased, as shown in the screen of FIG. 68 to the screen of FIG. 70, the height difference can be emphasized to finely inspect the height information, but the height range which can be inspected becomes narrow. Conversely, if the gain value is lowered, inspection can be performed to a wide height range as shown in FIG. 71, but fine changes are lost. As described above, when the gain is adjusted by the gain adjustment unit, the tone-converted image obtained under the tone conversion condition can be confirmed in the second image display area 121. As a result, the user can adjust to an appropriate gain value according to the inspection purpose etc. while checking the tone-converted image updated in real time.
(Setting extraction height)

  Furthermore, the setting method of the emphasizing method can include setting of the extraction height as well as the gain value. For example, in the screen of FIG. 72, when the “detail setting” button 158 provided at the lower right of the operation area 122 is pressed, the screen switches to the highlighting method detail setting screen 160 of FIG. In addition to 156, an "extraction height" setting field 162 is displayed. In the "extraction height" setting column 162, as height information to be extracted by the height extraction means, any of high height information, low height information, and height information of both high and low included in the area You can choose Here, as shown in FIG. 74, either “high side”, “low side”, or “both high and low” can be selected by the drop-down list provided in the “extraction height” setting field 162. For example, when “high side” is selected, gradation conversion is performed so that the height of the position designated by the pointer 146 becomes the lower limit of the distance range. As a result, a low gradation distance image is generated in which only the side higher than the specified height is extracted. Similarly, when “low side” is selected, gradation conversion is performed so that the height of the position designated by the pointer 146 becomes the upper limit of the distance range. As a result, a low gradation distance image is generated in which only the side lower than the specified height is extracted. Furthermore, in the case of “both high and low”, gradation conversion is performed so that the height of the position designated by the pointer 146 is in the middle of the above-described distance range. Pixels out of the range after gradation conversion are clipped to the low side at black (pixel value 0 in the case of eight gradations) and to the high side at white (pixel value 255).

Further, in the emphasizing method detail setting screen 160 of FIG. 73, a noise removal setting field 164 for removing noise and an invalid pixel designation field 166 for designating a value to be given to invalid pixels are also provided.
(Noise removal setting field 164)

  In the noise removal setting field 164, as one of the tone conversion parameters, it is specified how many mm of the difference from the reference surface should be removed as noise. For example, when the noise removal parameter is set to 0.080 mm, the difference of 0.080 mm from the reference surface is removed. Here, assuming that the pre-conversion height information has a resolution of 0.00025 mm per gradation, the difference of 0.080 [mm] /0.00025 [mm] = 320 [gradation] is ignored. It becomes. This situation will be described based on FIGS. 75A to 75C. In these figures, as in FIG. 69A, the cross-sectional profile of the distance image before conversion of 16 gradations is indicated by a solid line, its reference plane is indicated by a dashed line, and the noise removal range is indicated by an alternate long and short dash line. There is. As a result of performing noise removal on such an input image with reference to the reference plane, a profile shown in FIG. 75B is obtained. Furthermore, the profile of the low gradation distance image obtained by converting the gradation of 16 gradations to 8 gradations with respect to the distance image of FIG. 75B is as shown in FIG. 75C, and gains (conversion coefficients) for the remaining components. It will be in the state where it applied.

  Further, effects of gain adjustment and noise removal will be described based on FIGS. 76A to 76F. First, it is assumed that a luminance image as shown in FIG. 76A and a distance image of high gradation (16 gradations) as shown in FIG. 76B are obtained. Here, with the distance image of FIG. 76B, the low gradation (eight gradations) remains as it is initially set (here, the gain is 100 [gradation / mm] and the noise removal is 0.000 [mm]). FIG. 76C shows a low gradation distance image subjected to gradation conversion to. The low tone distance image has relatively low contrast. Therefore, when the gain is increased from this state, as shown in FIG. 76D, the low-gradation distance image with high contrast is newly gradation-converted from FIG. 76B and displayed. However, the noise component is also large in this image. In the example of FIG. 76D, the gain is set to 1000 [gradation / mm] and the noise removal is set to 0.000 [mm]. Accordingly, a low gradation distance image in which the noise removal amount is increased from FIG. 76D is shown in FIG. 76E. Here, the gain is set to 1000 [gradation / mm] and the noise removal is set to 0.080 [mm]. As a result, although the noise component was reduced, it can be confirmed that noise lower than the reference plane is present at the upper right of the upper left “E”. Therefore, if the “extraction height” setting column 182 shown in FIG. 74 and the like is set to “high side”, the reference surface is converted to the lowest value (pixel value 0), so the portion lower than the reference surface is As a result of being ignored and only the side higher than the reference plane being extracted, a low gradation distance image as shown in FIG. 76F is obtained. For example, when “extraction height” is set to “high side” for a distance image (16 tones) of a profile as shown in FIG. 77A, the result is that only the high side from the reference plane is extracted, A low gradation distance image (eight gradations) as shown in FIG. 77B is obtained. In this way, it is possible to obtain a low gradation distance image having high contrast and less noise components as compared to FIG. 76C and the like. In this example, as the final tone conversion parameters, set the gain to 1000 [tone / mm], the noise removal to 0.080 mm, and the “extraction height” to the “high side”, as shown in the figure. Gradation conversion is performed from the distance image 76B to the low gradation distance image in FIG. 76F.

  In this manner, when the conditions necessary for performing one-point specification are set, the input image is shifted from the high-gradation distance image to the low-gradation distance image according to the designated gradation conversion condition, that is, the reference height. The tone is converted and displayed in the first image display area 111 as shown in FIG.

Here, an example of a work in which the method of designating the reference plane by one point designation is effective will be described based on FIGS. 79A to 79B. FIG. 79A shows a workpiece WK7 which has no flat inclination or slight inclination on the measurement surface of the workpiece and does not affect inspection processing. Here, an inspection process of reading whether the character string is appropriate or not is performed on the workpiece WK7 in which numbers and character strings are three-dimensionally formed on the surface of the casting. In such an application, pressing the “extract” button 144 shown in FIG. 67 or the like causes the dropper-like icon SI to be displayed on the second image display area 121. Then, as shown in FIG. 79A, one point of the flat surface (background surface) on which the character string is not formed is designated by the pointer 146 as shown in FIG. 79A. Accordingly, tone conversion is performed with the height of the extraction area (16 pixels in the example of FIG. 67) designated by the pointer 146 as a reference plane, and converted to the low tone distance image shown in FIG. 79B. In this low gradation distance image, since the character string portion protruding from here is clearly extracted with the plane of the work WK 7 as the background, accurate OCR can be easily performed. Thus, one-point specification can be effectively used in the case where there is no influence on inspection processing even if the work is slightly inclined. There is also an advantage that one point specification can be processed at low load and at high speed.
(3 points specified)

  The above described the setting method of the gradation conversion conditions by one-point specification. Next, a method of setting gradation conversion conditions by three-point specification will be described based on the GUI screens of FIGS. The three-point specification is a method of performing gradation conversion of a distance image into a low gradation distance image using a plane obtained from three points specified by the user as a reference plane. The reference plane is also, for example, the height of the middle of the height range (distance range) in which the gradation conversion is performed to the low gradation distance image among the height information of the distance image, similarly to the reference height of one point designation. Alternatively, it may be set to the upper limit (the highest position where gradation conversion is performed) or the lower limit (the lowest position where gradation conversion is performed) of the distance range.

In the state of shifting to the height extraction selection screen 140 shown in FIG. 80 by pressing the “height extraction” button 116 on the GUI screen of the three-dimensional image processing program of FIG. , Select "Three points specified (plane)". As a result, a three-point designation screen 170 for setting height extraction shown in FIG. 81 is displayed.
(Three-point specification screen 170)

  In the three-point specification screen 170, three reference planes as reference for tone conversion are specified and set on the second image display area 121. For this reason, in the three-point designation screen 170 of FIG. 81, height extraction means is provided. Specifically, by selecting the “extract” button 144 provided in the operation area 122 at the right of the screen, the screen shown in FIG. 82 is displayed, and three arbitrary positions are selected on the second image display area 121 at the left of the screen. You will be able to specify points. Here, a point-like pointer 146 is displayed as height extraction means as in FIG. 67, and the user sequentially designates a desired position with a pointing device such as a mouse, a track ball, or a touch panel. First, when the first point is designated on the second image display area 121, the rectangular shape changes into a cross shape at the designated position as shown in FIG. 83 to indicate the designated position, and the second point, Similarly, it can be designated by the pointer 146. At this point, the color distance image displayed in FIG. 82 is subjected to gradation conversion on the basis of the horizontal plane including the specified first point height, and the low gradation distance image after gradation conversion is converted to a gray scale image Is displayed in the second image display area as When the second point is further designated, the position of the second point changes from rectangular to cross as shown in FIG. 84, and the third point can be designated. At this point in time, tone conversion is performed again on the basis of the inclined surface including the heights of the two designated points, and the low tone distance image is updated. When a third point is specified, the reference plane is set with a plane including these specified three points. Further, when each point is specified by the height extraction means, the height of the point currently specified on the height extraction screen display area may be displayed in the “Z height” display field 152.

  Furthermore, an inclination angle can be displayed as the information of the reference surface. In the example of FIG. 84, in the height extraction display field 172 provided in the operation area 122, the X-direction inclination, the Y-direction inclination, and the third Z-direction height of the reference surface are displayed.

  Also, as with single-point designation, it is also possible to designate an emphasis method as needed. For example, gain adjustment is performed using the gain adjustment means, or the three-point specification “detail setting” button 174 is pressed to call the three-point specification detail setting screen 180 as shown in FIG. In addition to the gain adjustment column 156, the "extraction height" setting column 182 is displayed, and as height information to be extracted by the height extraction means, "high side", "low side", "both high and low" from the drop down list You can choose one.

  In this way, it is possible to perform tone conversion of the distance image with an arbitrary plane defined by the specified three points as a reference plane. As a result, not only tone conversion based on a horizontal plane such as the one-point designation described above, but also tone conversion using an inclined plane as a reference plane becomes possible. For example, in an application where a flaw or foreign matter on a sloped surface of a work surface is inspected, the distance range narrows with the slope as it is, and the sloped surface can be canceled by setting the reference plane along the sloped surface. It can detect scratches and foreign matter. In this way, it is possible to realize flexible gradation conversion utilizing height information according to the work and inspection purpose.

  Here, an example of a work in which the method of designating the reference plane by three points designation is effective will be described based on FIGS. 86A to 86D. FIG. 86A shows a work WK8 that affects the result of the inspection process when there is a planar inclination or a slight planar inclination in the measurement surface of the work. Here, inspection processing for detecting a ball grid array (BGA) formed on a substrate is performed. In such an application, pressing the “extract” button 144 shown in FIG. 84 or the like causes the dropper-like icon SI to be displayed on the second image display area 121. Then, as shown in FIG. 86A, three points on the upper surface of the workpiece WK8 where the BGA is not formed are designated by the pointer 146. As a result, a plane including the three designated points is extracted as a reference plane, gradation conversion is performed, and a low gradation distance image shown in FIG. 86B is converted. In this low gradation distance image, the BGA protruding from the surface of the work WK 8 is clearly extracted with the background as a background, so that it can be binarized as shown in FIG. 86C to confirm the shape of the BGA. This method has an advantage that accurate detection can be performed even if the plane of the workpiece WK8 is inclined. If the workpiece WK8 shown in FIG. 86A has an inclination, for example, at one-point designation, a binarized image becomes as shown in FIG. 86D and can not be detected correctly. On the other hand, in the three-point specification, the inclination is corrected as described above to obtain an accurate detection result. Thus, three-point designation is effective in the case where the inclination of the plane affects the inspection processing result.

  Further, when there is a gentle depression on the upper surface of a planar work WK9 as shown in FIG. 87A, an inspection process for detecting this depression will be considered. Here, as shown in FIG. 87A, the measurement region ROI is set to a region including a depression. As a result, gradation conversion is performed using the plane obtained from height data of the entire measurement region ROI including the depression as a reference plane, and a low gradation distance image as shown in FIG. 87B is obtained. In this example, the reference plane is estimated by the method of least squares. Further, the obtained low gradation distance image is binarized to obtain a binarized image shown in FIG. 87C. By this, the inclination can be corrected and only the portion of the depression can be extracted stably. If a binarized image is obtained by designating one point in a state where there is inclination, the result is as shown in FIG. 87D, and it can be understood that detection of a depression becomes difficult due to the inclined surface. Thus, three-point designation is effective for highly accurate reference surface estimation. In terms of processing time, although processing time takes longer than one-point designation, processing can be performed relatively quickly.

The above has described static conversion in which tone conversion conditions are specified in advance at the setting stage, and tone conversion is performed under the conditions specified during operation. In other words, in static conversion, the gradation conversion parameter has a constant value regardless of the input image. Next, a specific example of dynamic conversion in which the gradation conversion condition is adjusted in accordance with the input image to be inspected will be described. First, in dynamic conversion, as a specific method for correcting the reference of height information to be left when performing gradation conversion of a distance image to a low gradation distance image,
(B1) an average height reference for performing gradation conversion using an average height (average distance) in an average extraction area designated for an input image as an average reference height;
(B2) A plane reference that generates an estimated plane in a designated area of an input image and performs tone conversion using this as a reference plane,
(B3) A free-form surface reference is included which generates a free-form surface obtained by removing high-frequency components from an input image and performs tone conversion using this as a reference surface. Each method will be sequentially described below.
(B1: average height standard)

  The average height reference is a method of calculating an average height within a designated average extraction area for each input image and performing tone conversion using this as the average reference height. The average extraction area for defining the average reference height is previously set prior to operation (step S83 in FIG. 8 described above). Hereinafter, an example of a procedure for specifying an average extraction area in step S83 of FIG. 8 will be described based on the GUIs of FIG. 62, FIG. 65, and FIG. 88 to FIG. First, select the “height extraction” button 116 on the GUI screen of FIG. 62, proceed to the height extraction selection screen 140 of FIG. 65, and select “real time extraction” corresponding to dynamic conversion by the extraction method selection means 142 Then, the height dynamic extraction setting screen 190 of FIG. 88 is displayed. In this example, an example using an eraser for the work is shown again. Next, in the “calculation method” selection column 192 provided below the extraction method selection means 142, the dynamic conversion criteria are specified. Here, as shown in FIG. 89, one of “average height reference”, “plane reference”, and “free-form surface reference” is selected in the drop-down box. Here, "average height reference" is selected. As a result, the screen moves to the average height reference setting screen 210 of FIG. In addition, in FIG. 90 and FIG. 91, the example which used the 50 yen coin for work is shown on account of description.

  When setting for dynamic conversion, it is necessary to set the average reference height or the like for an image different from the distance image input during actual operation. Therefore, an image corresponding to the work at the time of operation is captured in advance and stored as a registered image, and the registered image is read in setting of the dynamic conversion, and this is substituted for the work image at the time of operation Make various settings in the form. Therefore, in the screen of FIG. 65 and the like, the corresponding “registered image” is designated in the “display image” selection field 124.

In the average height reference setting screen 210, an inspection target area set separately is used as it is, or an arbitrary average extraction area is specified as needed. For designation of the average extraction area, any method such as designation of a rectangular shape or designation of four corners, a circle by designation of a center and a radius, or a free curve can be used. Alternatively, only one point of the work may be designated, or conversely, the entire work or the entire image displayed in the second image display area 121 may be used as the average extraction area. Alternatively, the inspection target area separately designated as described above can also be used as the average extraction area. In these cases, the work of specifying the average extraction area by the height extraction means may be omitted.
(Mask area)

  In addition, it is also possible to specify a mask area in which the average height is not extracted with respect to the average extraction area. For example, when the “extraction area” button 194 provided in the operation area 122 is pressed from the screen of FIG. 90, the screen shifts to a mask area setting screen 220 shown in FIG. From the mask area setting screen 220, one or more mask areas unnecessary for extraction of the average height can be designated. The mask area can also be specified by specifying an arbitrary area such as a rectangular shape or a circular shape from above the second image display area 121 as described above.

  Furthermore, gain adjustment can be performed as needed. For example, when the "detailed setting" button 196 provided at the lower right of the operation area 122 is pressed from the screen of FIG. 90, the screen moves to the average height reference detailed setting screen 230 shown in FIG. In addition to, detailed setting items such as extraction height specification and noise removal are displayed.

When the average extraction area is defined in this manner, the setting screen is ended. At the time of gradation conversion, gradation conversion is performed with the average value (average reference height) of the height information included in the average extraction area as a reference height. For example, tone conversion is performed so that the average reference height is the center value of the distance range (128 in the case of 2 ⁸ = 256 tones, the center value of the distance range of 0 to 255). Further, it is not necessary to use all the height information of all the points included in the average extraction area, and the processing can be simplified by appropriately thinning out or averaging.

  At the time of operation, dynamic conversion is performed according to the procedure shown in FIG. 133 described later. For example, the workpiece conveyed on the line is imaged to generate a distance image (step S13301), the average height of the average extraction area set above is calculated (step S13302), and tone conversion is performed based thereon The low gradation distance image is generated to generate a low gradation distance image (step S13303), and the obtained low gradation distance image is inspected (step S13304). With this method, even if there is a variation in the height direction of the workpiece, the reference plane of gradation conversion can be reset each time for each workpiece, so that an accurate inspection is realized regardless of the variation in the height direction of the workpiece. it can.

Here, an example of a work in which the method of designating the reference plane by the average height standard is effective will be described based on FIGS. 93A to 93B. FIG. 93A, like FIG. 79A described above, is a workpiece WK7 that does not affect the inspection processing even if there is no planar inclination or slight inclination on the measurement surface of the workpiece, and the numbers and character strings are on the surface of the casting An inspection process is performed to read whether the character string is appropriate or not by using the three-dimensionally formed workpiece WK7. In such an application, a measurement area ROI for determining an average height reference in a rectangular shape is set on the second image display area 121 on the extraction area setting screen shown in FIG. Here, as shown in FIG. 93A, a plane surrounding the character string on the upper surface of the workpiece WK7 is designated as the measurement region ROI. Thus, tone conversion is performed with the height of the measurement region ROI as a reference plane, and converted to a low tone distance image shown in FIG. 93B. Even in this low gradation distance image, as in the case of FIG. 79B, since the character string portion protruding from the flat surface of the work WK7 is clearly extracted as a background, accurate OCR can be easily performed. Also, since only one point on the workpiece WK7 is specified in FIG. 79A described above, the selected point may be affected by noise, whereas such a noise is specified because a plane is specified in FIG. 93A. It has the advantage of being able to reduce the effects of
(Plane reference)

  The above has described an example in which the dynamic conversion is performed on the basis of the average height. Next, as another dynamic conversion, a plane reference included in a reference surface estimation region designated in advance with respect to an input image is estimated, and a plane reference in which this estimation surface is used as a reference surface is described. In this method, for example, when the surface of the workpiece is inclined, since the inclination component can be canceled and gradation conversion can be performed, an advantage can be obtained which can be utilized similarly to the three-point designation of static conversion described above. Hereinafter, a specific setting method of the plane reference will be described. Also on the plane reference, as in the above-described average height reference, the reference surface estimation area for determining the reference surface is previously set prior to operation (step S83 in FIG. 8 described above). Hereinafter, an example of a procedure for specifying a reference surface estimation area in step S83 of FIG. 8 will be described based on the GUIs of FIG. 62, FIG. 88, and FIG. 92 to FIG. First, select the “height extraction” button 116 on the GUI screen of FIG. 62, proceed to the height extraction selection screen 140 of FIG. 80, and select “real time extraction” corresponding to dynamic conversion by the extraction method selection means 142 Then, the height dynamic extraction setting screen 190 of FIG. 88 is displayed. Next, in the “calculation method” selection column 192, when “plane reference” is selected as the reference of dynamic conversion as shown in FIG. 89, the screen is shifted to the plane reference setting screen of FIG.

  In the plane reference setting screen, similarly to the height extraction means in the height extraction selection screen 140 of FIG. 80 described above, a separately set inspection target area is used as it is, or an arbitrary reference plane estimation area is designated as necessary. . For designation of the reference surface estimation area, any method such as a rectangular shape, designation of four corners, a circle by designation of a center and a radius, or a free curve can be used. Alternatively, only one point of the work may be designated, or the entire work or the entire image displayed in the second image display area 121 may be used as the reference surface estimation area.

  Further, it is also possible to designate a mask area for which the estimated surface is not estimated with respect to the reference surface estimated area, as in the case of FIG. 90 and the like described above. Furthermore, it is also the same as described above that it is possible to perform gain adjustment, designation of extraction height, noise removal and the like as needed. For example, when the “detail setting” button 222 provided in the operation area 122 is pressed on the screen of FIG. 92, the plane reference detail setting screen 240 of FIG. 94 is displayed, and the emphasis method setting column 154 is added to the gain adjustment column 156, The “extraction height” setting column 162 for specifying the extraction height, the noise removal setting column 164 for noise removal, and the invalid pixel specification column 166 for specifying invalid pixels are displayed, and these detailed settings can be made. Become. Further, in the invalid pixel specification column 166, as shown in FIG. 95, in addition to the specified value, the invalid pixel for which the distance can not be obtained can be filled with the background pixel value or an arbitrary value specified by the user. it can.

  When the reference surface estimation area is defined in this manner, the setting screen is ended. At the time of gradation conversion, a planar estimation surface is calculated from the height information included in the reference surface estimation area. A known method such as the least squares method can be appropriately used for fitting the height information distributed in the reference surface estimation region. Note that the height information of all points included in the reference surface estimation area does not necessarily have to be used for calculation of the estimation surface, and the processing can be simplified, such as appropriately thinning out or averaging, as described above. It is.

  When the estimated surface is determined in this way, tone conversion is performed on the basis of this estimated surface. For example, gradation conversion is performed such that the estimated surface is at the center value of the distance range. Also, it is possible to display information of the calculated estimated surface. For example, in the example shown in FIG. 92, in the estimated surface display field provided in the operation area 122, the X direction inclination of the estimated surface, the Y direction inclination, and the Z direction height of the estimated surface are displayed. In this example, the estimation surface is one plane, but it may be an estimation surface in which a plurality of planes are combined. Further, although the estimation surface is a plane in this example, it is also possible to calculate the estimation surface as a simple curved surface such as a spherical surface.

Moreover, FIG. 86A and FIG. 87A which were mentioned above are mentioned as an example of a workpiece | work in which the method of designating a reference plane by plane reference is effective. By setting a plane reference to such a work, even if the upper surface of the work is inclined, the inclination is corrected and the reference plane is detected. Therefore, as with the three-point specification described above, an accurate BGA Pattern detection is realized.
(B3: free-form surface reference)

  Finally, a free-form surface reference will be described in which a free-form surface formed by removing high-frequency components from a predetermined area (free-form surface target area) of an input image is generated and is converted to a tone using this. For example, when approximation is difficult in a simple plane, such as when a workpiece has a curved surface, it is difficult to accurately extract height information of a region to be inspected. Therefore, by generating a simplified image by removing high-frequency components from the input image and using the surface shape (free-form surface) of this image as a reference plane, the rough shape and the gentle change are ignored, and an abrupt change is generated. It is possible to make an inspection that leaves only a part that is changing, that is, a fine shape.

  Hereinafter, a specific setting method of the free-form surface reference will be described based on the GUIs of FIGS. Also in the free-form surface reference, as in the case of the average height reference described above, necessary conditions for determining the reference surface are set in advance prior to operation (step S83 in FIG. 8 described above). First, select the “height extraction” button 116 on the GUI screen of FIG. 62, proceed to the height extraction selection screen 140 of FIG. 80, and select “real time extraction” corresponding to dynamic conversion by the extraction method selection means 142 Then, the height dynamic extraction setting screen 190 of FIG. 88 is displayed. Next, in the “calculation method” selection column 192, as shown in FIG. 89, “free-form surface reference” is selected as a reference of dynamic conversion. As a result, the screen moves to the free curved surface reference setting screen 250 of FIG.

Although it is possible to designate any area as the free curved surface target area also in the free curved surface reference setting screen 250, preferably, the whole of the image displayed in the second image display area 121 or the separately specified inspection target The area is used as a free-form surface target area as it is. A harmonic component is removed from the area designated as a free-form surface target area to generate a free-form surface. Then, the tone conversion image subjected to tone conversion using the free curved surface as a reference surface is superimposed and displayed on the free curved surface target region displayed on the second image display region 121. Further, at the time of gradation conversion, gain adjustment, designation of extraction height, noise removal and the like can also be performed as necessary, as in the above-mentioned FIG. 90 and the like.
(Extraction size adjustment means)

  Furthermore, it has an extraction size adjustment function for adjusting the fineness (extraction size) of the extraction surface extracted based on the free-form surface. Specifically, in FIG. 96, an “extraction size” designation column 252 is provided as an extraction size adjusting means in the “detail setting of extraction surface” column of the operation column. When the numerical value of the “extraction size” designation column 252 is increased or decreased, the curvature of the free-form surface changes accordingly, and the size of the extractable defect changes. Here, a free-form surface image is generated and displayed in the second image display area 121 so that the unevenness having a size equal to or less than the set size is extracted. When the extraction size is increased, a smooth free-form surface is generated, and defects having a size corresponding to the set extraction size can be extracted. On the other hand, when the extraction size is reduced, a free-form surface along the surface shape of the workpiece is generated, and only small defects corresponding to the set extraction size are extracted. For example, when the extraction size is increased, as shown in FIG. 97, the free-form surface becomes smooth with respect to the surface shape of the workpiece, so that the extracted unevenness becomes clear with the smoothed reference surface. Conversely, if the numerical value is reduced, as shown in FIG. 90, the free-form surface approaches a detailed shape along the unevenness of the surface shape of the workpiece, and as a result, the unevenness extracted with such a complex reference surface is unclear Become. In addition, when the numerical value in the “extraction size” designation column 252 is increased or decreased, the tone conversion image in the free curved surface target area displayed in the second image display area 121 is also internally generated as a reference plane accordingly. The state of the free-form surface changes, and the size of the object extracted and displayed as a difference from the reference surface changes in real time. The user can optimally adjust the numerical value of the “extraction size” designation field 252 while referring to the second image display area 121.

  When the free-form surface target area is defined in this manner, the setting screen is ended. At the time of gradation conversion, a free-form surface is calculated from the height information of the image included in the free-form surface target area. As shown in the flowchart of FIG. 161, the image reduction processing (step S1611), the filter processing (step S1612), and the image enlargement are performed for the fitting of the height information distributed in the free curved surface target area, as shown in the flowchart of FIG. The method (step S1613) can be used to generate a free-form surface image. For example, a median filter can be used for the filtering process. When the free-form surface is determined in this manner, the unevenness and the like to be extracted can be clarified by taking the difference from the original image (step S1614). Alternatively, known methods such as the least squares method can be appropriately used as described above. Note that it is not necessary to use the height information of all points included in the free curved surface target area for the calculation of the estimation surface, and as described above, the processing can be simplified by appropriately thinning out or averaging. When the free-form surface is determined in this way, tone conversion is performed on the basis of the free-form surface. For example, gradation conversion is performed so that the free-form surface has the center value of the distance range.

Here, an example of a work in which the method of designating the reference plane as a free-form surface is effective will be described based on FIGS. 98A to 98B. FIG. 98A shows an inspection process for detecting defects such as protrusions and depressions included in a curved surface of a workpiece. Here, a region including a defect is specified from the curved surface as a measurement region ROI. Then, a free-form surface is obtained from height information included in the designated measurement region ROI, and gradation conversion is performed using the obtained free-form surface as a reference surface. The obtained low gradation distance image is shown in FIG. 98B. As shown in this figure, with respect to the free-form surface, a portion higher than this surface is indicated as a protrusion (indicated by a white point in FIG. 98B) and a lower portion is indicated as a recess (indicated by a black point in FIG. It can be detected. As described above, the specification of the reference surface by the free curved surface can be effectively used for a workpiece having a curved surface shape. In addition, the processing load of detection of a free-form surface is high compared with one-point specification and three-point specification mentioned above.
(Defect extraction processing)

Depending on the state of the curved surface of the inspection object, the curved surface and the defect are separated if compression is performed in the vertical and horizontal directions of the distance image, that is, in the XY direction in the image reduction processing shown in step S1611 of FIG. It may not be possible to extract scratches as expected. For example, consider an example in which extraction target 1 and extraction target 2 are to be extracted in a distance image obtained by imaging a tube-like inspection target as shown in FIG. If a free curved surface reference is selected for this distance image and the image is compressed in the X and Y directions, as shown in FIG. 163, peaks and valleys present along the tube-like longitudinal direction are also extracted. On the other hand, the extraction target 2 that is originally intended to be extracted is not in a state of being finely extracted.
(Extraction direction specification means)

  Therefore, in order to realize accurate defect detection even in such a case, an extraction direction designation function of extracting a defect, that is, a local unevenness by excluding shapes continuously present in a specific direction on the surface to be inspected is disclosed. It can also be provided. For example, extraction direction specification means capable of specifying the extraction direction when specifying the inspection condition is provided. Specifically, when "free curved surface reference" is selected in the "calculation method" selection column 192 of the GUI shown in Fig. 89 etc., the GUI of the free curved surface reference setting screen 630 of Fig. 164 is displayed instead of Fig. 96 etc. Let In the free curved surface standard setting screen 630 of FIG. 164, an “extraction direction” designation field 632 is added below the “extraction size” designation field 252 as one form of extraction direction designation means. In the "extraction direction" designation field 632, a direction for extracting a local shape change such as a defect to be extracted is designated. In this example, any one of “X”, “Y”, and “XY” can be selected as a preset option by a drop-down list or the like. Similarly, when the “detail setting” button 222 is pressed on the screen of FIG. 164, the screen changes to the detail setting screen 640 shown in FIG. 165, and the highlighting method setting column 154 has a gain adjustment column 156 and an “extraction height” setting column 162, The noise removal setting field 164, the invalid pixel designation field 166, and the like are displayed, and these detailed settings can be made.

  For example, when “Y” is selected in the “extraction direction” designation column 632 on the screen of FIGS. 164 and 165, a local shape change in the Y direction (vertical direction in FIG. 162) of the image is extracted. In other words, a shape which does not change in the Y direction, for example, a groove extending in the vertical direction is ignored, and a detection result as shown in FIG. 166 is obtained. Similarly, when X is selected, the shape change in the X direction (lateral direction in FIG. 162) of the image is extracted. When XY is selected, local shape change in the XY direction as shown in FIG. 163 is extracted. Thus, by designating the extraction direction according to the shape of the inspection object, it is possible to cancel an unnecessary shape and appropriately extract a desired shape. For example, in the inspection object as shown in FIG. 167, when it is desired to extract the character string 1 and the character string 2, when extracting in the XY direction, as shown in FIG. By designating the Y direction as the extraction direction, as shown in FIG. 169, it is possible to remove a uniform shape in the Y direction and accurately extract a desired character string.

  Here, as a specific shape extraction method such as defect extraction processing, extraction processing in the Y direction shown in FIGS. 166 and 169 will be described as an example. In the above-described image reduction processing (step S1611 in FIG. 161), reduction is performed only in the Y direction of the image. As a result, the free-form surface image serving as a reference obtained by obtaining the filter processing (step S1612) and the image enlargement processing (step S1613) is free-form surface in which fine irregularities in the Y direction are removed and smoothed only in the Y direction. An image will be generated. In other words, it is possible to obtain an image in which only the shape to be canceled that is uniform in the Y direction is left. For this reason, if the difference with this free-form surface image is calculated with respect to the original image to be processed (step S1614), the shape of the cancellation target that is uniform in the Y direction is removed, and as a result, the shape is finely divided in the Y direction. Only the uneven shape change is left. This makes it possible to extract only the shape change in the desired direction.

  Such processing is particularly effective for inspection of a long inspection object in one direction. By specifying the extraction direction along the longitudinal direction of the inspection object, it is possible to cancel the continuous uniform shape along the longitudinal direction, and accurate defect extraction can be realized.

  Moreover, in order to acquire height information of an inspection object (for example, a product having a uniform shape in one direction such as a tire) continuous in such one direction, in addition to the structured illumination, the light described above is used. An inspection using a cutting method can be suitably applied. In particular, since the light cutting method is a method of acquiring the shape (profile) of a linear cut surface, the profile is continuously acquired while changing the cutting position for a long inspection object in one direction. A range image can be acquired by combining the selected profiles. With this method, even when the inspection object moves, in other words, the inspection object can be inspected without stopping it, so it is transported, for example, in applications such as inspection of products transported on a factory line. It becomes possible to detect a defective product without stopping the product.

In the above example, the defect extraction process is applied to the free curved surface standard. However, the present invention is not limited to the free curved surface standard, but can be applied to other reference planes. For example, when the average height reference is selected as the reference surface, the above-described defect extraction process is applied to obtain the average height for each line in the designated direction, and the difference from the average height is obtained. Can be taken. According to this method, since the processing is simple, the advantage that high-speed processing can be obtained is obtained.
(Extraction area setting dialog 148)

Furthermore, when height extraction is performed, the target area (extraction area) for calculating the reference plane from the input image is the same as the area to be inspected (measurement area) to be subjected to inspection processing, and separately from the measurement area. It can also be set to As an example, in FIG. 90, when an “extraction area” button 147 is pressed, an extraction area setting dialogue 148 capable of setting an extraction area is displayed as shown in FIG. An extraction area selection field 149 is provided in the extraction area setting dialog 148, and the user can select "Same as measurement area", "rectangle", "circular", "rotational rectangle" or the like from the extraction area selection field 149. If “Same as measurement area” is selected, the extraction area becomes the same as the measurement area as described above. On the other hand, by selecting other than “same as measurement area”, an area different from the measurement area can be set as the extraction area. For example, as shown in FIG. 100, when “rectangle” is selected in the extraction area selection field 149, a rectangular frame is displayed in the second image display area 121, and the user can specify a desired area by dragging the mouse or the like. When the "edit" button 324 provided on the right of the extraction area selection field 149 in FIG. 100 is pressed, the extraction area editing dialog 326 is displayed as shown in FIG. 101, and the rectangular extraction area is numerically expressed by xy coordinates. It can be specified by. When the extraction area is adjusted in the extraction area editing dialog 326, the change is reflected in the second image display area 121.
(Mask area setting field 330)

  Further, from the extraction area setting dialog 148, a mask area for specifying an area not to be an extraction area can also be set. In the example of FIG. 99, a mask area setting field 330 is provided below the extraction area selection field 149. In the mask area setting field 330, a plurality of mask areas can be set. Here, a maximum of four mask regions 0 to 3 can be specified, and each mask region can be set independently. For example, when “circular” is selected as the mask area 0 as shown in FIG. 102, the circular mask area 0 is displayed on the second image display area 121. Further, when the "edit" button 332 is pressed in this state, a mask area edit dialog 334 for defining the details of the circular mask area 0 is displayed as shown in FIG. The user can define a circular mask area 0 by the xy coordinates of the center and the radius. The mask area is displayed by a frame line of a color different from that of the extraction area in the second image display area 121, and the user can easily distinguish the extraction area and the mask area visually. In the example of FIG. 103, the extraction area is displayed in green and the mask area is displayed in yellow. However, it is needless to say that the present invention is not limited to this example, and it is possible to distinguish by different colors, by line thickness or line type (solid line, broken line etc.), highlight (flashing or highlighting) etc. .

In this way, the extraction area can be set independently of the measurement area. Also, the setting contents of the extraction area can be displayed as text information. For example, in the example of FIG. 90, the text "Same as measurement area" is displayed below the "extraction area" button 147, indicating that the extraction area for calculating the plane reference is the same as the measurement area. Further, “mask area: invalid” is displayed at the lower part, indicating that no mask area is set in the extraction area. This allows the user to check the outline of the "extraction area" also as text information.
(Pre-processing setting)

As described above, when the setting of height extraction is finished in the "area" processing unit, as shown in FIG. 104, in the third image display area, within the rectangle of the area set in the "area" processing unit, The low gradation distance image subjected to gradation conversion based on the extracted height is superimposed and displayed. Next, pre-processing settings are performed. The pre-processing is the common filter processing performed before generating the distance image as described above, and various filters can be selected here. Specifically, when the “pre-processing” button provided in the setting item button area 112 is selected from the screen of FIG. 104, the filter processing setting screen 340 of FIG. 105 is displayed, and a filter to be applied can be selected. As a filter which can be selected here, an averaging filter, a median filter, a Gaussian filter, etc. are mentioned. Furthermore, setting of a binarization level is also possible other than such a filter, for example. For example, in the binarization level setting screen 350 of FIG. 106, an upper limit value, a lower limit value, and the number of times for binarization can be set. In addition, a histogram indicating the distribution of binarized pixels may be displayed, or a function of updating the histogram in conjunction with the input image may be provided. Thus, when the filter processing setting is completed, as shown in FIG. 107, in the third image display area, the low gradation distance image binarized through filter processing is displayed overlappingly in the set area. .
(Judgment setting)

Furthermore, in the “area” processing unit, the input image is converted to a low gradation distance image according to the set conditions, height extraction, pre-processing, etc. It also stipulates the conditions for making judgments such as examinations and image examinations. For example, when the “determination condition” button provided in the setting item button area 112 is pressed on the screen of FIG. 107, the determination condition setting screen 360 of FIG. 108 is displayed, and the determination condition is set. In this example, the pixels of the binarized low-gradation distance image are counted, and when the numerical value is within a predetermined range is set as OK, and when not present, as NG. In the example of FIG. 108, since the determination condition is 0 to 30, and the current value is 166, it is determined to be NG, and as shown in FIG. 109, “determination result: NG” and red characters in the third image display area Is displayed. As described above, when the determination result is NG, the user can easily recognize it at the time of operation by setting it as a prominent aspect, such as making the character red.
("Blob" processing unit 267)

Further, although the determination based on the height inspection using the height information has been described above, the present invention is not limited to this, and it is also possible to add the determination processing based on the image inspection on the conventional luminance image. Such image inspection is called a blob. For example, as shown in FIG. 110, a “blob” processing unit 267 which performs blob (image inspection) is added to the lower part of the “area” processing unit in the flow display area 261. In the “blob” processing unit 267, as in the above, the target area is set and the pre-processing is set (FIG. 111) or the detection condition is set (FIG. 112). The results can be output (FIG. 113).
("Color inspection" processing unit 267B)

  Furthermore, in the case where a color optical camera can be input, such as when a color CCD camera is connected as an imaging means, it is also possible to combine color inspection. For example, as shown in FIG. 114, a "color inspection" processing unit 267B for performing color inspection as measurement processing is added to the lower part of the "blob" processing unit 267 in the flow display area 261. In the “color inspection” processing unit 267B, as in the above, the target area is set (FIGS. 115 and 116) and detailed setting such as density averaging is performed (FIG. 117) to set the determination conditions. The determination result can be output (FIG. 118).

As described above, after various settings are performed in the setting mode, the workpiece is actually imaged in the operation mode to acquire an input image, and the determination processing is performed based on the result of the height inspection and the image inspection. In the above example, the determination result can be output in the setting mode, so that the user can easily recognize the determination result at the setting stage. However, it is needless to say that the judgment result can be output only in the operation mode.
(Operation mode)

Next, processing at the time of operation in the head unit and the controller unit in the three-dimensional image processing apparatus shown in FIGS. 4A and 4B will be described based on the flowcharts in FIGS. 119 and 120. First, the flow of processing at the time of operation of the head unit of the 3D image processing apparatus according to the third embodiment shown in FIGS. 4A and 5 will be described based on the flowchart of FIG.
(Flow of processing according to the third embodiment)

  First, when a trigger is input from the outside (step S11901), one light projection pattern is projected onto a work from the first projector 20A (step S11902), and an image is taken by the image pickup means (step S11903). Next, it is determined whether or not imaging has been completed for all the light projection patterns (step S11904), and in the case where it is not yet received, the light projection patterns are switched (step S11905), and the process returns to step S11902 to repeat the processing. Here, a total of 16 pattern projection images, i.e., 8 pattern projection images with a light projection pattern using a phase shift method and 8 pattern projection images with a light projection pattern using a space coding method are taken. .

  On the other hand, if all the light projection patterns have been imaged in step S11904, the process branches to step S11906 and step S11907. First, in step S11906, a three-dimensional measurement operation is performed to generate a distance image A.

  On the other hand, in step S11907, an average image A 'is calculated by averaging the plurality of pattern projection images (pattern projection image group) captured by the phase shift method.

The above steps S11902 to S11906 are three-dimensional measurement by pattern projection from the first projector 20A. Next, three-dimensional measurement by pattern light projection from the second projector 20B is performed. Here, in step S11908 following step S11906, the light projection pattern is projected onto the work from the second projector 20B as in steps S11902 to S11905 (step S11908), and is imaged by the imaging means (step S11909) It is determined whether the imaging has been completed for all the light projection patterns (step S11910). If not, the light projection pattern is switched (step S11911), and the process returns to step S11902 to repeat the processing. If all the light projection patterns have been captured, the process branches to step S11912 and step S11912. In step S11912, a three-dimensional measurement operation is performed to generate a distance image B. On the other hand, in step S11913, an average image B 'of the pattern projection image group imaged by the phase shift method is calculated. Thus, when the three-dimensional distance images A and B are generated, the three-dimensional distance images A and B are combined in step S11914 to generate a distance image. In step S11915, using the average images A ′ and B ′, a luminance image (average two-dimensional gray-scale image) combining these is generated. In this manner, in the three-dimensional image processing apparatus of FIG. 5, a distance image having workpiece height information is acquired. If the luminance image is unnecessary, steps S11907, S11913, and S11915 can be omitted.
(Flow of Processing of Fourth Embodiment)

The flow of processing of the head unit of the three-dimensional image processing apparatus according to the third embodiment shown in FIG. 4A has been described above. Next, the process flow of the head unit of the three-dimensional image processing apparatus according to the fourth embodiment shown in FIG. 4B will be described based on the flowchart of FIG. explain. First, when a trigger is input from the outside (step S12001), one light projection pattern is projected from the light projecting means 20 onto the work (step S12002), and the first imaging means 10A picks up an image (step S12003), At the same time, the second imaging unit 10B also captures an image (step S12004). Then, it is determined whether the imaging has been completed for all the light projection patterns (step S12005). If not, the light projection pattern is switched (step S12006), and the process returns to step S12002 to repeat the process. If all the light projection patterns have been captured, the process branches to step S10027 to step S12010. In step S12007, a three-dimensional measurement operation is performed to generate a distance image A. In step S12008, a three-dimensional measurement operation is performed to generate a distance image B. On the other hand, in step S12009, an average image A 'of the pattern projection image group imaged by the phase shift method is calculated, and in step S12010, an average image B' of the pattern projection image group imaged by the phase shift method is calculated. Then, in step S12011, the three-dimensional distance images A and B obtained in steps S12007 and S12008 are combined to generate a distance image. Also, a luminance image is generated by combining the average images A ′ and B ′ obtained in steps S12009 and S10. According to this method, since two images can be simultaneously captured, the imaging time can be shortened.
(Inspection area setting means)

  At the time of inspection in actual operation, it is necessary to designate in advance an area (inspection target area) which is an object on which the inspection execution means 50 executes an inspection on a workpiece. Such setting of the inspection target area is performed by the inspection target area setting means in the setting stage. The inspection target area setting means may be provided on the controller unit 2 side as described above, or may be realized by a three-dimensional image processing program. Specifically, when the “area setting” button 115 corresponding to the inspection object area setting means of the three-dimensional image processing program shown in FIG. 62 is pressed as described above, the screen transits to the inspection object area setting screen 120 shown in FIG. In the inspection target area setting screen 120, an area to be inspected can be designated.

  When the inspection target area is designated in this manner, the three-dimensional image processing apparatus performs image processing on the inspection target area and further executes the inspection. That is, as shown in the flowchart of FIG. 121, the inspection target area is assigned to the distance image (step S12101), and the tone conversion parameter is set based on the distance image (step S12102). Tone conversion is performed (step S12103), and image processing is performed on the tone conversion image, and a predetermined inspection is performed (step S12104).

The tone conversion processing is performed on the inspection target area set by the inspection target area setting unit described above. That is, in this example, the inspection target area setting unit is common to the gradation conversion target area specification unit that specifies the gradation conversion target area. However, the area used to determine the tone conversion processing parameter may be set independently of the area to be inspected. For example, using the inspection target area setting unit or the gradation conversion parameter generation area specification unit prepared separately from this, the gradation conversion parameter generation area is specified.
(Operation flow of controller unit during operation)

Next, a procedure for processing the distance image obtained on the head unit side on the controller unit side at the time of operation will be described based on the flowchart of FIG. 122 and the GUIs of FIG. 123 to FIG. Here, FIGS. 123 to 126 show the GUI of the three-dimensional image processing program. The GUI of the 3D image processing program shown in FIG. 123 shows an initial screen 260, and a flow display area 261 is provided on the left side of the screen, and a third image display area 262 is provided on the right side. In the flow display area 261, a flow diagram in which the contents of each process performed by the three-dimensional image processing apparatus are connected in the form of processing units is displayed. Here, as a processing unit, an “imaging” processing unit 263, a “Shapetrax 2” processing unit 264, a “position correction” processing unit 265, and a “height measurement” processing unit 266 are displayed in the flow display area 261. In addition, in the setting stage before operation, each processing unit can be selected and detailed setting can be performed. Further, in the third image display area 262, a luminance image, a distance image, an inspection result or the like is displayed according to the processing content. In the example of FIG. 123, a luminance image obtained by imaging a work (IC in this example) is displayed, and a search target area SA described later is displayed in a green frame.
(Imaging step)

  First, in step S12201 in FIG. 122, the distance image and the luminance image are acquired from the head unit 1. Here, a pattern projection image is captured on the head unit side, and a distance image and a luminance image are generated. In the example of the GUI in FIG. 123, the “imaging” processing unit 263 displayed in the flow display area 261 corresponds.

In the image data, the distance image is first transmitted from the head unit to the controller unit, and then the luminance image is also transmitted to the controller unit. Conversely, after transferring the luminance image first, the distance image may be transferred, or these transfers may be performed simultaneously.
(Search step)

Further, in step S12202, the controller unit performs pattern search on the input image input during operation. That is, the part to be examined is specified so as to follow the movement of the workpiece included in the captured luminance image. In the example of FIG. 123, the “Shapetrax2” processing unit 264 displayed in the flow display area 261 corresponds to the pattern search. Here, the image search means 64 of FIG. 5 performs a pattern search on the luminance image (input image) obtained by imaging the workpiece, and performs positioning. Specifically, the position is specified where the inspection target area set in advance is included in the input luminance image. A search target area SA for performing a pattern search is set in advance. For example, in the example of FIG. 124, the search target area SA is designated in a rectangular shape on the luminance image displayed in the third image display area.
(Position correction step)

Next, in step S12203, the position of the inspection target area is corrected using the result of the pattern search. In the example of FIG. 123, the “position correction” processing unit 265 displayed in the flow display area 261 corresponds. The inspection target area is preset on the inspection target area setting screen. In the example of FIG. 125, a plurality of inspection target areas are set on the distance image displayed in the third image display area. Specifically, as shown in FIG. 126, a rectangular area is set for each pin of the IC which is a work. The position correction is performed by, for example, a method of calculating a positional shift amount by a normalized correlation search, a method based on a result of pattern search, or the like. In this manner, the position correction is performed to correct the position of the inspection target area of the inspection process performed in the next stage.
(Inspection processing step)

  Finally, in step S12204, an inspection is performed using the distance image at the corrected position. In the example of FIG. 123, the “height measurement” processing unit 266 displayed in the flow display area 261 corresponds. Further, in the example of FIG. 127, height measurement is performed. That is, the height is measured and inspected in the inspection target area at the corrected position. For example, it is determined whether or not the plane height is within a predetermined reference value, and the determination result is output.

In the above procedure, after performing position correction using a luminance image, an example of performing inspection based on a distance image has been described. However, the present invention is not limited to this, and an image to be subjected to position correction and an image to be subjected to inspection processing can be set arbitrarily. For example, contrary to the above, after performing position correction using a distance image, inspection processing can also be performed using a luminance image. As an example, in a case where accurate pattern search is difficult in luminance images, such as when a white work is placed on a white background, it is considered that pattern search based on height information is effective using distance images. Become. The inspection process can also be performed not only by the determination process based on the height information, but also by the result of image processing using a luminance image, such as reading a character string printed on a work by using an OCR.
(Output format of height information)

The measured three-dimensional height information is obtained as three-dimensional point cloud data having values of X, Y, and Z, respectively. In addition to the three-dimensional point cloud data, for example, Z values and XY equal pitch Z images can also be converted as to how to output the values actually obtained.
(1: Z image)

The Z image is height image data of only the Z coordinate. For example, in the case where unevenness of the position of the work taken by the image pickup means is important and the X and Y coordinates do not have to be accurate, the X and Y coordinates are not necessary. It is enough if it outputs. In this case, the amount of data to be transmitted is reduced, and the transmission time can be shortened. Further, as in the case of ordinary two-dimensional imaging means, data can be handled as an image, so image processing can also be performed using an existing image processing apparatus for two-dimensional images.
(2: XY equal pitch Z image)

  The XY equal pitch Z image is height image data in which the XY coordinates are equal pitch regardless of the height. Specifically, the Z coordinate at a position where the XY coordinates are equal pitches is interpolated from peripheral point cloud data to obtain an XY equal pitch Z image.

Generally, when the lens of the imaging means is not an objective telecentric lens, the position (X, Y coordinates) imaged by the imaging means differs depending on the height position (Z coordinate) of the imaged work. For this reason, the actual XY coordinate position will differ depending on the height, even if the object appears at the same position on the imaging device. For example, when it is desired to examine a three-dimensional difference such as a volume, if the work is near the camera, the value becomes large, and if it is far, the value becomes small, which is disadvantageous. Therefore, by obtaining Z data of XY uniform pitch from the point cloud data, it is possible to obtain a Z image having an XY position that is not influenced by the height.
(3: XYZ (Point Cloud Data))

Alternatively, point cloud data can be output as it is as three-dimensional information. For example, it is used when it is desired to handle the measured three-dimensional data as it is. In this case, although the amount of data is tripled as compared with the case of only Z coordinate, since it is raw data, it can be used for applications such as finding a three-dimensional difference with three-dimensional CAD data.
(Generation of equal pitch image)

Next, FIG. 128 shows a data flow diagram for creating an equal pitch image in which the angle of view is corrected, in addition to the distance image and the luminance image. As shown in this figure, first, a space code image is generated according to a space coding method from a space coding pattern projection work image group imaged by the head unit. On the other hand, a phase image is generated from the phase shift pattern projection work image group according to the phase calculation. Then, phase expansion calculation is performed from these space code image and phase image to generate an expanded phase image. Also, common filter processing may be applied such as applying a filter to the space coding pattern projection work image group and the phase shift pattern projection work image group. Common filter processing includes application of a two-dimensional filter such as a median filter, a Gaussian filter, and an averaging filter. On the other hand, a luminance image (average gray-scale image) is generated by averaging the phase shift pattern projection work image group.
(Evenly spaced processing)

  As described above, after the three-dimensional data and the luminance image are generated by the head unit, a distance image such as a Z image or an equal pitch Z image is further generated. First, the distance image is transferred from the head unit 1 to the controller unit 2, and gradation conversion is performed to convert phase information into height information. Here, X, Y, and Z images are obtained from the phase information, and these XYs are equalized to obtain XY equal pitch Z images and XY equal pitch Z average images in the XY plane. Such uniform interval processing is performed by the interval equalization processing setting means 47.

  Although an example in which the phase shift method and the spatial coding method are combined in the generation of the distance image is described in the example of FIG. 128, the distance image can also be generated only by the phase shift method without using the space coding method. . The space coding processing can be switched ON / OFF by the space coding switching means 45 shown in FIG. Such an example is shown in the data flow diagram of FIG. As shown in this figure, since the space coding method is not used, the number of imaging can be reduced, and a distance image can be generated at high speed.

  On the other hand, FIG. 130 shows a data flow diagram when obtaining a Z image by turning off the XY equal pitching function. In this example, since it is not necessary to separate into an X image and a Y image at the time of gradation conversion, the processing can be simplified accordingly.

Further, FIG. 131 shows an example of outputting point cloud data which directly outputs XYZ coordinate information. In this example, since it is possible to output the X image, the Y image, and the Z image after the phase → height conversion as they are, it is possible to obtain an advantage of being able to output with light load.
(Gradation conversion method)

Next, a procedure will be described in which the tone conversion means 46 of the three-dimensional image processing device automatically performs tone conversion of a high tone distance image into a low tone low tone distance image based on the distance image. . Here, in an inspection apparatus installed on a line on which a plurality of workpieces are transported, there are a plurality of applications such as gradation conversion to a low gradation distance image in real time with respect to sequentially input distance images (input images) A procedure for performing gradation conversion on an input image of FIG. The tone conversion processing in this case is roughly divided into (A) a method of determining the tone conversion parameter in advance (static conversion) and a method of determining the tone conversion parameter according to the (B) input image. There are two ways (dynamic method). These will be described below.
(A: static conversion)

First, static conversion in which tone conversion parameters are determined in advance will be described. Here, tone conversion parameters for tone conversion are adjusted with respect to an input image or a registered image registered in advance at the time of setting. Then, at the time of operation, the tone conversion of the distance image is performed by the tone conversion parameter set at the setting, and the inspection is performed on the low tone distance image after the tone conversion. The procedure at the time of setting is as described above based on the flowchart of FIG.
(Operational procedure)

When the tone conversion parameter is adjusted, tone conversion is performed on the input image input at the time of operation using this tone conversion parameter. Here, the procedure of static conversion at the time of operation will be described based on the flowchart of FIG. First, in step S13201, a distance image is acquired. Here, the controller unit 2 takes in the distance image generated by the distance image generation means 32. Next, in step S13202, tone conversion processing of the input distance image is performed. Here, tone conversion processing is executed according to the tone conversion parameter adjusted at the time of setting, and a low tone distance image is generated in which the number of tones of the distance image, that is, the dynamic range is reduced. Finally, in step S13203, the inspection execution unit 50 executes an inspection process. According to this method, by setting the tone conversion parameter in advance, it is not necessary to calculate the tone conversion parameter at the time of operation, and the processing can be made light load.
(B: Dynamic conversion)

  Next, dynamic conversion in which gradation conversion parameters at the time of gradation conversion are calculated based on an input image will be described. First, the setting procedure is performed according to the flowchart of FIG. 8 as described above. Specifically, in step S81, an input image or a registered image is acquired, and in step S82, a gradation conversion method is selected. Here, it is assumed that the user has selected dynamic conversion. Then, in step S83, the tone conversion parameter is adjusted. Here, with respect to the input image input at the time of operation, under what conditions the gradation conversion parameter is to be calculated or adjusted is set based on the image acquired in step S81.

  Thus, when the calculation condition of the tone conversion parameter is set, in operation, the tone conversion parameter according to the input image is individually calculated according to the set tone conversion parameter calculation condition. Next, a procedure at the time of operation of converting a distance image into a low gradation distance image by dynamic conversion will be described based on a flowchart of FIG. First, in step S13301, a distance image is acquired. Here, as in step S13201 described above, the controller unit 2 takes in the distance image generated by the distance image generation means 32. Next, in step S13302, tone conversion parameters are determined based on the distance image which is the input image. The method described above can be used as a method of adjusting the gradation conversion parameter. Further, in step S13303, gradation conversion is performed. Finally, inspection processing is executed in step S13304. According to this method, since the tone conversion parameter can be changed according to the input image, the tone conversion can be flexibly performed even on different works, and accurate inspection can be performed. For example, inspection of the workpiece surface with variation in height can also be performed without reducing the accuracy.

Here, as an example of dynamic conversion, a method of optimally creating a low gradation distance image after gradation conversion by adjusting the gradation conversion parameter will be described based on FIG. Here, there are two methods, a method of dynamically setting one gradation conversion parameter set, and a method of preparing a plurality of gradation conversion parameter sets in advance and dynamically selecting the gradation conversion parameter set. .
(How to set 1A 1 gradation conversion parameter set)

  First, a method of dynamically setting one tone conversion parameter set will be described. In this method, the value of the tone conversion parameter used for tone conversion is adjusted based on image information of a predetermined area of a plurality of distance images that are input images, and the adjusted tone conversion parameter is adjusted. The tone conversion means 46 executes tone conversion processing of the distance image using. Here, an example will be described in which a distance image of 16 gradations (image before gradation conversion) is gradation converted into a low gradation distance image of 8 gradations (image after gradation conversion). Further, as shown in the perspective view of FIG. 134, the workpiece to be inspected is such that the entire measurement surface of each workpiece is vertically shifted, and the range is 5 mm. In addition, the range in the height direction including the thickness and distortion of the entire measurement surface is 0.5 mm. In this way, for each workpiece having different surface heights, an example will be considered in which whether or not each surface has a flaw is inspected by image processing. By using the distance image and performing gradation conversion of the distance image into a two-dimensional gray-scale image (low gradation distance image), inspection can be performed with the existing image processing apparatus for two-dimensional images.

  First, an area to be inspected on the workpiece is designated in advance as an area to be inspected by the inspection area setting unit. In addition to specifying a part of the distance image which is the input image, the inspection target area may be the entire distance image. In this case, the work of specifying the inspection target area may be omitted.

  First, the average distance of the inspection area is determined. Next, a difference (distance range) between the maximum distance and the minimum distance of the inspection target area is obtained. Further, a numerical value obtained by multiplying the distance range by 1.2 is set as the distance range of the converted image. For example, in the distribution of workpieces in FIG. 134, assuming that the distance range is 0.5 mm, 0.6 mm obtained by multiplying this by 1.2 is set as the distance range of the image after gradation conversion. Therefore, the range of ± 0.3 mm around the average distance is the measurement range.

  Furthermore, the span for the distance image which is the input image is determined so that the range of 0.6 mm (± 0.3 mm around the average distance), which is the distance range of the image after gradation conversion, has 256 gradations. The span can also be a predetermined constant. Moreover, with respect to the distance image, it is also possible to set 0 for an average distance of −0.3 mm or less to 0, and set 255 for an average distance of +0.3 mm or more.

In this manner, a range necessary for inspection is determined from a plurality of distance images to be input, and height information in this range is appropriately maintained so as to be maintained in the low gradation distance image after gradation conversion. Gradation conversion parameters can be set. As a result, it becomes possible to appropriately inspect the presence or absence, the position, and the like of the flaws of the work dispersed in the height direction using the existing image processing apparatus.
(1B Method to prepare multiple tone conversion parameter sets in advance)

  In the above embodiment, the method of obtaining the gradation conversion parameter set appropriately so as not to lose the height information necessary for the detection of a flaw is described using the acquired height information of the distance image. On the other hand, a method of preparing a plurality of tone conversion parameter sets in advance will also be described below. In this method, a tone-converted image is generated by performing tone conversion using a plurality of tone conversion parameters set in advance for a plurality of distance images that are input images, and a predetermined inspection of the distance image is performed. An image after tone conversion to be used for inspection is selected based on the image information of the target area.

Such a specific example will be described based on the schematic view of FIG. 135 and the flow chart of FIG. Also in this example, as in the case of FIG. 134 described above, it is assumed that the entire measurement surface of each workpiece is vertically shifted within a range of 5 mm. For such a work, nine images after gradation conversion, in which gradation conversion is performed by changing the conversion center every 0.5 mm, are created. Then, the obtained after-gradation-converted images are preferably displayed side by side on the display means, and the user is allowed to select a converted image with a conversion center closest to the average value of the distance images which are input images. Then, the tone conversion parameter set applied to the image selected by the user is set. As a specific procedure, as shown in FIG. 136, first, a distance image of a work is generated (step S13601), and gradation conversion processing is performed multiple times while changing gradation conversion parameters as described above (step S13602) . Then, the user is made to select a desired image from the simple low gradation distance image which is the obtained gradation conversion image (step S13603), and the gradation conversion parameter is adjusted as necessary, and the obtained low value is obtained. An inspection is performed on the gradation distance image (step S13604).
(Details of static conversion, dynamic conversion)

Next, the details of the static conversion and the dynamic conversion will be described. First, static conversion will be described. In static conversion, as a specific method for correcting the reference of height information to be left when converting a distance image into a low gradation distance image,
(A1) One point designation to be corrected by the designated height (distance),
(A2) Three-point designation correcting on a plane can be used.
(A1: One point specified)

One-point specification is a method of performing gradation conversion of a distance image to a low gradation distance image based on the height (distance) of a point or area specified by the user. The reference height is, for example, the height in the middle of the height range (distance range) in which gradation conversion is performed to the low gradation distance image among the height information of the distance image. Alternatively, it may be set to the upper limit (the highest position where gradation conversion is performed) or the lower limit (the lowest position where gradation conversion is performed) of the distance range. The specific procedure of one-point specification is as described based on FIGS. 66 to 78 above.
(A2: Three points specified)

  The three-point specification is a method of performing gradation conversion of a distance image into a low gradation distance image using a plane obtained from three points specified by the user as a reference plane. The reference plane is also, for example, the height of the middle of the height range (distance range) in which the gradation conversion is performed to the low gradation distance image among the height information of the distance image, similarly to the reference height of one point designation. Alternatively, it may be set to the upper limit (the highest position where gradation conversion is performed) or the lower limit (the lowest position where gradation conversion is performed) of the distance range. A specific example of the three-point specification is as described based on the GUI screens of FIGS.

  Here, an example in which designation of a reference plane by static conversion is effective will be described based on FIG. In this example, the workpiece WK7 having the same height is conveyed on the same surface by the belt conveyor BC. In this case, since there is almost no change in the height of the workpiece WK7, dynamic conversion for changing the reference surface according to the workpiece is unnecessary, and static conversion can be used. In particular, static conversion is faster than dynamic conversion and can take advantage of static conversion.

Further, FIG. 138A is an example of a work for which it is desired to suppress the height variation itself. In this example, an inspection process is detected in which a cap of a bottle, which is the work WK10 transported by the belt conveyor BC, has a "float" as an abnormality. In this case, by adopting static conversion, it is possible to detect the fluctuation in height as in the low gradation distance image shown in FIG. 138B. On the contrary, if dynamic conversion is adopted, as a result of the fluctuation being corrected, it becomes impossible to detect an abnormality. Therefore, static conversion can be suitably used in such applications (B: specific example of dynamic conversion)

The above has described static conversion in which tone conversion conditions are specified in advance at the setting stage, and tone conversion is performed under the conditions specified during operation. Next, a specific example of dynamic conversion in which the gradation conversion condition is adjusted in accordance with the input image to be inspected will be described. First, in dynamic conversion, as a specific method for correcting the reference of height information to be left when performing gradation conversion of a distance image to a low gradation distance image,
(B1) an average height reference for performing gradation conversion using an average height (average distance) in an average extraction area designated for an input image as an average reference height;
(B2) A plane reference that generates an estimated plane in a designated area of an input image and performs tone conversion using this as a reference plane,
(B3) A free-form surface reference is included which generates a free-form surface obtained by removing high-frequency components from an input image and performs tone conversion using this as a reference surface.

  The average extraction area for defining the average reference height is previously set prior to operation (step S83 in FIG. 8 described above). One example of the procedure for specifying the average extraction area in step S83 in FIG. 8 is as described based on the GUI in FIG. 88 to FIG. On the other hand, at the time of operation, dynamic conversion is performed according to the procedure described in FIG. For example, the workpiece conveyed on the line is imaged to generate a distance image (step S13301), the average height of the average extraction area set above is calculated (step S13302), and tone conversion is performed based thereon The low gradation distance image is generated to generate a low gradation distance image (step S13303), and the obtained low gradation distance image is inspected (step S13304). With this method, even if there is a variation in the height direction of the workpiece, the reference plane of gradation conversion can be reset each time for each workpiece, so that an accurate inspection is realized regardless of the variation in the height direction of the workpiece. it can.

  Next, in the plane reference, as in the above-described average height reference, the reference surface estimation area for determining the reference surface is previously set prior to operation (step S83 in FIG. 8 described above). One example of the procedure for specifying the reference surface estimation area in step S83 of FIG. 8 is as described above based on the GUIs of FIG. 88 and FIGS. At the time of operation, dynamic conversion is performed according to the procedure described in FIG. For example, the workpiece conveyed on the line is imaged to generate a distance image (step S13301), the reference surface estimation area set in the above is extracted, and the estimation surface is calculated (step S13302). Gradation conversion is performed on the basis of the surface to generate a low gradation distance image (step S13303), and the obtained low gradation distance image is inspected (step S13304). According to this method, even if the surface of the workpiece has an inclination or the like, this can be canceled and accurate inspection can be realized regardless of the inclination of the workpiece.

  Finally, the specific setting method of the free-form surface reference is as described based on the GUIs of FIGS. 126 to 32. Also in the free-form surface reference, as in the case of the average height reference described above, necessary conditions for determining the reference surface are set in advance prior to operation (step S83 in FIG. 8 described above). At the time of operation, dynamic conversion is performed according to the procedure described in FIG. For example, the workpiece conveyed on the line is imaged to generate a distance image (step S13301), a free-form surface is calculated for the free-form surface target area set above (step S13302), and the obtained free-form surface is obtained. The tone conversion is executed on the basis of to generate a low tone distance image (step S13303), and the obtained low tone distance image is inspected (step S13304). According to this method, it is possible to obtain an advantage that the work such as the surface inspection of a curved workpiece can be performed with high accuracy even in the work in which the accurate inspection is difficult with the conventional method.

  Here, an example in which designation of a reference plane by dynamic conversion is effective will be described based on FIG. In this example, the workpieces WK11 having different heights are conveyed on the same surface by the belt conveyor BC. In this case, since the height of the work WK11 is individually different, the height of the work is changed by changing the reference surface according to the height of each individual and performing gradation conversion using an average height reference or the like. Even if there is an optimum gradation conversion is possible.

  Further, FIG. 140 shows an example of the work WK12 having a minute inclined surface and a dent on a plane. In this example, since a minute inclined surface exists on the surface of the work WK12 due to the dent DE, the inclination may affect the detection accuracy of the dent. Therefore, by using a plane reference or the like, a plane is dynamically obtained for each workpiece and gradation conversion is performed as a reference plane, so that a highly accurate inspection such as detection of a slight depression or dent can be performed.

Further, FIG. 141 shows an example of a curved workpiece WK13 having different radii. Also in this example, a free curved surface standard or the like is used to obtain a curved surface for each workpiece and gradation conversion is performed using this as a reference surface, thereby making it possible to perform an inspection in which the influence of variations in shape of each individual is reduced.
(Automatic adjustment of tone conversion parameters)

  The procedure for manually setting the tone conversion parameters based on the image after tone conversion has been described above. On the other hand, when gradation conversion parameters are manually set, when gradation conversion is performed using the gradation conversion parameters set first, an image suitable for inspection may not be obtained due to changes in work or environment. In such a case, without the user making a fine adjustment with reference to the image, the tone conversion parameters are corrected using the data after tone conversion, and image conversion suitable for inspection is performed by performing conversion again. It can also be done. In this method, after performing initial setting of tone conversion, first, tone conversion is performed with an arbitrary tone conversion parameter as an initial value, and thereafter, tone conversion parameter adjustment is performed.

  For example, the tone conversion condition automatic setting unit can function as the tone conversion condition automatic setting unit and the tone conversion condition manual setting unit. That is, the gradation conversion condition automatic setting unit sets simple gradation conversion conditions when the gradation conversion unit converts the distance image into a low gradation distance image. Also, in a state where the simple low gradation distance image subjected to gradation conversion is displayed on the display unit based on the simple gradation conversion condition set by the gradation conversion condition automatic setting unit, the gradation conversion condition manual setting unit , Accepts manual adjustment of gradation conversion conditions. As a result, the user is not prompted to set the tone conversion condition suddenly, but one or more obtained simple after automatically setting the provisional simple tone conversion condition and performing tone conversion. Since desired tone conversion conditions can be set while referring to the low tone distance image, even if the user is not familiar with the meaning of the tone conversion parameter or is unfamiliar with the setting, such a degree of automation can be achieved to some extent This can facilitate setting work.

  A specific procedure will be described based on the flowchart of FIG. First, in step S14201, a distance image creation process is performed. Next, in step S14202, initial tone conversion processing is performed using the initial value of the tone conversion parameter. Further, in step S14203, it is determined whether the obtained tone-converted image is appropriate. If not, the tone conversion parameter is again adjusted in step S14204, and the process returns to step S14202 to repeat the processing. On the other hand, if it is determined in step S14203 that an appropriate tone-converted image is obtained, the process advances to step S14205 to execute a predetermined inspection.

  Although the above method determines in step S14203 whether the gradation conversion parameter is appropriate or not, this procedure may be omitted. A procedure in this case is shown in FIG. Each procedure is substantially the same as the example of FIG. 142 described above, and in step S14301, processing for creating a distance image is executed, and next, in step S14302, initial tone conversion processing is performed. Then, after adjusting the tone conversion parameter in step S14303, a predetermined inspection is performed in step S14304.

  As an initial gradation conversion method, for example, a method of applying the conversion function f (x, y, z) to the input image, a method of shifting or spanning the input image, and compressing the result into n gradations, or any method The difference between the input image and the reference plane is calculated with the plane as the reference plane, shift and span are applied, the result is compressed to n gradations, or the difference between the input image and the reference image is taken, shift and span are calculated. And the method of compressing the result into n gradations.

Next, a specific example of automatically adjusting tone conversion parameters after tone conversion will be described. The method of automatically adjusting the gradation conversion parameters includes: (C1) using the median of the histogram of distance image data after conversion; A combination, etc. can be considered.
(C1: Method using median of histogram)

First, the method of using the median value of the histogram will be described based on the flowchart of FIG. First, in step S14401, the histogram of the distance image data after conversion is calculated. Next, in step S14402, the median value of the histogram is obtained. Then, in step S14403, the tone conversion parameter is changed so that the median becomes a predetermined value. The tone conversion processing is executed again using the tone conversion parameter changed here. Note that an average value of 2n + 1 including n pieces before and after the center may be used as the median. In addition, the maximum value and the minimum value may be median values after removing n pieces from the larger one and m pieces from the smaller one.
(C2: Method using maximum and minimum values of histogram)

First, the histogram of the distance image data after conversion is calculated. Next, the maximum value and the minimum value of the histogram are obtained, and the gradation conversion parameter is changed so that the width of the maximum value-minimum value becomes a predetermined value. Then, tone conversion processing is executed again using the changed tone conversion parameter. Here, each of the maximum value and the minimum value may be an average value of n pieces from the larger one and m pieces from the smaller one. Alternatively, the maximum value and the minimum value may be set to the maximum value and the minimum value after removing n pieces from the larger one and m pieces from the smaller one respectively.
(C3: combination of C1 and C2)

  The above C1 and C2 may be combined. That is, after the calculation of the histogram, the tone conversion parameter is changed based on the width of the median and the maximum value / minimum value, and tone conversion is performed again.

  In the above method, a low pass filter can be applied to the histogram in advance. In addition, a low pass filter may be applied to the distance image after conversion in advance to obtain a histogram.

By such gradation conversion, a distance image with a high number of gradations including height information can be converted into a low gradation low gradation distance image. Since this low gradation distance image can be processed as a two-dimensional image, even an image processing apparatus corresponding to an existing two-dimensional image can handle low gradation distance images. For example, in order to inspect the presence or absence of a flaw, height information of each pixel included in a distance image obtained by imaging a workpiece is expressed as a gray value by a binary number in hexadecimal. Here, when the difference between the distance image and the reference distance image is calculated, the defect information of shallow and small flaws appearing on the surface of the work is aggregated into the lower eight gradations. For this reason, the amount of information of the difference image can be greatly compressed while preventing a decrease in detection accuracy by, for example, reducing the upper eight gradations by the gradation conversion means. As described above, the gray level conversion means reduces the gray level of the upper half of the gray levels of the gray level of each pixel of the difference image, thereby preventing the detection accuracy from being lowered. The amount of information can be greatly compressed.
(Partial execution of gradation conversion processing)

Furthermore, gradation conversion processing can be performed not on all of the distance images but only on part of the distance images. Specifically, the gradation conversion unit executes the gradation conversion processing only on the designated inspection target area in the distance image. As a result, tone conversion processing can be reduced, and processing load can be reduced and speeded up. As an example, height inspection processing (first inspection processing) and image inspection processing (second inspection processing) are combined using work WK14 having a shape in which cylinders having different diameters overlap in a triple as shown in FIG. 145A. Consider the case where multiple inspections are performed. Here, the height of each cylindrical surface of the workpiece WK14 is measured as the height inspection processing, and as the image inspection processing, detection of chips or cracks in the cylindrical portions from the top of the workpiece WK14 up to the second stage Perform a visual inspection. Selection of inspection processing is performed by inspection processing selection means. Further, specific settings for the inspection process selected by the inspection process selection unit are performed by the inspection process setting unit.
(Inspection processing selection means)

The inspection process selection means is a means for selecting a plurality of inspection processes to be executed by the inspection execution means on the distance image. Here, as shown in FIG. 44, FIG. 56, etc., inspection processing is selected from the initial screen 260 when adding a processing unit. Specifically, a desired inspection process is selected from a list of inspection processes displayed by selecting the “measurement” process from the “add” submenu of the processing unit. In this example, "area", "pattern search", "Shapetrax2", "edge position", "edge width", "edge pitch", "edge angle", "pair edge", "scratch", "blob", " Gray blob "," trend edge position "," trend edge width "," trend edge defect "," gray inspection "," color inspection "," OCR "," 2D code reader "," 1D code reader "," high Select the desired inspection process from “Measurement”.
(Inspection processing setting means)

On the other hand, the inspection process setting unit sets details of each inspection process selected by the inspection process selection unit. Here, as shown in FIG. 46, FIG. 63, FIG. 78, etc., each setting item is configured to be set individually from each button arranged in the setting item button area 112. In this example, specific setting items are arranged as setting item buttons. For example, an "image registration" button 113, an "image setting" button 114, an "area setting" button 115, a "height extraction" button 116, " A pre-processing button 117, a "detection condition" button 118, a "detail setting" button 119, a "determination condition" button, a "display setting" button, a "save" button, and the like are included. Thus, the setting item button area 112 functions as inspection processing setting means.
(Height inspection process)

An example of a distance image obtained for a work as shown in FIG. 145A is shown in FIG. 145B. This distance image is expressed by 16 gradations before gradation conversion. When height measurement (height inspection processing) is performed on such a distance image, high accuracy is achieved by using height information with high accuracy with 16 gradations without performing gradation conversion. Inspection can be realized. Specifically, as shown in FIG. 145C, in order to measure the height of each surface of the workpiece, the inspection target area is set on each of the three surfaces of the workpiece, and the height of each inspection target area is 16 floors. Measure as it is.
(Image inspection process)

  On the other hand, when performing measurement (image inspection processing) not using height information in inspection processing, high gradation information is unnecessary, and the load is smaller if processing is performed with a lower gradation distance image. Therefore, gradation conversion is performed on a high gradation distance image to obtain a low gradation distance image, and then processing is performed. Here, it is not necessary to convert the entire distance image into a low gradation distance image, and it is sufficient to carry out the gradation conversion only to the target for image inspection processing. That is, the area to be subjected to gradation conversion is an area to be inspected which is set for image inspection processing without using height information among the areas to be inspected which are set one or more. In the example shown in FIG. 145D, as in the case of FIG. 145B, an inspection target area for image inspection processing is set for the high gradation distance image before gradation conversion. In this example, in order to detect a chipping or the like in two steps from the top of the workpiece, the inspection target area is set so as to surround the second stage cylindrical shape. In other words, since the cylindrical shape of the third row is not the object of the appearance inspection, this portion is set to be excluded from the region to be inspected. Then, tone conversion is performed on the image processing inspection target area. The resulting low tone distance image is shown in FIG. 145E. The low gradation distance image after gradation conversion shown in this figure is expressed by 8 gradations, and although the height information is somewhat lost compared to the high gradation distance image, the presence or absence of a chipping is detected. Since sufficient accuracy is maintained for appearance inspection applications, there is no problem in image inspection processing. On the other hand, the area requiring gradation conversion is greatly reduced as compared with FIG. 145B and the like, which contributes to simplification and speeding up of the processing.

Thus, the presence or absence of tone conversion is selected according to the inspection process. For example, in the “height measurement” processing unit and the second “height measurement” processing unit among the inspection processing displayed in the flow display area 261 of FIG. 110, the 16th floor of the distance image does not perform tone conversion. Highly accurate height measurement is realized by using the key height information. On the other hand, in the other inspection processes, for example, in the “numerical operation” processing unit, the “area” processing unit, and the “blob” processing unit 267, a low gradation grayscale image of eight gradations after gradation conversion is used. Although not shown, when inspection processing (for example, color inspection on a color luminance image) is performed on a luminance image, height information is not necessary in the first place, and so gradation conversion is naturally also unnecessary.
(Display of gradation conversion condition setting means 43)

  Further, with regard to image inspection processing that requires such tone conversion processing, the tone conversion condition setting means 43 sets tone conversion conditions for performing appropriate tone conversion. At this time, the tone conversion condition setting unit 43 can be set only when the tone conversion process is necessary, and conversely, in the inspection process where the tone conversion process is unnecessary, for example, in the height inspection process, the tone conversion condition By making the setting impossible, the user can set only the necessary items smoothly without being confused by the extra setting work. Therefore, in the present embodiment, when setting inspection processing that does not require image height information other than height inspection processing, a floor for setting gradation conversion parameters for performing gradation conversion of the distance image While the tone conversion condition setting unit 43 is displayed, the tone conversion condition setting unit 43 is not displayed when setting the height inspection process.

  A specific setting procedure will be described based on the flowchart of FIG. First, in step S14601, an inspection process is selected. Here, the inspection process to be executed by the inspection execution unit is selected in the “measurement” menu which is the inspection processing selection unit. In step S14602, it is determined whether the inspection process selected in step S14601 requires gradation conversion. If the inspection process requires gradation conversion, the process advances to step S14603 to validate the gradation conversion. The tone conversion setting means is displayed on the setting item.

  For example, when the image inspection process is performed on the workpiece of FIG. 145A by the “area” processing unit, as shown in FIG. 147, the gradation conversion condition setting unit 43 is set in the setting item button area 112 which is inspection processing setting unit. "Height extraction" button 116 is displayed. When the “height extraction” button 116 is pressed, a height extraction setting screen is displayed as shown in FIG. The user can set the conditions necessary for the tone conversion processing from this screen.

  In the example of FIG. 148, the low gradation distance image subjected to gradation conversion is displayed on the image display area in accordance with the gradation conversion condition set in the right operation area. As a result, the user can visually confirm whether or not the desired inspection result can be obtained under the current tone conversion condition, and there is obtained an advantage that the tone conversion condition can be easily adjusted. In particular, in the example of FIG. 148, the inspection target area is set to be circular and arranged so as to surround the second cylindrical shape of the workpiece. Also, by setting the detection method as dynamic conversion (real-time extraction) and setting a plane reference as the calculation method, two surfaces, the central top surface of the workpiece and its outer circumferential surface, are detected, and these are used as a reference plane. It detects scratches and pits on the surface of The parameter display of the reference plane detected according to the tone conversion condition can be switched and displayed by console operation.

  Further, in the example of FIG. 148, tone conversion is performed only within the area of the set inspection target area, instead of displaying a low tone distance image obtained by tone converting the entire work displayed on the image display area. The image of the low gradation distance image is displayed. In other words, the original distance image is displayed as it is for images other than the inspection target area. As a result, it is possible to visually indicate to the user that gradation conversion is not performed on the entire input image, but is performed only in a region to be inspected that is a part of the gradation. When the setting of the tone conversion conditions is completed as described above, the process proceeds to step S14604.

  On the other hand, in the case of the inspection process which does not require tone conversion, the process jumps to step S14604 without passing through step S14603. In this case, the gradation conversion condition setting means is not displayed on the setting screen of the inspection process. For example, when the height inspection process is performed by the “height measurement” processing unit, as shown in FIG. 46 etc., the gradation conversion condition setting means 43 is provided in the setting item button area 112 which is the inspection process setting means. "Height extraction" button is not displayed. As a result, the user can recognize that it is not necessary to set conditions regarding gradation conversion unnecessary for the height inspection process. Alternatively, it is possible to avoid confusion caused by unnecessary settings by making it impossible to set the tone conversion conditions for this inspection process.

  Then, in step S14604, setting of inspection processing is performed. Finally, in step S14605, it is determined whether or not the setting of all inspection processes is completed. If it is not yet done, the process returns to step S14601 to repeat from selection of inspection processes, and when the process is completed, the process is completed.

  On the other hand, the procedure at the time of operation will be described based on the flowchart of FIG. First, in step S14901, a distance image is input as an input image. Next, in step S14902, initialization is performed. Here, n is set to 1. Further, in step S14903, the n-th inspection process is executed. In step S14904, it is determined whether n <N (N is the number of set inspection processes). If YES, n is incremented by 1 in step S14905, and the process returns to step S14903 to perform the next inspection process. Repeat the process to execute. On the other hand, if n <N, it is determined that all inspection processing has ended, and the processing is completed. In this way, all inspection processes are sequentially executed.

Here, if the execution of the inspection process in step S14903 is described in detail, first, if gradation conversion is required in the inspection process, as shown in the flowchart of FIG. In contrast, gradation conversion is performed. Then, in step S15002, inspection processing is performed on the low gradation distance image after gradation conversion. On the other hand, in the case of inspection processing in which gradation conversion is not performed, as shown in the flowchart in FIG. Step S15101).
(Hide function of image selection)

  Further, at the time of setting, it is possible to limit the selection of an image to be set as the inspection process according to the type of the inspection process. That is, when the inspection process selected by the inspection process selection means can be executed on any of the distance image or the luminance image, it is possible to call out the distance image or the luminance image as a registered image. On the other hand, although inspection processing can be performed on distance images, some can not be performed on luminance images. For example, the height measurement process is performed on a distance image having highly accurate height information. It can not be performed on normal luminance images that do not have height information. Therefore, inspection processing that can be performed only on an image having height information, such as height measurement processing, that is, only on the distance image, enables selection of only the distance image at the time of calling the registered image at the setting, and conversely It is not possible to select a luminance image. In this way, it is possible to prohibit or eliminate the setting operation which can not be originally performed, such as setting the height image with respect to the luminance image by mistake, to eliminate waste of setting and to improve the operability of the user.

  Hereinafter, the procedure for setting the inspection processing condition will be described based on the flowchart of FIG. First, in step S15201, an inspection process is selected. Next, in step S15202, it is determined whether the inspection process selected in step S15201 is an inspection process that can specify either the distance image or the luminance image or an inspection process that can specify only the distance image. Here, in the case of inspection processing capable of designating either the distance image or the luminance image, the process proceeds to step S15203, and either the distance image or the luminance image is selected.

  For example, as shown in FIG. 61, consider the case where an "area" processing unit is selected as inspection processing. In this case, either a distance image or a luminance image can be specified. Therefore, for example, when the image setting screen 380 shown in FIG. 153 is displayed by pressing the “image setting” button 114 in FIG. 62 as an inspection processing setting item in the “area” processing unit, an image is displayed from the operation field 122 It can be selected. Here, in the “display image” selection field 124 provided in the image selection field 382, the image displayed in the second image display area 121 can be selected. In the image setting field 384, an input image and a registered image can be selected. From the image setting field 384, an input image can be designated by an image variable. Here, when the “input image” selection field 386 is selected, the image variable selection screen 390 of FIG. 154 is displayed, and a list of selectable images is displayed. In the image variable selection screen 390, any of the images captured by a plurality of imaging means (cameras) connected to the 3D image processing apparatus can be selected. Here, different image variables are assigned to each imaging means, and the imaging means and the image variables are associated. For example, the image variable "& Cam1Img" is for the distance image captured by the camera 1, the image variable "& Cam2Img" is for the distance image of the camera 2, the image variable "& Cam3Img" is for the distance image of the camera 3, and the distance image for the camera 4 The image variable “& Cam4Img” is assigned to each of the above. Further, an image variable “& Cam 1 Gray I mg” is attached to the luminance image captured by the camera 1. The user selects a desired image from the image variable selection screen 390. As described above, in the “area” processing unit, since either a distance image or a luminance image can be specified, not only the distance image but also the luminance image is included in the options candidate and displayed.

  Furthermore, as necessary, the image variables displayed in a list on the image variable selection screen 390 can be sorted and displayed, so that the user can easily select a desired image.

  On the other hand, if the inspection process is a process that can specify only the distance image, the process advances to step S15204 to select a distance image. That is, it is made impossible to select a luminance image. For example, as shown in FIG. 45, when the “height measurement” processing unit 266 is selected as the inspection processing, only the distance image can be selected. Therefore, as shown in FIG. 46, when the “image setting” button 114 is selected as the inspection processing setting item in the “height measurement” processing unit, the image setting screen 380 is similarly displayed, and an image can be selected. . Here, when the “input image” selection field 386 is similarly selected, the image variable selection screen 390 of FIG. 155 is displayed, and a list of selectable images is displayed. In this image variable selection screen 390, unlike in FIG. 154, the luminance image is not displayed as an option, and only the distance image is displayed as an option. With this configuration, it is possible to avoid a situation in which the user erroneously selects a luminance image having no height information in height measurement, and an operation environment with less confusion is provided.

Thus, when an image is selected, the process proceeds to step S15205, and inspection processing conditions are set for the selected image. As described above, it is determined for each inspection process whether the inspection process can specify either the distance image or the luminance image or the inspection process can specify only the distance image for each inspection process By defining the types of selectable images, setting errors can be avoided, which contributes to the convenience of the user.
(Invisible function of inspection process)

  The above is an example of restricting selectable images when selecting an image to be subjected to setting of inspection processing according to the type of inspection processing selected after having the user select the inspection processing first. Explained. Conversely, after selecting an image first, it is also possible to limit the types of inspection processing that can be performed on this image. That is, first, a distance image or a luminance image is selected by the image selection unit, and then, when one or more inspection processes executed by the inspection execution unit are selected by the inspection processing selection unit, the selected image is the distance image or the luminance Only the inspection process that can be performed on each image can be selected according to the image. With this configuration, it is possible to avoid selecting an inspection process that can not be set by mistake for the selected image or setting an inspection process condition related to this inspection process.

  Hereinafter, the procedure for setting the inspection processing condition by this method will be described based on the flowchart of FIG. First, in step S15601, an image is selected. Here, it is selected whether the distance image or the luminance image. Next, in step S15602, it is determined whether the image selected in step S15601 is either a distance image or a luminance image. In the case of the distance image, the process proceeds to step S15603, and the user is made to select an inspection process that can be performed on the distance image by the inspection process selection unit. On the other hand, in the case of a luminance image, the process proceeds to step S15604, and the inspection processing selection unit allows the user to select an inspection process that can be performed on the luminance image. When the inspection process is selected as described above, the process proceeds to step S15605, and the inspection process condition for the selected inspection process is set from the inspection process condition setting unit.

  For example, as shown in FIG. 157, consider the case of acquiring a luminance image and a distance image for a workpiece. As described above, when the image registration is performed, the luminance image and the distance image are simultaneously registered. The user selects one of the images, selects the inspection process for the next selected image, and sets the inspection process conditions. For example, when a luminance image is selected, as shown in FIG. 158, a tool corresponding to the inspection process performed on the luminance image is added. Here, as in the case of FIG. 44, FIG. 56, etc. described above, when addition is selected from the submenu displayed by selecting the image, a list of inspection processes that can be performed on the luminance image is displayed. For example, as an “measurement” process, an “area” processing unit, a “scratch” processing unit, and a “blob” processing unit 267 are displayed as options. At this stage, the inspection process that can not be performed on the luminance image, for example, the “height measurement” processing unit is not displayed. As a result, it is possible to avoid the situation where the user erroneously selects an inspection process that can not be set as a luminance image. Note that while all the measurement processes are displayed in a list, it may be grayed out to make it impossible to select. When the inspection process is selected, the processing unit is determined, and the setting of the inspection process condition necessary for the inspection process is performed from the inspection process condition setting unit.

  When a distance image is selected, a tool corresponding to inspection processing that can be performed on the distance image is added as shown in FIG. Also here, if addition is selected from the displayed submenu by selecting the distance image, a list of inspection processes that can be performed on the luminance image is displayed. For example, in addition to the “area” processing unit, the “scratch” processing unit, and the “blob” processing unit 267 as the “measurement” processing, the “height measurement” processing unit is also displayed as an option. When the user selects a desired inspection process, the processing unit is determined, and setting of the inspection process conditions necessary for the inspection process is performed from the inspection process condition setting means.

  As described above, by linking the inspection processing tool to the image, it is possible to physically exclude the combination of the non-selectable image and the inspection processing, and to easily avoid the setting error by the user.

  The three-dimensional image processing apparatus, the three-dimensional image processing method, the three-dimensional image processing program, the computer readable recording medium, and the recorded apparatus of the present invention can be used for an inspection apparatus etc. using the principle of triangular ranging.

100, 100 ', 200, 300, 400 ... three-dimensional image processing apparatus 1, 1B, 1C, 1D ... head unit 2, 2 ... ... controller unit 3 ... input unit 4 ... display unit 10, 10A, 10B ... imaging unit 20 20A: first projector 20B: second projector 21: measurement light source 22: pattern generation unit 23, 24, 25: lens 30: head side control unit 31: head side operation unit 32: distance image generation means 34: filter processing unit 36: head side communication means; 36A: interface for controller connection; 36B: interface for PC connection 38: storage means; 38a: distance image storage unit; 38b: luminance image storage unit 41: controller side setting means 42 Controller side communication means 43 Gradation conversion condition setting means 44 Reference plane setting means 45 Space coding switching means 4 ... gradation conversion means 47 ... interval equalization processing setting means 48 ... light projection switching means 49 ... shutter speed setting means 50 ... inspection execution means 51 ... main control unit 52 ... controller side connection unit 53 ... operation input unit 54 ... display control Unit 55: Communication unit 56: RAM
57 Controller side storage means 58 Auxiliary storage means 59 Output unit 60 Image processing portion 62 Abnormal point highlight means 64 Image search means 66 Real time update means 70 PLC
110 ... 3D image processing program GUI screen 111 ... first image display area 112 ... setting item button area 113 ... "image registration" button 114 ... "image setting" button 115 ... "area setting" button 116 ... "height extraction" Button 117 ... "preprocessing" button 118 ... "detection condition" button 119 ... "detailed setting" button 120 ... inspection object area setting screen 121 ... second image display area 122 ... operation area 124 ... "display image" selection column 126 ... "Measurement area" setting field 128 ... "Edit" button 130 ... Measurement area editing screen 140 ... Height extraction selection screen 142 ... Extraction method selection means 144 ... "Extraction" button 145 ... "Extraction area" specification field 146 ... Point-like pointer 147 ... "extraction area" button 148 ... extraction area setting dialog 149 ... extraction area selection field 150 ... one-point specification screen 152 ... "Z height""Display column 154 ... Emphasis method setting column 156 ... Gain adjustment column 158 ..." Detailed setting "button 160 ... Highlight method detail setting screen 162 ..." Extraction height "setting column 164 ... Noise removal setting column 166 ... Invalid pixel specification column 170 ... Three-point specification screen 172 ... Height extraction display column 174 ... Three-point specification "Detailed setting" button 180 ... Three-point specification detail setting screen 182 ... "Extraction height" setting field 190 ... Height dynamic extraction setting screen 192 ... "Calculation method" selection column 194 ... "Mask area" button 196 ... "Detailed setting" button 210 ... Average height reference setting screen 220 ... Mask area setting screen 222 ... "Detailed setting" button 230 ... Average height reference detailed setting screen 240 ... plane standard detailed setting screen 250 ... free curved surface standard setting screen 252 ... "extraction size" designation column 260 ... initial screen 261 ... flow display area 262 ... third image Display area 263 ... "imaging" processing unit 264 ... "Shapetrax2" processing unit 265 ... "position correction" processing unit 266 ... "height measurement" processing unit; 266B ... second "height measurement" processing unit 267 ... "blob" Processing unit: 267B: “color inspection” processing unit 268 ... “edit” button 269 ... imaging setting menu 270 ... image registration screen 271 ... “camera selection” column 272 ... “registration” button 280 ... imaging setting screen 282 ... “detailed setting Button 284 ... "imaging setting" button 290 ... three-dimensional measurement setting screen 292 ... "display with continuous update" field 294 ... shutter speed setting field 295 ... numerical value display field 296 ... density range setting field 310 ... preprocessing setting field 312 ... Measurement impossible standard setting column 314 ... Even interval processing setting column 316 ... Space code setting column 318 ... Projector selection setting column 322 ... "display image" selection column 324 ... "edit" button 326 ... extraction area editing dialog 330 ... mask area setting column 332 ... "edit" button 334 ... mask area editing dialog 340 ... filter processing setting screen 350 Binary level setting screen 360 Judgment condition setting screen 370 First sub menu 372 Second sub menu 373 "Measurement" menu 380 Image setting screen 382 Image selection column 384 Image setting column 386 ... " Input image "selection column 390 ... image variable selection screen 460 ... height measurement setting screen 620 ... area setting screen 630 ... free curved surface standard setting screen 632 ..." extraction direction "specification field 640 ... detail setting screen WK, WK7, WK8, WK9 , WK10, WK11, WK12, WK13, WK14 ... Work PC ... Personal computer Data SA ... searched area SI ... eyedropper tool icon ROI ... measured region BC ... belt conveyor DE ... dents

Claims (10)

A three-dimensional image processing apparatus capable of acquiring a distance image including height information of an inspection object and performing image processing based on the distance image,
An imaging unit that captures an image of an inspection object;
Distance image generation means capable of generating a distance image based on the image taken by the imaging means;
Display means for displaying the distance image generated by the distance image generation means;
In a setting mode for performing necessary settings, setting means for receiving an operation from the user and performing various settings;
In an operation mode in which three-dimensional image processing is performed by the three-dimensional image processing apparatus after the setting mode, height information of the distance image, the number of gradations lower than the number of gradations of the distance image, Tone conversion means for performing tone conversion to a low tone distance image in which
An inspection execution unit for executing a predetermined inspection process on the low gradation distance image;
The setting means is
Reference plane setting for setting a reference plane serving as a reference for performing gradation conversion as a gradation conversion parameter that constitutes a gradation conversion condition when performing gradation conversion on the distance image generated by the distance image generation unit Means,
An extraction method selection unit that receives selection of either static conversion or dynamic conversion;
In the operation mode, the gradation conversion unit
When static conversion is selected by the extraction method selection means, the floor by the tone conversion means is applied to the newly input distance image with reference to the height of the reference plane set in the setting mode. Perform key conversion,
When the dynamic conversion is selected by the extraction method selection means, the floor by the tone conversion means is set based on the height of the newly set reference plane based on the newly input distance image. A three-dimensional image processing apparatus characterized in that it is configured to perform tonal conversion. The three-dimensional image processing apparatus according to claim 1, wherein
The reference plane setting means is configured to set height information of the designated point as a reference plane by designating an arbitrary point in the distance image displayed on the display means. Three-dimensional image processing device characterized by The three-dimensional image processing apparatus according to claim 1, wherein
The reference plane setting means is configured to set height information of the designated point as a reference plane by specifying an arbitrary point in the distance image displayed on the display means. Three-dimensional image processing device characterized by The three-dimensional image processing apparatus according to claim 1, wherein
The reference surface setting means is configured to set height information of the specified surface as a reference surface by specifying an arbitrary surface in the distance image displayed on the display means. Three-dimensional image processing device characterized by The three-dimensional image processing apparatus according to claim 1, wherein
The reference surface setting unit is configured to set height information of a surface including the designated three points as a reference surface by specifying any three points in the distance image displayed on the display unit. A three-dimensional image processing apparatus characterized in that The three-dimensional image processing apparatus according to claim 1, wherein
Three-dimensional image processing apparatus characterized in that the reference surface setting means calculates a curved surface from the distance image displayed on the display means, and sets the calculated curved surface as a reference surface. . The three-dimensional image processing apparatus according to any one of claims 1 to 6 , further comprising: projecting incident light as structured illumination of a predetermined projection pattern from an oblique direction with respect to the optical axis of the imaging means. It is equipped with light emitting means,
The imaging unit acquires reflected light which is projected by the light projecting unit and reflected by the inspection object and captures a plurality of pattern projection images.
The three-dimensional image processing apparatus, wherein the distance image generation unit is configured to generate a distance image based on a plurality of pattern projection images captured by the imaging unit. The three-dimensional image processing apparatus according to claim 6 , further comprising: a projection unit for projecting a light onto the inspection object by a light cutting method,
The imaging unit is configured to capture an image projected by the projection unit, and the distance image generating unit is configured to combine the profiles obtained by the light cutting method to generate a distance image. Three-dimensional image processing device. A three-dimensional image processing program capable of acquiring a distance image including height information of an inspection object and performing image processing based on the distance image, the computer comprising
Based on a plurality of pattern projection images captured by projecting incident light as structured illumination of a predetermined projection pattern from a direction oblique to the optical axis of the imaging means and acquiring reflected light reflected by the inspection object And a distance image generation function capable of generating a distance image,
A function of displaying a distance image generated by the distance image generation function;
In the setting mode for performing necessary settings, as a setting function that accepts operations from the user and performs various settings,
Reference plane setting for setting a reference plane serving as a reference for performing gradation conversion as a gradation conversion parameter constituting a gradation conversion condition when performing gradation conversion on a distance image generated by the distance image generation function Function,
An extraction method selection function that accepts selection of either static conversion or dynamic conversion;
In the operation mode in which three-dimensional image processing is performed after the setting mode, the distance image is replaced with the gray level value of the image of the distance image having a gradation number lower than that of the distance image. An image tone conversion function for tone conversion to a low tone distance image;
An inspection execution function for executing a predetermined inspection process on the low gradation distance image is realized, and the gradation conversion function operates in the operation mode,
When static conversion is selected by the extraction method selection function, the floor by the gradation conversion function is applied to the newly input distance image with reference to the height of the reference plane set in the setting mode. Perform key conversion,
When the dynamic conversion is selected by the extraction method selection function, the floor by the gradation conversion function is referenced based on the height of the newly set reference plane based on the newly input distance image. A three-dimensional image processing program characterized by performing tone conversion. A computer readable recording medium or an apparatus recorded with the three-dimensional image processing program according to claim 9 recorded thereon.