JP2019215240A - Teacher image generation method of appearance inspection device - Google Patents

Abstract

To provide a teacher image generation method of an appearance inspection device capable of easily generating a teacher image.SOLUTION: A teacher image generation method of an appearance inspection device virtually arranges an object 8 of an inspection target, a viewpoint 6 and a light source 7 in a virtual space 1 of a computer 20, constructs the surface of the object 8 as a three-dimensional model on the basis of data representing a three-dimensional shape of the object 8, calculating optical responses between the surface of the object 8 constructed as the three-dimensional model and light 9 emitted from the light source 7 to calculate the intensity and the color of light 11 entering the viewpoint 6 from the surface of the object 8, converts calculated data into an image by computer graphics, and outputs the image as a teacher image 5 showing how the object 8 in a prescribed virtual plane 12 is seen.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a teacher image generation method for a visual inspection device that inspects a surface defect such as a scratch, unevenness, or distortion.

  2. Description of the Related Art Conventionally, as an appearance inspection method for inspecting a surface defect such as a scratch, unevenness or distortion, a predetermined image processing is performed on a plane image obtained by imaging a surface (inspection surface) of an object to be inspected to be inspected (an inspection surface) with a camera. By comparing the inspection image with a reference image obtained by imaging a non-defective surface (reference surface) of a non-defective sample of an object to be inspected with a camera and performing the same image processing on the acquired flat image, the presence or absence of a defect is determined. There is a method of estimating the size and position of the inspection object and determining the quality of the inspection object. In such a method, it is common to compare an inspection image with a reference image and determine that there is a defect in an area where the difference in luminance, contrast, or color exceeds a threshold.

  In the conventional visual inspection method, it is necessary to prepare a reference image and an inspection image of the object, and the reference image only needs to be one image of a surface having no defect. Is required.

  By the way, since the captured image is generated by receiving the light interacting with the inspection target surface by the sensor of the camera, it is clear that the imaging result is affected by the three-dimensional shape of the inspection surface.

  The above-described three-dimensional shape of the inspection surface includes, for example, distortion or deformation of a relatively large size from a minute shape such as so-called grinding or grinding roughness of cutting or turning.

  Even if the defect to be detected on the inspection surface can be easily identified by a human, it is not easy to automatically and accurately distinguish the defect based on the global three-dimensional shape in the background. This is because an image processing method capable of detecting a specific defect has a narrow range in which an applicable shape can be changed, and the imaging conditions become severe.

  As a specific example, if the scratch due to the contact of the tool or product on the turning surface of the lathe part is at an angle with the turning trace made by feeding the bite, detection is relatively easy, If they are parallel, it is difficult to separate and distinguish the turning marks and scratches. If it is made so that parallel scratches can be detected, there arises a problem that the risk of determining non-scratched cuts as scratches increases.

  It is very difficult to suppress such a three-dimensional shape change over time until it can be easily separated from defects.In recent years, efforts have been made to apply a machine learning method such as deep learning that is remarkably progressing to realize an inspection that is strong against environmental changes. It is expanding rapidly and practical use is progressing.

  For machine learning, teacher images are required. Collect images that are labeled and classified according to the type of defects found (scratch, dent, distortion, etc.) and size (size, depth, location, etc.) and normal images without defects, These are applied to machine learning as teacher images to generate a classification network model.

  If a new inspection image is given to the generated network model, an inspection method that outputs the label of the closest teacher image, that is, an inspection method that can detect the presence or absence of a defect and perform label classification is provided. Since the classification model is created using the image itself that has been visually inspected and classified by humans as a model, it is resistant to variations in the appearance caused by variations in the three-dimensional shape, and if the quality of the teacher image is appropriate, the generated model will be accurate. I know I will work.

  Although such an effective means, a large amount of teacher signals is required to ensure the accuracy of a model created by machine learning. Generally, at least several hundreds to several thousand images, and in some cases, tens of thousands to hundreds of thousands of images are required.

  It is not easy to collect a large number of images of defects. In the first place, manufacturing sites are not aimed at creating defects. Also, at the collection stage, there is no method to automatically find defects, it is a work of visual inspection by humans, and it is necessary to take an image of the found defect and confirm that the image is the same as the naked eye It takes a lot of trouble.

  There is no doubt that the application of machine learning to visual inspection such as surface defect inspection is promising, but there is a problem in collecting teacher images.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a teacher image generating method of a visual inspection device capable of easily generating a teacher image.

  The object to be inspected, the viewpoint and the light source are virtually arranged in the virtual space of the computer, and the surface of the object is constructed as a three-dimensional model based on data representing the three-dimensional shape of the object, and the three-dimensional model is constructed. The surface of the object constructed as a model and the optical response of light emitted from the light source are calculated to calculate the intensity and color of light entering the viewpoint from the surface of the object, and the calculated data is calculated. A teacher image generation method for a visual inspection device that converts the image into an image by computer graphics and outputs the image as a teacher image indicating how the object looks in a predetermined virtual plane.

  A teacher image generation method for a visual inspection device, which may add a shape of a surface defect of the object to the three-dimensional model and output an image including the surface defect as the teacher image.

  A teacher image generation method for a visual inspection device, which may automatically change the position or orientation of the object and automatically output the appearance of the object as the teacher image for each change.

  A teacher image generation method for a visual inspection device, wherein the teacher image is compared with an image of an actual object of the object captured by a camera, and the appearance of the teacher image is adjusted based on a result of the comparison.

  According to the present invention, a teacher image of a visual inspection device can be easily generated.

1 shows a configuration of a computer for performing a teacher image generation method of a visual inspection device according to the present invention. 1 conceptually illustrates a teacher image generation method of a visual inspection device according to the present invention. 1 shows a specific example of a data acquisition device that acquires data for generating a teacher image. It is a front view which shows the specific example of an object position and attitude | position control means. It is a side view which shows the specific example of an object position-and-orientation control means. It is a side view for explaining operation of the example of an object position and posture control means.

  FIG. 1 shows a configuration of a computer for implementing a teacher image generating method of a visual inspection device according to the present invention. FIG. 2 conceptually shows a teacher image generating method of the visual inspection apparatus according to the present invention. The computer 20 operates so as to form a virtual space 1 in which the object 8 to be inspected, the viewpoint 6 and the light source 7 are virtually arranged, and the data representing the three-dimensional shape of the object 8 Data processing means 3 for constructing the surface of the object 8 as a three-dimensional model from 2 and an optical response 10 between the surface of the object 8 and light 9 emitted from the light source 7 are calculated, and the viewpoint 6 Calculate the intensity and color of the incoming light 11, convert the data into an image by CG (computer graphics), and output the image as a teacher image 5 showing how the object 8 appears on a predetermined virtual plane 12. Data processing means 4.

  The data 2 representing the three-dimensional shape of the object 8 may be any format that can represent the three-dimensional shape, and is generally in the STL format, but may be a file in which XYZ coordinate values are described in text data. Further, the intermediate file format of the three-dimensional CAD, such as STEP and IGES, may be programmed to be readable.

  The process for generating the teacher image 5 begins with the user arranging the light source 7 and the viewpoint 6 at appropriate XYZ positions in the virtual space 1, and then the tertiary of the object 8 whose appearance is to be known. The original data 2 is arranged at a designated position of coordinates (for example, the origin of XYZ), and then the three-dimensional data 2 is given to the data processing means 3 to construct a three-dimensional model of the object 8 and the position of the surface of the object 8 And passes the data to the data processing means 4.

  The data processing means 4 calculates the optical response of the light beam that has arrived at the surface of the object 8 from the light source 7 arranged first based on the data received from the data processing means 3, and uses the reflected light and the diffused light as the response light. Calculate the direction, intensity and color of light. Next, a response light entering the viewpoint 6 is obtained, and a point where the response light intersects the virtual plane 12 perpendicular to the direction in which the viewpoint 6 is directed is given a predetermined light intensity and color by CG, and is output as the teacher image 5.

  The output of the teacher image 5 is performed every time the user operates the computer 20, or is automatically performed by an electronic program installed in the computer 20.

  The teacher image 5 is one or both of an image including a surface defect (scratch, dent, distortion, etc.) of the object 8 and an image not including the surface defect. The surface defect is not limited to a defect that has actually occurred, but may be a defect that is expected to occur.

  By giving the teacher image 5 to a neural network (deep learning) having a multilayer structure and learning it, a network model for appearance inspection is generated, and an inspection image of the object 23 is given to the network model. The quality of the appearance is automatically determined.

  FIG. 3 shows a specific example of a data acquisition device for acquiring data for generating a teacher image. In this specific example, the data acquisition device captures an image of the surface of the object 23 and a three-dimensional shape measuring device 21 that measures the actual three-dimensional shape of the object 23 that is manufactured based on the three-dimensional shape data of the object 8. Two-dimensional camera 22, position and orientation control means 24 for controlling the position and orientation of object 23, three-dimensional shape measurement data of object 23 output from three-dimensional shape measuring device 21, object output from two-dimensional camera 22 And a computer 20 for receiving the image data of the object 23 and the position and orientation data of the object 23 output from the position and orientation control means 24.

  The positional relationship between the two-dimensional camera 22 and the object 23 in FIG. 3 is the same as the positional relationship between the viewpoint 6 and the object 8 in the virtual space 1 in FIG. The positional relationship between the illumination (not shown) illuminating the object 23 and the object 23 in FIG. 3 is the same as the positional relationship between the light source 7 and the object 8 in the virtual space 1 in FIG.

  The teacher image 5 of the object 8 is all obtained by calculation in the virtual space, and its appearance quality must be close to the actual appearance of the real object 23. In the specific example of FIG. 3, the positional relationship between the object 23, the illumination (not shown), and the two-dimensional camera 22 is made the same as the positional relationship in the virtual space 1 in FIG. By comparing the generated teacher image 5 and the image of the object 23 captured by the two-dimensional camera 22, the degree of similarity (difference) between them can be measured. By quantifying the degree of approximation and adjusting optical characteristics such as reflection, diffusion, and color of the surface of the object 8 so as to reduce the difference, the teacher image 5 can be made closer to the actual appearance.

  FIG. 4 is a front view showing a specific example of the object position and orientation control means. FIG. 5 is a side view showing a specific example of the object position and orientation control means. FIG. 6 is a side view for explaining the operation of a specific example of the object position and orientation control means. 4 to 6 show the same specific examples. This specific example is a specific example of the object position / posture control means 24 shown in FIG. This specific example includes a drive link group including a first drive link 31a, a second drive link 31b, a third drive link 31c, and a fourth drive link 31d, a first driven link 32a, a second driven link 32b, and a third driven link 32b. A driven link group including a driven link 32c and a fourth driven link 32d, a YZθ-axis first drive rotary shaft (also X-axis linear guide) 33a, and a YZθ-axis second drive rotary shaft (also X-axis linear guide) 33b. YZθ-axis drive rotary shaft (also X-axis linear guide) 33c composed of YZθ-axis third drive rotary shaft (also X-axis linear guide) 33c and YZθ-axis fourth drive rotary shaft (also X-axis linear guide) A YZθ-axis first drive motor 34a (not shown), a YZθ-axis second drive motor 34b (not shown), a YZθ-axis third drive motor 34c (not shown), and a YZθ-axis fourth drive motor 34d axis And dynamic motor group, and X-axis ball screw 35, the X-axis slider 36, the X-axis driving motor 37, and a work stand 38, in the parallel link mechanism constituted.

  The upper ends of the first drive link 31a, the second drive link 31b, the third drive link 31c, and the fourth drive link 31d are respectively a first driven link 32a, a second driven link 32b, a third driven link 32c, And each lower end of the fourth driven link 32d is connected rotatably in the θ direction.

  Upper ends of the first driven link 32a, the second driven link 32b, the third driven link 32c, and the fourth driven link 32d are respectively connected to the lower part of the work placing table 38 so as to be rotatable in the θ direction. .

  The lower ends of the first drive link 31a, the second drive link 31b, the third drive link 31c, and the fourth drive link 31d are respectively YZθ-axis first drive rotary shaft (also X-axis linear motion guide) 33a, YZθ. Shaft second drive rotation shaft (also X-axis linear motion guide) 33b, YZθ-axis third drive rotation shaft (also X-axis linear motion guide) 33c, and YZθ-axis fourth drive rotation shaft (also X-axis linear motion guide) 33d Are connected rotatably in the θ direction.

  YZθ-axis first drive rotation axis (also X-axis linear guide) 33a, YZθ-axis second drive rotation axis (also X-axis linear guide) 33b, YZθ-axis third drive rotation axis (also X-axis linear guide) 33c , And one end of the YZθ-axis fourth drive rotary shaft (also X-axis linear guide) 33d are respectively a YZθ-axis first drive motor 34a (not shown), a YZθ-axis second drive motor 34b (not shown), The YZθ-axis third drive motor 34c (not shown) and the YZθ-axis fourth drive motor 34d are connected to respective power shafts.

  The X-axis slider 36 is held by the X-axis ball screw 35 so as to be able to move directly in the X direction.

  One end of the X-axis ball screw 35 is connected to the power shaft of the X-axis drive motor 37.

  In the parallel link mechanism described above, when each drive motor of the YZθ-axis drive motor group is selectively operated, each power causes the respective YZθ-axis drive rotation axis (also X-axis linear guide) group to rotate in the θ direction. Selectively rotate, and by the rotation, each drive link of the drive link group is selectively rotated at a predetermined angle in the θ direction about each rotation axis, and each driven link of the driven link group is linked to the rotation. Takes a predetermined posture, for example, as shown in FIG. 6, the work mounting table 38 moves in the Y direction and rotates in the θ direction.

  Further, when the X-axis drive motor 37 is operated, the X-axis ball screw 35 is rotated by the power thereof, and the X-axis slider 36 moves in the X direction along the extending direction of the X-axis ball screw 35 by the rotation, When the power due to the movement is transmitted directly or indirectly to the drive link group, the drive link group moves in the X direction along the extending direction of the YZθ-axis drive rotation axis (also X-axis linear motion guide) group, Along with the movement, the work placing table 38 moves in the X direction.

  When the object 23 is placed on the work table 38, the position and the posture of the object 23 can be changed together with the work table 38. While changing the position and orientation of the object 23 stepwise or continuously, the three-dimensional shape measuring device 21, the two-dimensional camera 22, and the computer 20 use the three-dimensional shape data and two-dimensional image of the object 23 at that position and orientation. And the position and orientation data are acquired in association with each other, data for generating the teacher image 5 can be efficiently collected. This series of processes may be configured to be automatically executed by the position and orientation control means 24, the three-dimensional shape measuring machine 21, the two-dimensional camera 22, and the computer 20 working together.

  The specific examples shown in FIGS. 4 to 6 are configured to move the object 23 in the XYZ directions and around the X axis, but may be configured to move the object 23 also around the YZ axes.

Reference Signs List 1 virtual space 2 data representing 3D shape of object 3 data processing means 4 data processing means 5 teacher image 6 viewpoint 7 light source 8 object 9 light 10 optical response 11 light 12 virtual plane 20 computer 21 3D shape measuring machine 22 Two-dimensional camera 23 Object 24 Object position and orientation control means 31a First drive link 31b Second drive link 31c Third drive link 31d Fourth drive link 32a First driven link 32b Second driven link 32c Third driven link 32d Fourth Followed link 33a YZθ axis first drive rotation axis (also X axis linear motion guide)
33b YZθ axis second drive rotation axis (also X axis linear motion guide)
33c YZθ axis third drive rotation axis (also X axis linear motion guide)
33d YZθ axis fourth drive rotation axis (also X axis linear motion guide)
34a YZθ axis first drive motor (not shown)
34b YZθ axis second drive motor (not shown)
34c YZθ axis third drive motor (not shown)
34d YZθ-axis fourth drive motor 35 X-axis ball screw 36 X-axis slider 37 X-axis drive motor 38 Work platform

Claims (4)

  The object to be inspected, the viewpoint and the light source are virtually arranged in the virtual space of the computer, and the surface of the object is constructed as a three-dimensional model based on data representing the three-dimensional shape of the object, and the three-dimensional model is constructed. The surface of the object constructed as a model and the optical response of light emitted from the light source are calculated to calculate the intensity and color of light entering the viewpoint from the surface of the object, and the calculated data is calculated. A teacher image generation method for a visual inspection apparatus, comprising: converting an image into an image by computer graphics; and outputting the image as a teacher image indicating how the object looks in a predetermined virtual plane.   The teacher image generation method according to claim 1, wherein a shape of a surface defect of the object is added to the three-dimensional model, and an image including the surface defect is output as the teacher image.   The teacher of the visual inspection apparatus according to claim 1, wherein a position or a posture of the object is automatically changed, and the appearance of the object is automatically output as the teacher image for each change. Image generation method.   4. The method according to claim 1, wherein the teacher image is compared with an image of an actual object captured by a camera, and the appearance of the teacher image is adjusted based on a result of the comparison. A teacher image generating method for the visual inspection device according to any one of the first to third aspects.

Images