TWI791174B - Optical inspection method for detecting internal threads by using ai deep learning technique - Google Patents

Abstract

An optical inspection method for detecting internal threads by using AI deep learning technique, including: performing an image capturing step to obtain a screw image, performing an image classification step to classify all the obtained screw images into multiple screw reference images and multiple screw abnormal images, performing an image rotation step on the multiple reference screw images and the multiple abnormal screw images to obtain the multiple reference rotation images and the multiple abnormal rotation images respectively, performing an image cropping step and an image optimization step on the multiple reference rotation images and the multiple abnormal rotation images respectively, and then obtaining multiple reference optimized images and multiple abnormal optimized images, performing a neural network training step on the multiple reference optimized images and the multiple abnormal optimized images, and establishing a training model, providing an image of the screw to be identified, and sending the image into the training model for an identification and comparison step, and finally, performing a classification suggestion step, and classifying the image to be identified as normal screw nut misaligned screw nut or defective screw nut based on the result of the identification comparison step.

Description

Optical inspection method for internal thread thread inspection using deep learning

The invention relates to an optical detection method, in particular to an optical detection method using deep learning to detect internal thread threads.

With the advancement of science and technology, the nuts used in some mechanical devices are becoming more and more precise, and the nuts often encounter internal thread bevels or defects during the production process. When these abnormal conditions increase, the nut production The lower the quality and yield rate, and these abnormal phenomena are not easy to be detected because they are located in the internal thread, because when the size and length of the nut are longer, this will cause the screw and the nut to not be completely screwed together , or the occurrence of improper engagement between the screw teeth, due to its high-precision components, this is a very serious defect problem.

The conventional technology of the Republic of China Invention Patent No. 1345936 is mainly to determine whether there is a positioning pin in the guide and positioning cylinder. The magnetic element can avoid the skewing of the thimble caused by the traditional screw teeth engagement structure, and improve the accuracy of the positioning pin assembly.

The conventional technology of the Republic of China New Patent No. M355371 mainly uses a magnifying glass to enlarge the image of the internal thread captured by the image capture unit to increase the recognition of the image, and it is still manually recognized, so it cannot be accurately recognized and cannot be automated. It makes it impossible to identify whether the thread is good or bad.

The known technology of the Republic of China Application No. 099138839 judges whether the thread front is deformed through the state of the reflected light of the thread front. However, this conventional technology cannot accurately identify the defect state of the thread, the bevel, etc., and there is no artificial intelligence to automate the entire identification system. .

None of the above-mentioned conventional technologies discloses how to solve the problem of defects and bevels in the screw teeth when the factory manufactures screws and nuts, so it is necessary to improve them.

One purpose of the present invention is to provide an optical detection method using deep learning to detect internal thread threads, and to detect the state of the thread pattern of the thread through image recognition.

Another object of the present invention is to provide an optical detection method using deep learning to detect internal thread threads, which increases the learning efficiency of the recognition algorithm and improves the recognition accuracy by rotating the thread image.

In order to achieve the above and other objectives, an optical detection method of the present invention that uses deep learning to detect internal thread screws includes: an illumination imaging step, illuminating the nut from directly above the screw hole to obtain an illumination image of the screw teeth ; an image classification step, classifying all obtained screw tooth illumination images into a plurality of screw tooth reference images and a plurality of screw tooth abnormal images; performing an image rotation step on the plurality of reference screw tooth images and a plurality of abnormal screw tooth images, the image The rotation step is to rotate each image at an angle of 0 to 360 degrees and capture the image, respectively obtain a plurality of reference rotation images and a plurality of abnormal rotation images; perform an image cropping on the plurality of reference rotation images and the plurality of abnormal rotation images respectively step and an image optimization step, and obtain plural reference optimized images and plural abnormal optimized images; perform a neural network training step on the plural reference optimized images and the plural abnormal optimized images, and establish a training model; provide a screw to be identified tooth image, sending the image of the thread to be identified into the training model for an identification and comparison step; and performing a classification suggestion step, classifying the image of the thread to be identified as normal thread and defect according to the result of the identification and comparison step Threaded or beveled thread.

In some embodiments of the present invention, the plurality of thread reference images are normal thread images, and the plurality of abnormal thread images are oblique thread images or defective thread images.

In some embodiments of the present invention, in the image rotating step, each image is rotated 3-5 times.

In some embodiments of the present invention, wherein, in the image rotation step, each image is rotated at equal angles for 3-5 times and the image is captured, and the sum of the angles is 360 degrees.

In some embodiments of the present invention, the image optimization step is to perform gray scale processing on the image.

In some embodiments of the present invention, the identification comparison step is provided with an identification standard range value, the identification standard range is between -1~1, wherein, the identification standard range value of the normal screw thread is greater than 0.75 and less than or equal to 1, the identification standard range value of the beveled thread is greater than 0.5 and less than or equal to 0.75, and the identification standard range value of the defective thread is greater than or equal to -1 and less than or equal to 0.5.

S0: Illuminated imaging step

S1: Image classification step

S2: Image rotation step

S3: Image cropping step

S4: Image optimization steps

S5: Neural Network Training Steps

S6: Identification and comparison steps

S7: Classification Proposal Step

Figure 1 is a flow chart of an embodiment of the optical detection method of the present invention using deep learning to detect internal thread threads; Figure 2 is the normal thread and bevel angle of the present invention's optical detection method using deep learning to detect internal thread threads Screw tooth comparison chart; Fig. 3 is a comparison diagram of normal thread and defective thread of the optical detection method using deep learning to detect internal thread threads of the present invention; Fig. 4 is a normal thread of the present invention using deep learning to detect optical detection method of internal thread threads Perspective view of teeth and bevel screw teeth.

FIG. 1 is a flowchart of an embodiment of the optical detection method of the present invention using deep learning to detect internal thread threads, please refer to FIG. 1 . An optical detection method of the present invention using deep learning to detect internal thread threads is described as follows. An illumination imaging step S0 , illuminating the nut from directly above the screw hole to obtain an illumination image of the screw teeth. The illumination imaging step S0 is to capture the image after illuminating the nut to be classified. This step can obtain image samples of three screw tooth states: normal, beveled, and defective. When the light is irradiated from the top to the bottom of the screw hole , the image of the thread pattern similar to concentric circles can be obtained through image shooting.

An image classification step S1 , classifying all the acquired screw tooth illumination images into a plurality of screw tooth reference images and a plurality of screw tooth abnormal images, which are used as training data for neural network learning in a subsequent neural network training step S5 . Preferably, in the image classification step S1, the plurality of thread reference images belong to normal thread images, and the plurality of abnormal thread images are further divided into oblique reference thread images and defect reference thread images. In the invention, the normal reference thread image and the oblique reference thread image are divided into one group, and the normal reference thread image and the defective reference thread image are divided into one group.

Next, an image rotation step S2 is performed on the plurality of screw reference images and the plurality of abnormal screw images. In the image rotation step S2, each image is rotated at an angle of 0 to 360 degrees and the image is captured to obtain the plurality of reference rotations respectively. An image and a plurality of abnormally rotated images, wherein the image is rotated in step S2 The system first coordinates and locates each image, and then rotates each image according to the requirements. For example, an image rotated by 0 degrees means that it is the original image, and a rotation of 90 degrees means that the image is rotated 90 degrees counterclockwise. By rotating the image angle, you can increase the number of samples number, and increase the accuracy of subsequent neural network learning.

Then, the plurality of reference rotation images and the plurality of abnormal rotation images are respectively subjected to an image cropping step S3 and an image optimization step S4, and a plurality of reference optimized images and a plurality of abnormal optimized images are obtained. The image cropping step S3 can delete Necessary image blocks, preferably, the image optimization step S4 is to process the image in grayscale, which can improve the feature value and facilitate the efficiency of subsequent neural learning and identification and judgment.

The complex number of reference optimization images and the plurality of abnormal optimization images are subjected to a neural network training step S5, and a training model is established. In the present invention, the neural network training step S5 utilizes the deep learning structure of the convolutional neural network It mainly consists of a convolutional layer (Convolutional), a pooling layer (Pooling) and a fully connected layer (Fully-connected). A training model database of normal screw teeth and defective screw teeth.

Figure 2 is a comparison of the normal thread and the beveled thread of the optical detection method using deep learning to detect internal thread threads of the present invention. Figure 3 is the normal thread of the present invention using deep learning to detect the optical detection method of internal thread threads Please refer to Figure 2 and Figure 3 for the comparison between thread and defective thread, in which Figure 2(a) is a beveled thread, and Figure 3(a) is a defective thread. Provide an image of the screw teeth to be identified, and send the image of the screw teeth to be identified into the training model to perform a recognition comparison step S6. In the identification and comparison step S6, the image of the nut to be identified is taken by the light Step 1: Obtain the image of the thread to be identified and send it into the training model for image recognition. In the identification comparison step S6, first identify whether it is a beveled thread, and if it is not a beveled thread, then identify whether it is Normal thread or defective thread, if it is a normal thread, it will be recognized as a normal thread. It can be seen from Figure 2(a) that the screw thread of the bevel nut is in the shape of a non-concentric circle.

Fig. 4 is a perspective view of the normal thread and beveled thread of the optical inspection method of the present invention using deep learning to detect internal thread threads, please refer to Fig. 4 . From Figure 4(a), it can be clearly seen that the centerline of the beveled nut presents an included angle with the centerline of the normal nut.

Executing a classification suggestion step S7, classifying the image of the thread to be identified as normal thread, beveled thread or defective thread according to the result of the identification and comparison step S6, the classification suggestion step S7 is based on the identification and comparison step S6 The result in output a result report, the result report contains the statistical data of classification results for each nut to be identified.

Preferably, in the image rotation step, each image is rotated 3 to 5 times, such as 3 rotations, and when the number of rotations and efficiency are 3 to 5 times, it will be better for the neural network training step S5. training effect.

Preferably, in the image rotation step, each image is rotated 3 to 5 times at equal angles and the image is captured, and the sum of the angles is 360 degrees. One image is captured at 270 degrees and 270 degrees, and at this time, there will be a better training effect for the neural network training step S5.

Preferably, the number of the plurality of reference rotation images and the plurality of abnormal rotation images are the same, wherein, when the number of normal reference thread images and bevel reference thread images are the same, or the normal reference thread images and defect reference images When the number of thread images is the same, there will be a better effective training result in the neural network training step S5.

Preferably, the identification comparison step S6 is provided with an identification standard range value, the identification standard range is between -1~1, wherein, the identification standard range value of the normal screw thread is greater than 0.75 and less than or equal to 1, the skew The identification standard range of angular screw teeth is greater than 0.5 and less than or equal to 0.75, and the identification standard range value of the defective screw tooth is greater than or equal to -1 and less than or equal to 0.5. The identification standard range refers to the image of the screw tooth to be identified In this identification comparison step S6, compare the value range with the similarity of the training model, when the similarity ratio When the pair value is greater than 0.75 and less than or equal to 1, it is judged as a normal thread; when the comparison value of the similarity is greater than 0.5 and less than or equal to 0.75, it is judged as a beveled thread; when the comparison value of the similarity is When it is greater than or equal to -1 and less than or equal to 0.5, it is judged as the defective thread.

The spirit of the present invention is to capture images of various angles by rotating the images as neural network learning training samples, and divide the normal reference thread image and the oblique reference thread image into one group, and separate the normal reference thread image and the defect The reference thread images are divided into one group, which helps to improve the recognition accuracy.

The above-mentioned embodiments are only to illustrate the technical ideas and characteristics of the present invention, and its purpose is to enable those with ordinary knowledge in the technical field to understand the content of the present invention and implement it accordingly, and should not limit the scope of the present invention. The scope of the patent, all equivalent changes or modifications made in accordance with the spirit of the present invention and the content of the description shall be covered within the scope of the patent of the present invention.

S0: Illuminated imaging step

S1: Image classification step

S2: Image rotation step

S3: Image cropping step

S4: Image optimization steps

S5: Neural Network Training Steps

S6: Identification and comparison steps

S7: Classification Proposal Step

Claims (3)

An optical detection method using deep learning to detect internal thread screws, including: an illumination imaging step (S0), illuminating the nut from directly above the screw hole to obtain an illumination image of the screw teeth, and obtaining normal, bevel and Image samples of the three kinds of thread states of defects; an image classification step (S1), classifying all obtained light images of the thread into plural thread reference images and plural thread abnormal images, the plural thread reference images including a normal The reference thread images, the plurality of abnormal thread images include a beveled reference thread image and a defective reference thread image, and then the normal reference thread image and the beveled reference thread image are divided into one group, and the The normal reference thread image and the defective reference thread image are divided into a group as training data for a subsequent neural network training step (S5); an image rotation step is performed on the plurality of thread reference images and the plurality of abnormal thread images (S2), the image rotation step is to rotate each image 3 to 5 times at an equal angle and capture the image, and the sum of the angles is 360 degrees, respectively obtain a plurality of reference rotation images and a plurality of abnormal rotation images, wherein the plurality of reference The numbers of the rotated image and the plurality of abnormally rotated images are the same; the plurality of reference rotated images and the plurality of abnormally rotated images are respectively subjected to an image cropping step (S3) and an image optimization step (S4), and a plurality of reference optimized images and Plural anomaly optimization images; the neural network training step (S5) is performed on the plurality of reference optimization images and the plurality of anomaly optimization images, and a training model is established, the training model includes a training model of normal screw teeth and beveled screw teeth Database, and a training model database of normal screw teeth and defective screw teeth; provide a screw tooth image to be identified, send the image of screw teeth to be identified into the training model for an identification comparison step (S6), first borrow From the training model database of the normal thread and beveled thread, identify whether it is a beveled thread, If it is not a beveled thread, then use the training model database of the normal thread and defective thread to identify whether it is a normal thread or a defective thread, and if it is a normal thread, then identify and confirm that it is a normal thread; and perform a In the classification suggestion step (S7), the image of the thread to be identified is classified as a normal thread, a defective thread or a beveled thread according to the result of the identification and comparison step (S6). According to claim 1, an optical detection method using deep learning to detect internal thread threads, wherein the image optimization step (S4) is to perform grayscale processing on the image. An optical detection method using deep learning to detect internal thread threads as described in claim 1, wherein the identification and comparison step (S6) is provided with an identification standard range value, and the identification standard range is between -1~1, wherein , the identification standard range value of the normal thread is greater than 0.75 and less than or equal to 1, the identification standard range value of the beveled thread is greater than 0.5 and less than or equal to 0.75, the identification standard range value of the defective thread is greater than or Equal to -1 and less than or equal to 0.5.

Images