TWI745946B - A golf ball computer inspection system and automatic optic inspection apparatus - Google Patents

Abstract

An optical inspection system for inspecting golf ball detects using deep learning is provided. The golf ball detects are identified and classified by use of a pre-trained deep-learning model for defect inference, a pre-trained deep-learning model for logo inference and a pre-trained deep-learning model for wind-tunnel inference. The golf ball defects can be validated as identical as human inspection based on a comparison of the outcomes from the pre-trained deep-learning models with a limitation logic.

Description

Golf computer detection system and automatic optical detection equipment

The invention relates to a golf computer detection system and automatic optical detection equipment.

AOI (Automatic Optic Inspection) is an optical inspection equipment that uses an image sensor to automatically scan the image information of the object to be tested, and to detect defects or blemishes on the object to be tested. It is often used in the factory production process to automatically detect defective products or product defects and blemishes through optical inspection equipment. Automatic optical inspection equipment usually needs to be equipped with a corresponding optical design for a certain kind of defect or defect. Therefore, in order to detect multiple types of defects or different defects on the object to be tested, a variety of corresponding optical designs are usually required to detect. Therefore, automatic optical inspection equipment is often limited to a set of specific optical designs that can only detect a corresponding defect or flaw. To detect multiple defects or flaws, multiple sets of optical designs are required.

The golf computer inspection system of the first aspect of the present invention includes a microprocessor set to receive image information of a golf ball to be tested. The microprocessor is configured to identify and classify defects on the surface of the golf ball to be tested according to a first deep learning inference model and output a defect feature vector, and identify and classify the golf ball to be tested according to a second deep learning inference model The trademark also outputs a trademark feature vector, recognizes the wind tunnel style of the golf ball surface to be tested according to a third deep learning inference model, and outputs a wind tunnel style feature vector. A computer recognition module calculates a set of limit values based on the defects, trademarks, wind tunnel style feature vectors, and the image parameters of the golf ball to be tested. The set of limit values corresponds to the values of at least one limit dimension and is based on a limit logic Compare the limit value of the group with an acceptance criterion to determine Whether the set of limit values is consistent with the acceptance criteria, so as to screen out golf defects that are consistent with those detected by human judgment.

The automatic optical inspection equipment of the second aspect of the present invention at least includes an image sensor for capturing image information of the surface of a golf ball to be tested, and a golf ball for carrying the golf ball to be tested and capable of rotating the golf ball to be tested Carrying, and the golf computer detection system described in the first point of view

According to the present invention, a single automatic optical inspection system can be used to complete the optical inspection function of multiple defects and flaws, without the need for additional sets of optical designs.

100: Automatic optical inspection equipment

101: image sensor

102: golf ball bearing

103: Auxiliary light source device

104: Computer Identification Expert Group

105: Golf ball to be tested

200: Golf detection software system

201: Deep Learning Training Unit

202: Deep Learning Inference Unit

203: Material number establishment management unit

204: defect mark management unit

205: device control subunit

206: Report Management Subunit

301: Defect identification deep learning inference model

303: Inference model of trademark recognition deep learning

305: Wind tunnel style deep learning inference model

S401~S407: steps

Fig. 1 shows an automatic optical inspection device according to an embodiment of the present invention.

FIG. 2 is a diagram showing the architecture of a golf ball detection system according to an embodiment of the present invention.

FIG. 3 is a diagram showing the structure of a golf defect inference model according to an embodiment of the present invention.

Fig. 4 is a schematic diagram of a logical inference flow of utilization limit according to an embodiment of the present invention.

DETAILED DESCRIPTION The present disclosure will now be described more fully hereinafter with reference to the accompanying drawings. The drawings show exemplary embodiments of the present disclosure. However, the present disclosure is implemented in many different forms and should not be interpreted as being limited to the exemplary embodiments set forth herein. On the contrary, these exemplary embodiments are provided to make the present disclosure thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

The terms used herein are only used for the purpose of describing specific exemplary embodiments, and are not intended to limit the present invention. As used herein, unless the context clearly dictates otherwise, the singular forms "a", "an" and "the" are intended to also include the plural forms. In addition, when used herein, "includes" and/or "includes" or "includes" and/or "includes" or "has" and/or "has" steps, operations, components and/or groups thereof, But it does not exclude the presence or addition of one or more other features, steps, operations, components and/or groups thereof.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this disclosure belongs. In addition, unless clearly defined in the context, terms such as those defined in a general dictionary should be interpreted as having meanings consistent with their meanings in the related art and the present disclosure.

The following content will describe exemplary embodiments with reference to the accompanying drawings. It should be noted that the components depicted in the reference drawings are not necessarily shown to scale; and the same or similar components will be given the same or similar reference signs or similar technical terms.

Fig. 1 shows an automatic optical inspection (AOI) device according to an embodiment of the present invention. The automatic optical inspection equipment 100 includes an image sensor 101, a golf ball carrier 102, an auxiliary light source device 103 and a computer recognition module 104. The automatic optical inspection device 100 is an equipment used to detect whether a golf ball to be tested has defects, blemishes or defects. In the embodiments of the present invention, the golf ball’s defects, flaws, and defects mean that the golf ball is imperfectly detected through artificial judgment, and the defects/defects/defects can be substituted for each other in the context of the present disclosure. The image sensor 101 is used to capture an image of a golf ball 105 to be tested on the golf ball carrier 102. The image sensor 101 can be an area scan camera (Area Scan Camera) and a line scan camera (Line Scan Camera), and the two types of cameras or other types of photographing devices described in accordance with actual needs may be used. .

The auxiliary light source device 103 is used to illuminate the golf ball to be tested, and provides sufficient brightness so that the image sensor 101 can capture a clear enough surface image of the golf 105. The auxiliary light source device 103 may be a parallel light lamp, a diffuse light lamp, a dome lamp, etc.; in some embodiments, more than two sets of auxiliary light sources 103 may be used.

The golf ball carrier 102 is configured to rotate the golf ball 105 so that the image sensor 101 can capture at least sufficient image information of the golf ball surface, such as 360-degree surface surrounding image information. The image sensor 101 transmits the captured image information to the computer recognition module 104.

The computer identification module 104 includes at least one micro-processing unit and at least one storage medium. At least one storage medium stores more than one pre-trained deep learning inference model. Based on the image information of the golf ball to be tested, it is determined whether the golf ball to be tested has flaws or defects and can screen out true defects consistent with human judgment.

After the computer recognition module 104 receives the golf ball image information from the image sensor 101, the at least one processing unit executes the recognition process and infers the golf ball to be tested according to a plurality of pre-trained deep learning inference models. Ball defect types and defect locations, trademarks and trademark locations, wind tunnel styles and other characteristics.

In another embodiment of the present invention, the computer recognition module 104 can be assembled in the image sensor 101, so that the image sensor can directly use the deep learning inference model in the recognition module for golf defect identification and inference Function without additional computing hardware.

FIG. 2 shows the architecture of a golf ball detection software system according to an embodiment of the present invention. The golf inspection software system 200 includes a deep learning training unit 201, a deep learning inference unit 202, a material number establishment management unit 203, a defect mark management unit 204, an equipment control sub-unit 205 and a report management sub-unit 206, the interfaces between the units and The interaction is exemplified by arrows.

The training data required for golf identification is a training data set consisting of golf pictures with labels such as defective categories, trademarks, and wind tunnel styles. The golf training data set is used as the training data required for deep learning. Defect category labels include, but are not limited to, dirty paint, uneven painting, flow paint, surface scratches, paint residue, roughness, pin marks and lint, etc.; trademark labels include a single material number logo and head; wind tunnel style The label includes the arrangement and combination of wind tunnel and equatorial style, such as circular, hexagonal wind tunnel, wind tunnel size and total number of wind tunnels, etc.

The golf training data set can be stored in the defect mark management unit 204. The material number establishment management unit 203 establishes material number information of the golf ball to be tested. The deep learning training unit 201 trains the identification and classification of golf defects, trademarks, and wind tunnel styles using the label images of the training data set. The deep learning training unit 201 executes the training of the deep learning model, and the completed model parameters and weights can be downloaded Load it to the deep learning inference unit 202 for access. The device control sub-unit 205 is used to control various hardware devices of the automatic optical inspection device 100, such as the rotation speed and the degree of rotation carried by the golf ball, and the light intensity of the auxiliary light source device. The deep learning inference unit 202 makes inferences about golf defects, trademarks, and wind tunnel styles according to the deep learning model after the training is completed, and outputs the inference results to the report management sub-unit 206. The report management sub-unit 206 outputs an inference result report and stores the report in a database.

The deep learning training unit 201 receives instructions from the user through a graphical interface to perform deep learning training, and can store the training results in a network storage space NAS (Network Attached Storage). The deep learning inference unit 202 includes a device control sub-unit 205 and a report management sub-unit 206, and allows the user to control the device-related hardware through a graphical interface.

In an embodiment of the present invention, the computer recognition module 104 includes a deep learning inference unit 202, a device control sub-unit 205, and a report management sub-unit 206. Other units can be built in the same computer hardware as the computer identification module or in separate computer hardware.

The equipment control sub-unit 205 provides hardware equipment control, automation control and equipment status record files, and also includes the motor speed, motor forward and reverse rotation, input and output actuation and programmable logic control PLC (Programmable Logic Control) communication of each hardware device. Related parameters.

In an embodiment of the present invention, the deep learning training unit 201 is configured to build a deep learning model for identifying defects, for learning to identify and classify various golf defects or defects. The defect recognition deep learning model uses a convolutional neural network (Convolutional Neural Network) algorithm to train the identification and classification of golf defects with the label images in the golf image data set.

In an embodiment of the present invention, the defect recognition deep learning model includes an input layer, a convolution layer, a pooling layer, a full connection layer, and a The output layer. The golf training image is input by the input layer, and the convolutional layer and the pooling layer compress and process the input image data to extract features related to golf defects. The fully connected layer uses weights to identify and classify the image features obtained through voting. Finally, the output layer is normalized (for example, softmax operation) to output a defect feature vector that contains the confidence of various defect categories Information related to degree and location. The degree of confidence can correspond to the probability of belonging to a defect category. The pooling layer can use Max Pooling or Mean Pooling for subsampling to further simplify the image features obtained from the convolutional layer.

Through the mutual verification of the results of the output layer and the expected results, the function weights of the fully connected layer of the defect identification model are adjusted and updated. After several mutual verifications, the deep learning iterative process will tend to converge, and the training of the defect model will be completed at this time. The defect identification model after training has a specific weight combination, which is used to infer actual golf defects.

The types of golf defects that can be identified by the defect recognition deep learning model include, but are not limited to: dirty paint, uneven paint, flow paint, surface scratches, paint slag, roughness, pin marks, and lint, etc.

In an embodiment of the present invention, the deep learning training unit 201 is configured to build a deep learning model for identifying trademarks for learning to identify and classify various golf trademarks. The trademark recognition deep learning model uses the Convolutional Neural Network (CNN) algorithm to train the identification and classification of golf trademarks with the label images in the golf training data set.

In an embodiment of the present invention, the trademark recognition deep learning model includes an input layer, a convolution layer, a pooling layer, a full connection layer, and a The output layer. The golf training image is input by the input layer. The input image data is compressed by the convolutional layer and the pooling layer to extract the image features related to the trademark. The fully connected layer uses the weight to perform the trademark identification and the voting on the obtained image features. Classification, and finally the output layer outputs a trademark feature vector through a normalization process (for example, a softmax operation), the trademark feature vector includes the confidence level, position, and associated information of various trademark categories. The degree of confidence may correspond to the probability of belonging to a trademark. Pooling layer can use Max Pooling or Average Pooling (Mean Pooling) to perform subsampling to further simplify the image features obtained from the convolutional layer.

Through the mutual verification of the results of the output layer and the expected results, the function weights of the fully connected layer of the trademark recognition model are adjusted and updated. After several mutual verifications, the deep learning iterative process will tend to converge, and the training of the trademark model will be completed at this time. The trained trademark recognition model has a specific combination of weights and is used to infer actual golf trademarks.

In an embodiment of the present invention, the deep learning training unit 201 is combined to build a deep learning model for identifying the wind tunnel style, for learning to identify and classify the golf wind tunnel style. The wind tunnel style recognition deep learning model uses a convolutional neural network (Convolutional Neural Network) algorithm to train golf wind tunnel style recognition using label images in the golf training data set.

In an embodiment of the present invention, the wind tunnel pattern recognition learning model includes an input layer, a convolution layer, a pooling layer, a full connection layer, and a The output layer. The golf training image is input by the input layer. The input image data is compressed by the convolutional layer and the pooling layer to extract image features related to the wind tunnel style. The fully connected layer uses weights to vote the obtained image features into the wind tunnel style. Identification, and finally the output layer through normalization processing (for example, softmax operation) to output a wind tunnel style feature vector, the wind tunnel style feature vector contains the confidence level of the wind tunnel style category, location-related information. The confidence level can correspond to the probability of being in a wind tunnel style category. The pooling layer can use Max Pooling or Mean Pooling for subsampling to further simplify the image features obtained from the convolutional layer.

By comparing the results of the output layer with the expected results, adjust and update the function weights of the wind tunnel style identification model. After several mutual verifications, the deep learning iterative process will tend to converge, and the training of the wind tunnel style model will be completed at this time. The wind tunnel style identification model after training has a specific weight combination, which is used to infer the actual golf wind tunnel style.

In another embodiment of the present invention, during the training process of each of the above-mentioned deep learning models, according to practical requirements, the convolutional layer, the pooling layer, and the fully connected layer can be configured as a single layer or multiple layers, respectively.

In an embodiment of the present invention, each deep learning model trained by the deep learning training unit 201 can be downloaded to the deep learning inference unit 202 by the computer recognition module. In another embodiment, the computer recognition module can access the pre-trained deep learning inference models in the deep learning inference management unit 202 through the network storage space NAS.

When the image information of the golf ball to be tested is input to the computer recognition module, the computer recognition module can directly identify and infer golf defects according to the deep learning inference model that has been trained in advance; or, the depth of the training is completed. The learning inference model is directly downloaded or accessed to the network storage space for golf identification and inference. The defect recognition deep learning inference model outputs a defect feature vector based on the input image information, the trademark recognition deep learning inference model outputs a trademark feature vector based on the input image information, and the wind tunnel style recognition deep learning inference model outputs a wind tunnel based on the input image information Style feature vector.

In an embodiment of the present invention, the defect feature vector includes information related to the inference confidence level and coordinate position of each defect category. The computer recognition module calculates a set of limit values from the inference confidence level and the coordinate position in combination with the image information. The set of limit values includes the confidence value, area value, length value and depth value belonging to each defect category or any combination of the above values.

The confidence value is a quantitative value of the confidence level corresponding to each specific defect category, and its value is between 0 and 1.

The area value is the area size calculated after the coordinate position is transformed through the image pixels, and the unit is mm square. In an embodiment of the present invention, the number of darker pixels in the area framed by the defect recognition deep learning inference model is counted by image binarization, and then the number of pixels is multiplied by the spatial resolution , To obtain the size of the area belonging to the defect category. The spatial resolution is the value of the size of a pixel in the image corresponding to the area of the entity. For example, one pixel area corresponds to The size of the solid area is 0.0607mm square. In another embodiment of the present invention, the upper left corner endpoint coordinates (x1, y1) and the lower right corner endpoint coordinates (x2, y2) of the area framed by the defect identification model are used to calculate the new upper left corner based on the linear regression formula The coordinates (x1', y1') and the coordinates of the lower right corner (x2', y2'). Based on (x1', y1') and (x2', y2'), the length l and width w of the selected area of the identification model are calculated. The actual area value of the defect is calculated by multiplying the length, width, spatial resolution and a factor constant get.

The length value is the length of the longest side or diagonal of the area, and its unit is mm.

The depth value is calculated according to the contrast difference between the color of each specific defect category area and the background color of the image, and its value is between 0 and 1. In an embodiment of the present invention, binarization is used to further frame the pixels with darker colors in the area framed by the defect recognition deep learning inference model; in the frame selection area of the inference model, another frame is selected to select a non-defective pixel. Area: Calculate the average of the grayscale values for the darker pixel framed area and the non-defective framed area respectively, and then use the average of the two to calculate the difference between the two; finally, the difference is divided by 255 to obtain the shade value.

Based on the calculated limit values, the computer identification module compares the limit values with an acceptance standard according to a limit logic, and determines whether the set of limit values are consistent with the acceptance standard, so as to screen out that it is consistent with the detection by human judgment Defects of golf balls.

Specifically, the limit logic system is configured to establish comparison conditions for multiple limit dimensions. The limit value is a set of numerical sizes corresponding to the multiple limit dimensions. The limit dimensions include but are not limited to the confidence dimension, area dimension, length dimension, depth dimension and any combination thereof corresponding to each defect category. In order to more accurately screen out true defects that are consistent with human identification, the limit dimensions may not be limited to the above dimensions; any information dimension that can increase the detection rate of true defects can be included in the limit dimension of the limit logic. For example, the limit logic can be configured to additionally include the limit dimension associated with the trademark category or the wind tunnel style category.

Acceptance criteria is a set of numerical ranges or numerical magnitudes corresponding to the comparison conditions of each limit dimension. In an embodiment of the present invention, the acceptance criterion may be a set of numerical ranges, and the limit value that matches this set of ranges is deemed to be a true defect detected correctly, otherwise it is a false defect that is detected incorrectly. For example, the confidence value belonging to a certain defect category needs to be greater than 0.8, and the area value is greater than or equal to 0.7mm ² , the length value is greater than or equal to 0.7mm, and the depth value is less than or equal to 0.01. If the calculated limit value falls within these range intervals, it is deemed to be a correct and true defect detected; if not, it is a false defect that has been detected incorrectly.

In another embodiment of the present invention, the acceptance criterion may be a set of specific values, and the limit value equal to this set of specific values is considered to be a true defect that is detected correctly, otherwise it is a false defect that is detected incorrectly; The confidence value of a defect category is equal to 0.9, the area value is equal to 0.8 mm ² , the length value is equal to 0.7 mm, and the depth value is equal to 0.1. If the limit value is equal to these values, it is deemed that the correct and true defect has been detected; if not, it is the false defect that has been detected incorrectly.

In an embodiment of the present invention, a limit logic may include multiple acceptance criteria. These multiple acceptance standards can correspond to different defect detection accuracy levels. For example, the first acceptance standard corresponds to a good product level with no defects detected, the second acceptance standard corresponds to a defect level with a high detection accuracy, or the third acceptance standard corresponds to a defect level with a low detection accuracy. As shown in Figure 3, the golf ball image information to be tested is input to the defect recognition deep learning inference model 301, which is processed and identified through the input layer (input), convolution layer, pooling layer (Conv.+pool), and fully connected layer (FC). And classification, and then output the defect feature vector in the output layer; the golf image information to be tested is input to the trademark recognition deep learning inference model 303 and the wind tunnel style deep learning inference model 305. After an inference process similar to the defect identification model, the Equivalent inference model identification and classification output individual feature vectors. The computer recognition module calculates a set of limit values based on the feature vectors and image information. This set of limit values classifies the golf ball to be tested to a specific detection accuracy level through various acceptance criteria set by a limit logic. The symbol OK represents a good quality grade, which indicates a good quality golf ball that is judged to be flawless by the limit logic. Symbols NG1 to NG2 represent two defect detection accuracy levels, where NG1 can represent the high accuracy level output by the limit logic judgment inference model is consistent with the human judgment, and NG2 can represent the high accuracy level output by the limit logic judgment inference model The result is a low accuracy level that does not match the human judgment.

In an embodiment of the present invention, the computer recognition module may further include a test unit to determine whether the inference results of the deep learning inference models (such as 301, 303, and 305) are compared with the acceptance criteria of the limit logic. Extra training. The input terminal of the test unit receives the detection result directly output by the deep learning inference model and the detection result determined by the limit logic, so as to calculate the defect detection accuracy rate of the direct inference and the detection accuracy rate through the limit logic. If the defect detection accuracy rate of the limit logic is not significantly higher than the detection accuracy rate of direct inference, it indicates that the limit of the limit logic is insufficient, and the computer identification module adjusts the acceptance standard of the limit logic or adds an additional limit dimension comparison Conditions; or, the user can add more training images to the deep learning model to improve the training results.

Fig. 4 is a schematic diagram of an inference flow using limit logic according to an embodiment of the present invention. The golf defect inference process using limit logic includes: capturing image information of the golf ball to be tested and transferring the captured image information to the computer identification module for processing (step S401); inputting the captured image information to the defect identification deep learning inference model And output the defect feature vector, input the captured image information to the trademark recognition deep learning inference model and output the trademark feature vector, and input the captured image information to the wind tunnel style recognition deep learning inference model and output the wind tunnel style feature vector (step S403); Defect feature vector, trademark feature vector, wind tunnel style feature vector and captured image information, calculate a set of limit values associated with the degree of confidence, area size, length, and depth of the defect (step S405); use the limit logic to compare the group The limit value and an acceptance criterion determine whether the result of the defect inference is consistent with the result of artificial detection (step S407).

The method steps of the inference process described in an embodiment of the present invention can also be implemented as a computer program product for storage in a hard disk of a network server, a memory device, such as app store, google play, window market, or Other similar online distribution platforms for applications.

The present invention has been described in detail above, but what is described above is only one embodiment of the present invention, and should not be used to limit the scope of implementation of the present invention, that is, all the equivalent changes and changes made in accordance with the scope of the patent application of the present invention Modifications should still fall within the scope of the patent of the present invention.

301: Defect identification deep learning inference model

303: Inference model of trademark recognition deep learning

305: Wind tunnel style deep learning inference model

OK: good grade

NG1: High accuracy level for artificial judgment of anastomosis

NG2: Low accuracy level for artificial judgment of mismatch

Claims (8)

A golf computer inspection system, comprising: a microprocessor configured to receive image information of a golf ball to be tested; identifying and classifying defects on the surface of the golf ball to be tested according to a first deep learning inference model and outputting A defect feature vector; identify and classify the trademark of the golf ball to be tested according to a second deep learning inference model and output a trademark feature vector; identify and output the wind tunnel style of the golf ball surface to be tested according to a third deep learning inference model A wind tunnel style feature vector; wherein a golf computer identification module calculates a set of limit values based on the defects, trademarks, wind tunnel style feature vectors and the image parameters of the golf ball to be tested, and the set of limit values corresponds to at least one The value of the limit dimension, and compare the group of limit values with an acceptance standard according to a limit logic to determine whether the group of limit values are consistent with the acceptance standard, and the acceptance standard includes the allowable value corresponding to the at least one limit dimension The range or value size, and the limit logic system are combined to provide a combination of conditions for comparison with the allowable value range or value size, so as to screen out golf defects that are consistent with those detected by human judgment; and a test unit, the test The unit is configured to compare the defect detection results obtained through the limit logic with the inference results directly output by the deep learning inference models, so as to adjust the limit logic or add training data. Such as the golf computer inspection system of claim 1, wherein: the defect feature vector includes information related to the probability and location of a defect. Such as the golf computer detection system of claim 1, in which: The at least one limit dimension includes a degree of confidence belonging to a defect category, area, surface length, surface depth, and any combination thereof. For example, the golf computer inspection system of claim 3, wherein: the set of limit values includes the confidence value, the area value, the length value, the depth value and any combination thereof belonging to a defect category; wherein the confidence value is determined by the first depth The probability value of the defect category generated by the learning inference model, the area value is the area size of the defect category calculated through image pixels, the length value is the length of the longest side or diagonal of the defect category, and the depth value is The size of the contrast difference between the surface image color and the background color of the defect category. Such as the golf computer inspection system of claim 1, wherein: the first, second and third deep learning inference models individually use a Convolutional Neural Network algorithm to mark defects, trademarks and wind tunnels The golf picture of the style is formed by training. Such as the golf computer detection system of claim 1, wherein: the golf ball image parameter to be measured includes the number of darker pixels in the area framed by the first deep learning inference model and a spatial resolution. An automatic optical inspection equipment, comprising: an image sensor for capturing image information on the surface of a golf ball to be tested; a golf ball carrier for bearing the golf ball to be tested and capable of rotating the golf ball to be tested; and Such as the golf computer detection system of any one of claims 1 to 6. For example, the automatic optical inspection equipment of claim 7 further includes: an auxiliary light source device configured to illuminate the golf ball to be tested to provide sufficient brightness so that the image sensor can capture sufficiently clear image information.

Images