TW202147176A - Article detection device、article detection method、storage media - Google Patents

Abstract

The invention provides an article detection device, the device includes: at least one lighting device, at least one camera and at least one processor, the at least one processor is electrically connected with the at least one lighting device and the at least one camera, wherein, the at least one lighting device and the at least one camera are arranged on opposite sides of a vehicle. The processor is used for controlling the at least one lighting device to emit a light beam to the vehicle; the processor is also used for controlling the at least one camera to capture an image of the vehicle; the processor is also used for calling a pre-trained object detection model to detect placement of objects in the vehicle based on the captured image. The invention also provides an article detection method and a storage medium.

Description

Item detection device, item detection method, storage medium

The invention relates to the technical field of computers, and in particular to an article detection device, an article detection method and a storage medium.

At present, when wafers are manually or robotically placed on wafer carriers such as wafer boat boxes or wafer transfer boxes, there is a certain probability that the wafers may be tilted. At present, operators use manual visual inspection to observe whether the wafer is tilted. However, since the appearance of the wafer carrier is not completely transparent, the operator usually needs to carefully observe from six directions of the wafer carrier to determine whether the wafer is tilted. In addition, the wafers are arranged in many combinations in the wafer carrier and when filled with a certain number, the arrangement is very dense. Therefore, it is difficult for the operator to observe whether the wafer is placed at an angle, which leads to the occurrence of misidentification.

In view of the above, it is necessary to propose an article detection device, an article detection method, and a storage medium, which can quickly and accurately detect the placement of articles in a carrier, improve detection efficiency and save labor costs.

A first aspect of the present invention provides an article detection device, the article detection device includes at least one lighting device, at least one camera, and at least one processor, the at least one processor is connected with the at least one lighting device and the at least one A camera is electrically connected; wherein, the at least one lighting device and the at least one camera are arranged on opposite sides of a carrier; the processor is used to control the at least one lighting device to emit light beams to the carrier ; the processor is also used to control the at least one camera to take images of the carrier; the processor is also used to call a pre-trained item detection model based on the captured images for the carrier Check the placement of the items in the tool.

Preferably, the processor is further configured to train the item detection model, including: acquiring a preset number of images corresponding to different placement situations, and labeling the images corresponding to each placement situation with categories, such that Images corresponding to different placement situations carry category labels, and the preset number of images corresponding to different placement situations after category labels are used as training samples; the training samples are randomly divided into first pre- Setting a training set of a proportion and a verification set of a second preset ratio, using the training set to train a deep neural network to obtain the object detection model, and using the verification set to verify the accuracy of the object detection model; and if When the accuracy rate is greater than or equal to the preset accuracy rate, end the training; if the accuracy rate is less than the preset accuracy rate, increase the number of training samples to retrain the deep neural network until the re-acquired The accuracy rate of the item detection model is greater than or equal to the preset accuracy rate.

Preferably, the different placement conditions include correct placement of each item in the carrier, and at least one item in the carrier being placed obliquely.

Preferably, the at least one lighting device is arranged above the carrier, and the at least one camera is arranged below the carrier; or the at least one lighting device and the at least one camera are arranged on the carrier left and right sides of the tool.

Preferably, the articles carried by the carrier are regular articles.

Preferably, the carrier is a wafer carrier, and the items in the carrier are wafers.

A second aspect of the present invention provides an article detection method using the article detection device, the method comprising: controlling the at least one lighting device to emit light beams to the carrier; controlling the at least one camera to take pictures of the carrier and calling the pre-trained object detection model to detect the placement of the objects in the carrier based on the captured images.

Preferably, the method further includes training the item detection model, including: acquiring a preset number of images corresponding to different placement situations, and labeling the images corresponding to each placement situation with categories, so that the images corresponding to different placement situations are The images used in the situation carry category labels, and the preset number of images corresponding to different placement situations after category labels are used as training samples; the training samples are randomly divided into a first preset ratio. A training set and a verification set of a second preset ratio, using the training set to train a deep neural network to obtain the object detection model, and using the verification set to verify the accuracy of the object detection model; and When the accuracy rate is greater than or equal to the preset accuracy rate, end the training; if the accuracy rate is less than the preset accuracy rate, increase the number of training samples to retrain the deep neural network until the item detection model is re-obtained The accuracy of is greater than or equal to the preset accuracy.

Preferably, the different placement conditions include correct placement of each item in the carrier, and at least one item in the carrier being placed obliquely.

A third aspect of the present invention provides a computer storage medium, wherein the computer storage medium stores a plurality of modules, and when the plurality of modules are executed by a processor, the article detection method is implemented.

The article detection device, article detection method, and storage medium described in the embodiments of the present invention can quickly and accurately detect the placement of articles in the carrier, improve detection efficiency and save labor costs.

In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments may be combined with each other under the condition of no conflict.

In the following description, many specific details are set forth in order to facilitate a full understanding of the present invention, and the described embodiments are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terms used herein in the description of the present invention are for the purpose of describing specific embodiments only, and are not intended to limit the present invention.

Referring to FIG. 1 , it is a structural diagram of an article detection device provided by a preferred embodiment of the present invention.

In one embodiment, the article detection apparatus 1 includes, but is not limited to, at least one lighting device 11 , at least one camera 12 , at least one processor 13 , and storage 14 . In this embodiment, the at least one processor 13 is electrically connected to the at least one lighting device 11 , the at least one camera 12 , and the storage 14 .

It should be noted that, what is shown in FIG. 1 is only an illustration of the article detection device 1 according to the embodiment of the present invention, and should not be construed as a limitation of the present invention. The article detection device 1 may comprise more or fewer elements than those shown in FIG. 1 . For example, the article detection device 1 may further include an operating system, a communication module, and the like.

In this embodiment, the article detection device 1 is used to detect the placement of one or more articles 21 carried by the carrier 2 . For example, it is detected whether the one or more items 21 are placed obliquely within the carrier 2 .

In one embodiment, the one or more objects 21 may be regular objects, for example, wafers with the same shape such as a circle, or other products such as motherboards with the same shape such as a square. The one or more items 21 are regularly arranged on the carrier 2 . In this embodiment, the carrier 2 is used to carry the one or more items 21 . For example, the carrier 2 is a wafer box or a wafer transfer box.

It should be noted that a wafer refers to a silicon wafer used in the fabrication of silicon semiconductor integrated circuits, and is called a wafer because of its circular shape.

In this embodiment, the article detection device 1 may be a computer device, and the at least one lighting device 11 and the at least one camera 12 may be externally placed on the computer device. The at least one processor 13 and the storage 14 may be built in the computer device.

In one embodiment, the at least one lighting device 11 may be an LED (Light-emitting diode) device. The at least one camera 12 may be a high-definition camera.

In one embodiment, the at least one lighting device 11 and the at least one camera 12 are arranged on opposite sides of the carrier 2 . The at least one lighting device 11 is used to emit light beams to the carrier 2 . The at least one camera 12 is used to capture images of the carrier 2 .

In order to clearly illustrate the present invention, the present embodiment takes the carrier 2 as a wafer box and the one or more items 21 as wafers as an example.

Please also refer to FIG. 2 , in one embodiment, the at least one lighting device 11 is arranged above the carrier 2 , and the at least one camera 12 is arranged below the carrier 2 . In other embodiments, the at least one lighting device 11 and the at least one camera 12 may also be disposed on the left and right sides of the carrier 2 .

In some embodiments, the storage 14 is used to store code and various data, for example, to store the item detection system 140 installed in the item detection device 1 , and to achieve high-speed, high-speed, Access to programs or data is done automatically. The storage 14 includes a Read-Only Memory (ROM), a Programmable Read-Only Memory (PROM), and an Erasable Programmable Read-Only Memory (Erasable Programmable Read). -Only Memory, EPROM), One-time Programmable Read-Only Memory (OTPROM), Electronically-Erasable Programmable Read-Only Memory (EEPROM) , Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other computer-readable storage medium that can be used to carry or store data.

In some embodiments, the at least one processor 13 may be composed of an integrated circuit, for example, may be composed of a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same function or different functions , including one or more central processing units (Central Processing Units, CPUs), microprocessors, digital word processing chips, graphics processors and combinations of various control chips, etc. The at least one processor 13 is the control core (Control Unit) of the article detection device 1, uses various interfaces and lines to connect various components of the entire article detection device 1, and is stored in the storage 14 by running or executing. program or module, and call the data stored in the storage 14 to perform various functions of the item detection device 1 and process data, for example, by executing the item detection system 140 to detect the items in the carrier 2 The function of detecting the placement situation of 21 (such as whether it is placed obliquely).

In one embodiment, the processor 13 is configured to control the at least one lighting device 11 to emit light beams to the carrier 2 . The processor 13 also controls the at least one camera 12 to capture images of the carrier 2 while controlling the at least one lighting device 11 to emit light beams to the carrier 2 . The processor 13 is further configured to call a pre-trained item detection model to detect the placement situation of the one or more items 21 in the carrier 2 based on the captured images. For example, it is detected whether there is at least one item 21 in the carrier 2 placed obliquely.

In one embodiment, the processor 13 may pre-train the item detection model, and store the pre-trained item detection model in the storage 14, so that when the items in the carrier 2 need to be checked When the placement situation of 21 is detected, the processor 13 can call the item detection model from the storage 14, and based on the image captured by the camera 12 of the carrier 2, realize the detection of all the items. The placement situation of the items 21 in the carrier 2 is detected.

In one embodiment, the method for the processor 13 to train the item detection model includes (a1)-(a3): (a1) Obtain a preset number of images corresponding to different placement situations, and label the images corresponding to each placement situation, so that images corresponding to different placement situations carry category labels, which will be used as The preset number of images corresponding to different placement situations after category annotations are used as training samples.

In this embodiment, the different placement situations include a first placement situation and a second placement situation. The first placement situation is the correct placement of each item 21 in the carrier 2 . The second placement situation is that at least one item 21 in the carrier 2 is placed obliquely.

It should be noted that the correct placement of the article 21 is relative to the oblique placement, that is, the article 21 is not placed obliquely, or it can be defined as placing the article 21 according to the placement requirements/placement standards for the article 21 .

Specifically, 500 images corresponding to the first placement situation may be selected, and the 500 images are marked as "1" respectively, that is, "1" is used as a label. Similarly, 500 images corresponding to the second placement situation are selected, and the 500 images are marked as "2" respectively, that is, "2" is used as a label. The 1000 images with class annotations are used as training samples.

In this embodiment, the preset number of images may be captured by the at least one camera 12 or other cameras while the at least one lighting device 11 emits light beams to the carrier 2 from above the carrier 2 . The device is obtained by photographing and aiming at the carrier 2 from below the carrier 2 .

Of course, in other embodiments, when the at least one lighting device 11 is arranged on the left side of the carrier 2 and the at least one camera 12 is arranged on the right side of the carrier 2, the preset number of images The image can be that while the at least one lighting device 11 emits light beams from the left side of the carrier 2 to the carrier 2 , the at least one camera 12 or other photographing device looks at the camera from the right side of the carrier 2 . The vehicle 2 was photographed and obtained. When the at least one lighting device 11 is arranged on the right side of the carrier 2 and the at least one camera 12 is arranged on the left side of the carrier 2, the preset number of images may be the at least one image While the lighting equipment 11 emits light beams from the right side of the carrier 2 to the carrier 2 , the at least one camera 12 or other photographing device captures the carrier 2 from the left side of the carrier 2 .

(a2) Randomly divide the training sample into a training set with a first preset ratio and a verification set with a second preset ratio, use the training set to train a deep neural network to obtain the item detection model, and use the verification set set to verify the accuracy of the item detection model. For example, images corresponding to different placement situations may be distributed to different folders according to the marked categories. For example, the images corresponding to the first placement situation are distributed to the first folder, and the images corresponding to the second placement situation are distributed to the second folder. Then, a first preset ratio (for example, 70%) of images is extracted from the first folder and the second file respectively as a training set, and the training set is used to train a deep neural network to obtain the object detection model ; and respectively take the remaining images of a second preset ratio (eg, 30%) from the first folder and the second file as a verification set, and use the item detection model to analyze the verification set Perform detection, and calculate the accuracy of the item detection model based on the detection result of the item detection model.

In one embodiment, the deep neural network includes an input layer, a convolutional layer, a pooling layer, and a fully connected layer. The input layer inputs an image, and the convolutional layer extracts image features and outputs feature vectors. The pooling layer compresses the feature vector to extract the main features. The fully connected layer connects all features and sends the output value to a classifier (such as a softmax classifier).

(a3) If the accuracy rate is greater than or equal to the preset accuracy rate, end the training; if the accuracy rate is less than the preset accuracy rate, increase the number of training samples to retrain the deep neural network until it restarts. The obtained accuracy of the article detection model is greater than or equal to the preset accuracy.

Referring to FIG. 3 , in this embodiment, the article detection system 140 may include a plurality of functional modules composed of program code segments. The code of each program segment in the article detection system 140 can be stored in the storage 14 of the article detection device 1 and executed by the at least one processor 13 to realize the detection of the inner part of the carrier 2 . The placement of the one or more items 21 is detected.

In this embodiment, the article detection system 140 can be divided into a plurality of functional modules according to the functions it performs. The function module may include: an execution module 1401 and a detection module 1402 . The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor (eg, the processor 13 ) and can perform fixed functions, which are stored in the storage 14 . Each of the above modules includes a number of instructions to cause the at least one processor 13 to execute to implement the detection of the placement of the one or more items 21 in the carrier 2 according to various embodiments of the present invention. Features. The function of each module will be described in detail with reference to FIG. 4 .

Referring to FIG. 4 , a flowchart of an article detection method provided by a preferred embodiment of the present invention is shown. According to different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

Step S1 , the execution module 1401 controls the at least one lighting device 11 to emit light beams to the carrier 2 . The execution module 1401 also controls the at least one camera 12 to capture images of the carrier 2 while the at least one lighting device 11 emits light beams to the carrier 2 .

Step S2, the detection module 1402 invokes the pre-trained item detection model to detect the placement of the item 21 in the carrier 2 based on the captured image.

Specifically, the detection module 1402 inputs the captured image into the item detection model to obtain the placement status of the items 21 in the carrier 2 .

In one embodiment, the method for training the item detection model by the detection module 1402 includes (a1)-(a3): (a1) Obtain a preset number of images corresponding to different placement situations, and label the images corresponding to each placement situation, so that images corresponding to different placement situations carry category labels, which will be used as The preset number of images corresponding to different placement situations after category annotations are used as training samples.

In this embodiment, the different placement situations include a first placement situation and a second placement situation. The first placement situation is the correct placement of each item 21 in the carrier 2 . The second placement situation is that at least one item 21 in the carrier 2 is placed obliquely.

It should be noted that the correct placement of the article 21 is relative to the oblique placement, that is, the article 21 is not placed obliquely, or it can be defined as placing the article 21 according to the placement requirements/placement standards for the article 21 .

Specifically, 500 images corresponding to the first placement situation may be selected, and the 500 images are marked as "1" respectively, that is, "1" is used as a label. Similarly, 500 images corresponding to the second placement situation are selected, and the 500 images are marked as "2" respectively, that is, "2" is used as a label. The 1000 images with class annotations are used as training samples.

In this embodiment, the preset number of images may be captured by the at least one camera 12 or other cameras while the at least one lighting device 11 emits light beams to the carrier 2 from above the carrier 2 . The device is obtained by photographing and aiming at the carrier 2 from below the carrier 2 .

Of course, in other embodiments, when the at least one lighting device 11 is arranged on the left side of the carrier 2 and the at least one camera 12 is arranged on the right side of the carrier 2, the preset number of images The image can be that while the at least one lighting device 11 emits light beams from the left side of the carrier 2 to the carrier 2 , the at least one camera 12 or other photographing device looks at the camera from the right side of the carrier 2 . The vehicle 2 was photographed and obtained. When the at least one lighting device 11 is arranged on the right side of the carrier 2 and the at least one camera 12 is arranged on the left side of the carrier 2, the preset number of images may be the at least one image While the lighting equipment 11 emits light beams from the right side of the carrier 2 to the carrier 2 , the at least one camera 12 or other photographing device captures the carrier 2 from the left side of the carrier 2 .

(a2) Randomly divide the training sample into a training set with a first preset ratio and a verification set with a second preset ratio, use the training set to train a deep neural network to obtain the item detection model, and use the verification set set to verify the accuracy of the item detection model.

For example, images corresponding to different placement situations may be distributed to different folders according to the marked categories. For example, the images corresponding to the first placement situation are distributed to the first folder, and the images corresponding to the second placement situation are distributed to the second folder. Then, a first preset ratio (for example, 70%) of images is extracted from the first folder and the second file respectively as a training set, and the training set is used to train a deep neural network to obtain the object detection model ; and respectively take the remaining images of a second preset ratio (eg, 30%) from the first folder and the second file as a verification set, and use the item detection model to analyze the verification set Perform detection, and calculate the accuracy of the item detection model based on the detection result of the item detection model.

In one embodiment, the deep neural network includes an input layer, a convolutional layer, a pooling layer, and a fully connected layer. The input layer inputs an image, and the convolutional layer extracts image features and outputs feature vectors. The pooling layer compresses the feature vector to extract the main features. The fully connected layer connects all features and sends the output value to a classifier (such as a softmax classifier).

(a3) If the accuracy rate is greater than or equal to the preset accuracy rate, end the training; if the accuracy rate is less than the preset accuracy rate, increase the number of training samples to retrain the deep neural network until it restarts. The obtained accuracy of the article detection model is greater than or equal to the preset accuracy.

It should be noted that, in the several embodiments provided by the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and other division methods may be used in actual implementation.

The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they can be located in one place or distributed to multiple networks. on the unit. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, or can be implemented in the form of hardware plus software function modules.

It will be apparent to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, but that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments are to be regarded in all respects as illustrative and not restrictive, and the scope of the present invention is defined by the appended claims rather than the foregoing description, and is therefore intended to fall within the scope of the claims. All changes within the meaning and range of the equivalents of , are included in the present invention. Any reference sign in a claim should not be construed as limiting the claim to which it relates. Furthermore, it is clear that the word "comprising" does not exclude other units or, and the singular does not exclude the plural. A plurality of units or means stated in the device claim may also be implemented by one unit or means by software or hardware. The terms first, second, etc. are used to denote names and do not denote any particular order.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent substitutions can be made without departing from the spirit and scope of the technical solutions of the present invention.

1: Item detection device 11: Lighting equipment 12: Camera 13: Processor 14: Storage 140: Item Detection System 1401: Execution module 1402: Detection module 2: Vehicle 21: Items

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to the provided drawings without creative work.

FIG. 1 is a block diagram of an article detection device provided by a preferred embodiment of the present invention.

Figure 2 illustrates the light emitting device and camera setup.

FIG. 3 is a functional module diagram of an article detection system provided by a preferred embodiment of the present invention.

FIG. 4 is a flow chart of an article detection method provided by a preferred embodiment of the present invention.

The following specific embodiments will further illustrate the present invention in conjunction with the above drawings.

Claims (10)

An object detection device improved in that the object detection device includes at least one lighting device, at least one camera, and at least one processor, the at least one processor being electrically connected to the at least one lighting device and the at least one camera connect; Wherein, the at least one lighting device and the at least one camera are arranged on opposite sides of a carrier; the processor for controlling the at least one lighting device to emit light beams to the vehicle; the processor is further configured to control the at least one camera to capture images of the vehicle; The processor is further configured to call a pre-trained item detection model to detect the placement of items in the carrier based on the captured images. The article detection device according to claim 1, wherein the processor is further configured to train the article detection model, including: Obtain a preset number of images corresponding to different placement situations, and label the images corresponding to each placement situation, so that the images corresponding to different placement situations carry the class label, and the class label will be made The following preset number of images corresponding to different placement situations are used as training samples; The training samples are randomly divided into a training set of a first preset ratio and a verification set of a second preset ratio, and the training set is used to train a deep neural network to obtain the item detection model, and the verification set is used to verify all the samples. the accuracy of the said article detection model; and If the accuracy rate is greater than or equal to the preset accuracy rate, end the training; if the accuracy rate is less than the preset accuracy rate, increase the number of training samples to retrain the deep neural network until the re-acquired The accuracy of the item detection model is greater than or equal to the preset accuracy. The article detection device according to claim 2, wherein the different placement conditions include that each article in the carrier is placed correctly, and at least one article in the carrier is placed obliquely. The article detection device according to claim 1, wherein the at least one lighting device is arranged above the carrier, and the at least one camera is arranged below the carrier; or the at least one lighting device and The at least one camera is arranged on the left and right sides of the carrier. The article detection device according to claim 1, wherein the article carried by the carrier is a regular article. The article detection device according to claim 5, wherein the carrier is a wafer carrier, and the article in the carrier is a wafer. An article detection method realized by using the article detection device of claim 1, the improvement is that the method includes: controlling the at least one lighting device to emit a light beam to the vehicle; controlling the at least one camera to capture images of the vehicle; and The pre-trained item detection model is invoked to detect the placement of items in the carrier based on the captured images. The article detection method according to claim 7, wherein the method further includes training the article detection model, including: Obtain a preset number of images corresponding to different placement situations, and label the images corresponding to each placement situation, so that the images corresponding to different placement situations carry the class label, and the class label will be made The following preset number of images corresponding to different placement situations are used as training samples; The training samples are randomly divided into a training set of a first preset ratio and a verification set of a second preset ratio, and the training set is used to train a deep neural network to obtain the item detection model, and the verification set is used to verify all the samples. the accuracy of the said article detection model; and If the accuracy rate is greater than or equal to the preset accuracy rate, end the training; if the accuracy rate is less than the preset accuracy rate, increase the number of training samples to retrain the deep neural network until the re-acquired The accuracy of the item detection model is greater than or equal to the preset accuracy. The article detection method according to claim 8, wherein the different placement conditions include that each article in the carrier is placed correctly, and at least one article in the carrier is placed obliquely. A storage medium, the storage medium stores a plurality of modules, and when the plurality of modules are executed by a processor, the article detection method according to any one of claim 7 to claim 9 is implemented.

Images