CN117565062B - Automatic control method and system for wafer carrying manipulator based on machine learning - Google Patents
Automatic control method and system for wafer carrying manipulator based on machine learning Download PDFInfo
- Publication number
- CN117565062B CN117565062B CN202410060511.6A CN202410060511A CN117565062B CN 117565062 B CN117565062 B CN 117565062B CN 202410060511 A CN202410060511 A CN 202410060511A CN 117565062 B CN117565062 B CN 117565062B
- Authority
- CN
- China
- Prior art keywords
- wafer
- image
- mass distribution
- uniform mass
- negative pressure
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 104
- 238000010801 machine learning Methods 0.000 title claims abstract description 34
- 235000012431 wafers Nutrition 0.000 claims description 404
- 238000009826 distribution Methods 0.000 claims description 107
- 230000006978 adaptation Effects 0.000 claims description 39
- 238000001179 sorption measurement Methods 0.000 claims description 36
- 230000008569 process Effects 0.000 claims description 26
- 238000012549 training Methods 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 16
- 238000007781 pre-processing Methods 0.000 claims description 16
- 238000012360 testing method Methods 0.000 claims description 12
- 238000004458 analytical method Methods 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 10
- 238000006073 displacement reaction Methods 0.000 claims description 9
- 238000003708 edge detection Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 8
- 238000003860 storage Methods 0.000 claims description 8
- 238000002372 labelling Methods 0.000 claims description 7
- 238000012935 Averaging Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 5
- 238000009499 grossing Methods 0.000 claims description 4
- 230000008859 change Effects 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 239000004065 semiconductor Substances 0.000 description 3
- 238000000926 separation method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000005484 gravity Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Container, Conveyance, Adherence, Positioning, Of Wafer (AREA)
Abstract
The invention belongs to the technical field of carrying manipulators, and discloses an automatic control method and an automatic control system for a wafer carrying manipulator based on machine learning.
Description
Technical Field
The invention relates to the technical field of carrying manipulators, in particular to an automatic control method and an automatic control system of a wafer carrying manipulator based on machine learning.
Background
The prior art comprises the following steps: the Chinese patent with the publication number of CN116277037A discloses a wafer handling manipulator control system and a method, which relate to the technical field of intelligent control of wafer handling manipulators, and a machine learning model for predicting negative pressure of a wafer handling manipulator suction cup is trained based on a historical training data set by collecting the historical training data set of the wafer handling manipulator in advance, the wafer handling manipulator acquires wafer images and production data in real time before handling the wafer, and acquires the geometric center position and the gravity center position of the wafer based on the wafer images in a control background of the wafer handling manipulator, and a negative pressure change curve is generated in advance for the wafer handling manipulator based on the production data, the machine learning model, the geometric center position and the gravity center position of the wafer; the phenomenon that the wafer falls off due to too small negative pressure or is extruded and deformed due to too large negative pressure caused by speed change in the moving process of the wafer carrying manipulator is avoided.
The following problems still exist in the prior art:
1. for the wafer with non-uniform mass distribution, the mass distribution condition of the wafer to be conveyed is not judged, so that the conveying manipulator cannot select a proper adsorption area, and the conveying stability and efficiency are affected;
2. the prior art does not consider the influence of the difference of wafer types (such as uniform mass distribution wafer and non-uniform mass distribution wafer) on the negative pressure curve, so that the risk in the carrying process cannot be effectively reduced;
in view of the above, the present invention provides an automatic control method and system for a wafer handling robot based on machine learning to solve the above-mentioned problems.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention provides the following technical scheme for achieving the purposes:
the automatic control method of the wafer carrying manipulator based on machine learning comprises the following steps:
collecting a wafer image of a wafer to be conveyed;
preprocessing the wafer image;
extracting the outline of the preprocessed wafer image;
adding the outline of the extracted wafer image into the wafer image after pretreatment, and marking the obtained wafer image as a boundary wafer image;
calculating absolute values of sum and difference values of pixel values of corresponding pixel blocks on two sides of a boundary passing through the center of a boundary wafer image circle, and marking the absolute values as difference pixel values;
determining a wafer type according to the difference pixel value, wherein the wafer type comprises a uniform mass distribution wafer or a non-uniform mass distribution wafer;
selecting a manipulator adsorption area according to the uniform mass distribution wafer and the non-uniform mass distribution wafer;
and inputting the difference pixel values and the wafer types into a pre-constructed negative pressure curve adaptation model to obtain negative pressure curves corresponding to the wafer with uniform mass distribution and the wafer with non-uniform mass distribution in the carrying process.
Further, the method for preprocessing the wafer image comprises the following steps:
step S101, converting a wafer image into a wafer gray scale image by using an averaging method;
step S102, smoothing the image noise of the wafer by using a median filtering method.
Further, the method for extracting the outline of the preprocessed wafer image comprises the following steps of;
performing edge detection on the preprocessed wafer image;
performing binarization processing on the wafer image after edge detection, setting the point with the pixel value higher than the preset pixel threshold value as white, and setting the point with the pixel value lower than the preset pixel threshold value as black;
and (3) performing contour extraction by adopting a method of hollowing out internal points, if the target pixel in the wafer image is black and all 8 adjacent pixel points are black, deleting the target pixel point, namely setting the target pixel to be white, and traversing each pixel with the boundary removed to obtain an image only containing edge information, namely the contour of the wafer image.
Further, the method for determining the dividing line comprises the following steps:
and drawing a boundary line passing through the center of the boundary wafer image by the direction that the tail end of the manipulator enters the wafer to be adsorbed.
Further, the method for determining whether the wafer is a uniform mass distribution wafer or a non-uniform mass distribution wafer by the difference pixel values comprises the following steps:
comparing the difference pixel value with a preset pixel error threshold value for analysis, and judging the wafer as a wafer with uniform mass distribution if the difference pixel value is smaller than the preset pixel error threshold value; if the difference pixel value is greater than or equal to the preset pixel error threshold, the wafer is determined to be a non-uniform mass distribution wafer.
Further, the method for selecting the robot adsorption area according to the uniform mass distribution wafer and the non-uniform mass distribution wafer comprises the following steps:
if the wafer is uniformly mass distributed, selecting a dividing line as an adsorption area; if the wafer is a non-uniform mass distribution wafer, the demarcation line is shifted to the side with the larger sum of pixel values by a unit displacement amount, and the demarcation line is selected as an adsorption area after shifting.
Further, the training method of the negative pressure curve adaptation model comprises the following steps:
collecting a sample data set, wherein the sample data set comprises wafer characteristic data and a negative pressure value set in a carrying process corresponding to the wafer characteristic data, and the wafer characteristic data comprises a wafer type and a difference pixel value corresponding to the wafer type;
before training the negative pressure curve adaptation model, respectively carrying out numerical labeling on a wafer with uniform mass distribution, a wafer with non-uniform mass distribution and a negative pressure value set in the carrying process, taking wafer characteristic data as input of the negative pressure curve adaptation model, and taking the numerical labeling of the negative pressure value set as output of the negative pressure curve adaptation model; dividing sample data into a training set and a testing set, training the negative pressure curve adaptation model by using the training set to obtain an initial negative pressure curve adaptation model, and testing the initial negative pressure curve adaptation model by using the testing set to obtain a final negative pressure curve adaptation model meeting preset accuracy.
Further, the wafer image is obtained by the high-definition camera arranged at the execution end of the manipulator, and the mounting position of the high-definition camera cannot cause interference on the adsorption wafer at the execution end of the manipulator.
The automatic control system of the wafer carrying manipulator based on machine learning implements the automatic control method of the wafer carrying manipulator based on machine learning, and comprises the following steps:
the wafer image acquisition module is used for acquiring wafer images of wafers to be conveyed;
the wafer image preprocessing module is used for preprocessing the wafer image;
the contour extraction module is used for extracting the contour of the preprocessed wafer image;
the contour adding module is used for adding the contour of the extracted wafer image into the wafer image after pretreatment, and marking the obtained wafer image as a boundary wafer image;
the pixel value calculating module calculates the absolute value of the sum and difference value of pixel values of the corresponding pixel blocks at two sides of the boundary passing through the center of the boundary wafer image, and marks the absolute value as a difference pixel value;
the wafer type analysis module is used for judging the wafer type according to the difference pixel value, wherein the wafer type comprises a uniform mass distribution wafer or a non-uniform mass distribution wafer;
the adsorption area selection module is used for selecting a manipulator adsorption area according to the uniform mass distribution wafer and the non-uniform mass distribution wafer;
and the negative pressure curve self-adapting module is used for inputting the difference pixel value and the wafer type into a pre-constructed negative pressure curve adapting model to obtain negative pressure curves corresponding to the wafer with uniform mass distribution and the wafer with non-uniform mass distribution in the carrying process.
An electronic device comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the automatic control method of the wafer handling manipulator based on machine learning when executing the computer program.
A computer readable storage medium having a computer program stored thereon, the computer program when executed by a processor implementing the machine learning based wafer handling robot automatic control method.
The automatic control method and the system for the wafer carrying manipulator based on machine learning have the technical effects and advantages that:
according to the method, through analysis of the wafer types, whether the wafer to be conveyed is a wafer with uniform mass distribution or a wafer with non-uniform mass distribution is judged, if the wafer with uniform mass distribution is the wafer with uniform mass distribution, a dividing line passing through the center of a boundary wafer image is selected as an adsorption area by the conveying manipulator, if the wafer with non-uniform mass distribution is the wafer with uniform mass distribution, the dividing line is translated to one side with a large sum of pixel values by a unit displacement amount, and the dividing line is selected as the adsorption area by the conveying manipulator after translation, so that factors of unstable conveying caused by deviation of the mass center of the wafer from the geometric center are reduced, and further the wafer is conveyed more stably by the manipulator, and conveying efficiency is effectively improved;
secondly, the absolute value of the sum and difference value of the pixel values of all the pixel blocks at two sides of the dividing line is marked as a difference pixel value, the difference pixel value and the wafer type are input into a pre-built negative pressure curve adaptation model, and a negative pressure curve corresponding to the wafer type conveying process is obtained, so that risks in the conveying process, such as shaking, sliding or even falling of the wafer from a mechanical hand, are reduced.
Drawings
FIG. 1 is a schematic diagram of an automatic control system of a wafer handling robot based on machine learning according to embodiment 1 of the present invention;
FIG. 2 is a flowchart illustrating a method for hollowing out internal points in the contour extraction module according to embodiment 1 of the present invention;
FIG. 3 is a schematic view showing the boundary of a non-uniform mass distribution wafer in the adsorption zone selection module according to embodiment 1 of the present invention;
FIG. 4 is a schematic diagram of an automatic control system of a wafer handling robot based on machine learning according to embodiment 2 of the present invention;
FIG. 5 is a flow chart of an automatic control method of a wafer handling robot based on machine learning according to embodiment 3 of the present invention;
fig. 6 is a schematic diagram of an electronic device in embodiment 3 of the present invention;
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1
Referring to fig. 1, the automatic wafer handling manipulator control system based on machine learning according to the present embodiment includes a wafer image acquisition module, a wafer image preprocessing module, a contour extraction module, a contour adding module, a pixel value calculation module, a wafer type analysis module, an adsorption area selection module, and a negative pressure curve self-adaptation module, where each module is connected by a wired and/or wireless manner to realize data transmission;
in a semiconductor manufacturing factory, wafer handling robots are used to automatically handle wafers, the robots are provided with negative pressure suction devices, and different types of wafers are adapted by adjusting negative pressure change curves;
different types of wafers include uniform mass distribution wafers and non-uniform mass distribution wafers;
the wafer adsorption position is generally in the geometric center position for uniform mass distribution; for non-uniform mass distribution wafers, the mass of such wafers is unevenly distributed on the surface thereof, resulting in a center of mass that deviates from the geometric center, which is caused by material differences, errors in the production process, or other factors, and for example, in some applications, other components or structures are attached to the wafer, such as chips, sensors, etc., the presence of these additional components can change the center of mass position of the wafer;
for a uniformly mass distributed wafer, the geometric center position of the wafer is absorbed by the carrying manipulator, and for a non-uniformly mass distributed wafer, because the mass center of the wafer deviates from the geometric center, if the geometric center position of the absorbed wafer is still adopted, the wafer can not be stably carried by the manipulator, which can lead to the risk of shaking, sliding or even falling from the manipulator during carrying;
the negative pressure change curves of the manipulator in the process of carrying the wafers with uniform mass distribution and the wafers with non-uniform mass distribution are different, and the wafers with different types (namely the wafers with uniform mass distribution and the wafers with non-uniform mass distribution) are required to be carried by using the corresponding negative pressure change curves, so that the manipulator can be quickly and stably adapted to carrying scenes and tasks of the wafers with different types;
the wafer image acquisition module is used for acquiring a wafer image of a wafer to be carried, and is arranged at the execution tail end of the manipulator, when the execution tail end of the manipulator moves above or below the wafer to be carried, the wafer image acquisition module can acquire the wafer image of the wafer to be carried, and the installation position of the wafer image acquisition module cannot cause interference on the wafer adsorbed by the execution tail end of the manipulator; the wafer image acquisition module is a high-definition camera;
the wafer image preprocessing module is used for preprocessing the wafer image;
the specific method for preprocessing the wafer image comprises the following steps:
step S101, converting a wafer image into a wafer gray level image by using an averaging method, reducing the data processing amount and improving the processing speed, and facilitating the contour extraction of the wafer image;
the specific method using the averaging method is as follows: the method comprises the steps of carrying out average processing on the values of R, G, B channels of each pixel of a wafer image, enabling the values of R, G, B channels of each pixel to be equal, enabling the values of R, G, B channels of each pixel to be equal, namely, using a numerical value to represent the gray value of the pixel, and giving the calculated gray value of each pixel to each corresponding pixel in the wafer image, so that a wafer gray image is obtained;
step S102, performing smoothing treatment on wafer image noise generated by dust and particles in a production environment by using a median filtering method to reduce the influence of the noise, wherein common noise types are spiced salt noise and Gaussian noise;
the specific method of the median filtering method is as follows: the image is smoothed and noise reduced by using a sliding window, i.e. a square or rectangular window of odd size, such as a 3x3, 5x5 square or rectangular window, by ordering the pixel values in the neighborhood around each pixel and taking the median value instead of the value of the center point.
The contour extraction module is used for extracting the contour of the preprocessed wafer image, and the specific method for extracting the contour of the preprocessed wafer image comprises the following steps:
and performing edge detection on the preprocessed wafer image to detect the outline in the wafer image, wherein important features such as object boundaries, textures and the like can be extracted from the wafer image through the edge detection.
Performing binarization processing on the wafer image after edge detection, setting a point with a pixel value higher than a preset pixel threshold value as 255 (white), and setting a point lower than the threshold value as 0 (black), wherein the preset pixel threshold value is set according to actual conditions;
extracting the outline by adopting a method of hollowing out the internal points, specifically, if the target pixel in the wafer image is black and all 8 adjacent pixel points are black, setting the target pixel as white, traversing each pixel with the boundary removed to obtain an image only containing edge information, namely, the outline of the wafer image; exemplary, please refer to fig. 2, which illustrates a target pixel H 1 8 pixel points are arranged around, and the pixel points are set to be white; target pixel point H in the figure 2 The periphery is not provided with 8 pixel points, so that the periphery is not set to be white;
the contour adding module is used for adding the contour of the extracted wafer image into the wafer image after pretreatment, and marking the obtained wafer image as a boundary wafer image;
the extracted outline of the wafer image is added into the preprocessed wafer image and can be realized through an OpenCV library;
a pixel value calculation module for calculating the absolute value of the sum and difference of pixel values of the corresponding pixel blocks at two sides of the boundary passing through the center of the boundary wafer image, and marking the sum of pixel values of the pixel blocks at one side of the boundary with Z 1 Marking the sum Z of pixel values of the pixel blocks at the other side of the dividing line 2 ，|Z|=Z 1 -Z 2 Z is the absolute value of the sum and difference of the pixel values of the corresponding pixel blocks at two sides;
drawing a boundary line passing through the center of the boundary wafer image by the direction that the tail end of the manipulator enters the wafer to be adsorbed; calculating the sum of pixel values of all pixel blocks at two sides of the dividing line, and marking the absolute value of the sum difference value of the pixel values of all pixel blocks at two sides of the dividing line as a difference pixel value;
the wafer type analysis module judges whether the wafer is a uniform mass distribution wafer or a non-uniform mass distribution wafer according to the difference pixel values;
comparing the difference pixel value with a preset pixel error threshold value, and judging whether the wafer is a uniform mass distribution wafer or a non-uniform mass distribution wafer or not;
since the gray value of each pixel in the wafer image after pretreatment is given to each corresponding pixel in the wafer image according to the wafer image pretreatment module, the pixel value of each pixel in the wafer image is the gray value, if the gray value of the pixel point F is higher, the pixel point F corresponds to higher density, and if the gray value of the pixel point E is lower, the pixel point E corresponds to lower density, so that the mass distribution condition on the wafer can be determined by analyzing the size of the pixel value;
the method for determining whether the wafer is a uniform mass distribution wafer or a non-uniform mass distribution wafer comprises the following steps:
comparing the difference pixel value with a preset pixel error threshold value, judging that the wafer is a uniform mass distribution wafer if the difference pixel value is smaller than the preset pixel error threshold value, and judging that the wafer is a non-uniform mass distribution wafer if the difference pixel value is larger than or equal to the preset pixel error threshold value, wherein the preset pixel error threshold value is set automatically according to actual conditions;
the adsorption area selection module is used for selecting a manipulator adsorption area according to the wafer with uniform mass distribution and the wafer with non-uniform mass distribution;
the specific method for selecting the adsorption area of the manipulator according to the uniform mass distribution wafer and the non-uniform mass distribution wafer comprises the following steps:
if the wafer is uniformly mass distributed, selecting a dividing line as an adsorption area; if the wafer is a non-uniform mass distribution wafer, shifting the demarcation line to the side with the larger sum of pixel values by a unit displacement amount, selecting the shift demarcation line as an adsorption area, setting the unit displacement amount by a technician according to actual conditions, wherein the unit displacement amount is not particularly limited, as shown in fig. 3, a virtual straight line is the demarcation line, and the straight line is the demarcation line shifting by the unit displacement amount, namely, the distribution mass of the wafer at the shift side is heavier;
the negative pressure curve self-adaptation module is used for inputting the difference pixel value and the wafer type into a pre-constructed negative pressure curve adaptation model to obtain a negative pressure curve corresponding to the wafer type conveying process; the wafer type is a uniform mass distribution wafer or a non-uniform mass distribution wafer;
the training method of the negative pressure curve adaptation model comprises the following steps:
collecting a sample data set, wherein the sample data set comprises wafer characteristic data and a negative pressure value set in the process of carrying the wafer characteristic data correspondingly; the wafer characteristic data comprises a wafer type and a difference pixel value corresponding to the wafer type;
in the process of collecting the sample data set, a plurality of groups of wafer characteristic data can be matched with a negative pressure value set in a group of carrying processes, namely, a plurality of groups of wafer characteristic data use the negative pressure value set in the group of carrying processes, so that the conditions of shaking, sliding or even falling from a manipulator cannot occur;
before training the negative pressure curve adaptation model, respectively labeling the negative pressure value sets in the process of carrying the wafer with uniform mass distribution and the wafer with non-uniform mass distribution; the wafer characteristic data is used as the input of the negative pressure curve adaptation model, and the numerical label of the negative pressure value set is used as the output of the negative pressure curve adaptation model; dividing sample data into a training set and a testing set, training the negative pressure curve adaptation model by using the training set to obtain an initial negative pressure curve adaptation model, and testing the initial negative pressure curve adaptation model by using the testing set to obtain a final negative pressure curve adaptation model meeting preset accuracy; the negative pressure curve adaptation model is a deep neural network model or other suitable models, and is not particularly limited herein;
according to the method, through analysis of the wafer types, whether the wafer to be conveyed is a wafer with uniform mass distribution or a wafer with non-uniform mass distribution is judged, if the wafer with uniform mass distribution is the wafer with uniform mass distribution, a dividing line passing through the center of a boundary wafer image is selected as an adsorption area by the conveying manipulator, if the wafer with non-uniform mass distribution is the wafer with uniform mass distribution, the dividing line is translated to one side with a large sum of pixel values by a unit displacement amount, and the dividing line is selected as the adsorption area by the conveying manipulator after translation, so that factors of unstable conveying caused by deviation of the mass center of the wafer from the geometric center are reduced, and further the wafer is conveyed more stably by the manipulator, and conveying efficiency is effectively improved;
secondly, the absolute value of the sum and difference value of the pixel values of all the pixel blocks at two sides of the dividing line is marked as a difference pixel value, the difference pixel value and the wafer type are input into a pre-built negative pressure curve adaptation model, and a negative pressure curve corresponding to the wafer type conveying process is obtained, so that risks in the conveying process, such as shaking, sliding or even falling of the wafer from a mechanical hand, are reduced.
Example 2
Referring to fig. 4, the present embodiment provides an automatic control system for a wafer handling robot based on machine learning, and further includes a geometric center determining module;
the geometric center determining module is used for calculating the outline of the wafer, and is realized by calculating the curvature of a point on the outline, and for any point P on a curve, the calculation formula of the curvature K is as follows:
K=|dT/ds|；
wherein T represents a tangential vector at point P, s represents an arc length parameter, and dT/ds is the derivative of the tangential vector T with respect to the arc length parameter s;
calculating the curvature of each point on the contour to obtain a curvature set { K1, K2, K3...Kn };
traversing the obtained curvature sets { K1, K2, K3...Kn } to obtain a maximum curvature Ka and a minimum curvature Kb, judging that the wafer corresponding to the contour is a standard circle if the obtained |Ka-Kb| < C, C is a curvature threshold, and judging that the wafer corresponding to the contour is a non-standard circle if the |Ka-Kb|is not less than C, wherein the curvature threshold is self-set according to actual conditions, and the specific value in the embodiment is 0.1;
for standard circles, the geometric center can be calculated by using a method of minimum circumscribing, and the specific method comprises the following steps:
step S201, finding the distances from the points on all the contours to other points, then selecting three points with the smallest distances on the contours as the vertexes of the triangle,
step S202, connecting the three vertexes in sequence to obtain a triangle;
step S203, drawing vertical bisectors of three sides of the triangle, wherein points at which the vertical bisectors of the three sides intersect are the geometric centers;
for the non-standard circle, three points with the smallest distance and the same curvature are found on the outline, and the steps S202-S203 are repeated to obtain an offset center;
finding out a point with the maximum curvature on the outline, drawing a straight line passing through the point with the maximum curvature and the offset center to obtain a separation line, and controlling the execution tail end of the manipulator to adsorb the wafer by taking the area where the separation line is located as an adsorption area; therefore, the separation line is used as an adsorption area, two parts can be divided relatively uniformly, and the stability of carrying the wafer by the manipulator is further improved.
Example 3
Referring to fig. 5, the present embodiment provides an automatic control method for a wafer handling robot based on machine learning, including:
collecting a wafer image of a wafer to be conveyed;
preprocessing the wafer image;
extracting the outline of the preprocessed wafer image;
adding the outline of the extracted wafer image into the wafer image after pretreatment, and marking the obtained wafer image as a boundary wafer image;
calculating absolute values of sum and difference values of pixel values of corresponding pixel blocks on two sides of a boundary passing through the center of a boundary wafer image circle, and marking the absolute values as difference pixel values;
determining a wafer type according to the difference pixel value, wherein the wafer type comprises a uniform mass distribution wafer or a non-uniform mass distribution wafer;
selecting a manipulator adsorption area according to the uniform mass distribution wafer and the non-uniform mass distribution wafer;
and inputting the difference pixel values and the wafer types into a pre-constructed negative pressure curve adaptation model to obtain negative pressure curves corresponding to the wafer with uniform mass distribution and the wafer with non-uniform mass distribution in the carrying process.
Further, the method for preprocessing the wafer image comprises the following steps:
step S101, converting a wafer image into a wafer gray scale image by using an averaging method;
step S102, smoothing the image noise of the wafer by using a median filtering method.
Further, the method for extracting the outline of the preprocessed wafer image comprises the following steps of;
performing edge detection on the preprocessed wafer image;
performing binarization processing on the wafer image after edge detection, setting the point with the pixel value higher than the preset pixel threshold value as white, and setting the point with the pixel value lower than the preset pixel threshold value as black;
and (3) performing contour extraction by adopting a method of hollowing out internal points, if the target pixel in the wafer image is black and all 8 adjacent pixel points are black, deleting the target pixel point, namely setting the target pixel to be white, and traversing each pixel with the boundary removed to obtain an image only containing edge information, namely the contour of the wafer image.
Further, the method for determining the dividing line comprises the following steps:
and drawing a boundary line passing through the center of the boundary wafer image by the direction that the tail end of the manipulator enters the wafer to be adsorbed.
Further, the method for determining whether the wafer is a uniform mass distribution wafer or a non-uniform mass distribution wafer by the difference pixel values comprises the following steps:
comparing the difference pixel value with a preset pixel error threshold value for analysis, and judging the wafer as a wafer with uniform mass distribution if the difference pixel value is smaller than the preset pixel error threshold value; if the difference pixel value is greater than or equal to the preset pixel error threshold, the wafer is determined to be a non-uniform mass distribution wafer.
Further, the method for selecting the robot adsorption area according to the uniform mass distribution wafer and the non-uniform mass distribution wafer comprises the following steps:
if the wafer is uniformly mass distributed, selecting a dividing line as an adsorption area; if the wafer is a non-uniform mass distribution wafer, the demarcation line is shifted to the side with the larger sum of pixel values by a unit displacement amount, and the demarcation line is selected as an adsorption area after shifting.
Further, the training method of the negative pressure curve adaptation model comprises the following steps:
collecting a sample data set, wherein the sample data set comprises wafer characteristic data and a negative pressure value set in a carrying process corresponding to the wafer characteristic data, and the wafer characteristic data comprises a wafer type and a difference pixel value corresponding to the wafer type;
before training the negative pressure curve adaptation model, respectively carrying out numerical labeling on a wafer with uniform mass distribution, a wafer with non-uniform mass distribution and a negative pressure value set in the carrying process, taking wafer characteristic data as input of the negative pressure curve adaptation model, and taking the numerical labeling of the negative pressure value set as output of the negative pressure curve adaptation model; dividing sample data into a training set and a testing set, training the negative pressure curve adaptation model by using the training set to obtain an initial negative pressure curve adaptation model, and testing the initial negative pressure curve adaptation model by using the testing set to obtain a final negative pressure curve adaptation model meeting preset accuracy.
Further, the wafer image is obtained by the high-definition camera arranged at the execution end of the manipulator, and the mounting position of the high-definition camera cannot cause interference on the adsorption wafer at the execution end of the manipulator.
The automatic control system of the wafer carrying manipulator based on machine learning implements the automatic control method of the wafer carrying manipulator based on machine learning, and comprises the following steps:
the wafer image acquisition module is used for acquiring wafer images of wafers to be conveyed;
the wafer image preprocessing module is used for preprocessing the wafer image;
the contour extraction module is used for extracting the contour of the preprocessed wafer image;
the contour adding module is used for adding the contour of the extracted wafer image into the wafer image after pretreatment, and marking the obtained wafer image as a boundary wafer image;
the pixel value calculating module calculates the absolute value of the sum and difference value of pixel values of the corresponding pixel blocks at two sides of the boundary passing through the center of the boundary wafer image, and marks the absolute value as a difference pixel value;
the wafer type analysis module is used for judging the wafer type according to the difference pixel value, wherein the wafer type comprises a uniform mass distribution wafer or a non-uniform mass distribution wafer;
the adsorption area selection module is used for selecting a manipulator adsorption area according to the uniform mass distribution wafer and the non-uniform mass distribution wafer;
and the negative pressure curve self-adapting module is used for inputting the difference pixel value and the wafer type into a pre-constructed negative pressure curve adapting model to obtain negative pressure curves corresponding to the wafer with uniform mass distribution and the wafer with non-uniform mass distribution in the carrying process.
Example 4
Referring to fig. 6, the disclosure provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the automatic control method of the wafer handling robot based on machine learning provided by the above methods when executing the computer program.
Since the electronic device described in this embodiment is an electronic device used to implement the automatic control method of the wafer handling robot based on machine learning in this embodiment, based on the automatic control method of the wafer handling robot based on machine learning described in this embodiment, those skilled in the art can understand the specific implementation manner of the electronic device and various modifications thereof, so how to implement the method in this embodiment of the present application for this electronic device will not be described in detail herein. As long as those skilled in the art implement the electronic device adopted by the automatic control method of the wafer handling manipulator based on machine learning in the embodiments of the present application, the electronic device is within the scope of protection intended in the present application.
Example 5
Referring to fig. 6, the disclosure provides a computer readable storage medium, which includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor, wherein the processor implements the automatic control method of the wafer handling robot based on machine learning provided by the above methods when executing the computer program.
The above embodiments may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with embodiments of the present invention are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center over a wired network or a wireless network. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more sets of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state disk.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the several embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely one, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Finally: the foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (7)
1. The automatic control method of the wafer carrying manipulator based on machine learning is characterized by comprising the following steps:
collecting a wafer image of a wafer to be conveyed;
preprocessing the wafer image;
extracting the outline of the preprocessed wafer image;
adding the outline of the extracted wafer image into the wafer image after pretreatment, and marking the obtained wafer image as a boundary wafer image;
calculating absolute values of sum and difference values of pixel values of corresponding pixel blocks on two sides of a boundary passing through the center of a boundary wafer image circle, and marking the absolute values as difference pixel values;
determining a wafer type according to the difference pixel value, wherein the wafer type comprises a uniform mass distribution wafer or a non-uniform mass distribution wafer;
selecting a manipulator adsorption area according to the uniform mass distribution wafer and the non-uniform mass distribution wafer;
inputting the difference pixel values and the wafer types into a pre-constructed negative pressure curve adaptation model to obtain negative pressure curves corresponding to the wafer with uniform mass distribution and the wafer with non-uniform mass distribution in the carrying process;
the method for determining the dividing line comprises the following steps:
drawing a boundary line passing through the center of the boundary wafer image by the direction that the tail end of the manipulator enters the wafer to be adsorbed;
the method for judging whether the wafer is a uniform mass distribution wafer or a non-uniform mass distribution wafer through the difference pixel values comprises the following steps:
comparing the difference pixel value with a preset pixel error threshold value for analysis, and judging the wafer as a wafer with uniform mass distribution if the difference pixel value is smaller than the preset pixel error threshold value; if the difference pixel value is greater than or equal to the preset pixel error threshold value, the wafer is judged to be a wafer with non-uniform mass distribution;
the method for selecting the adsorption area of the manipulator according to the uniform mass distribution wafer and the non-uniform mass distribution wafer comprises the following steps:
if the wafer is uniformly mass distributed, selecting a dividing line as an adsorption area; if the wafer is a non-uniform mass distribution wafer, translating the demarcation line to the side with the larger sum of pixel values by a unit displacement amount, and selecting the translated demarcation line as an adsorption area;
the training method of the negative pressure curve adaptation model comprises the following steps:
collecting a sample data set, wherein the sample data set comprises wafer characteristic data and a negative pressure value set in a carrying process corresponding to the wafer characteristic data, and the wafer characteristic data comprises a wafer type and a difference pixel value corresponding to the wafer type;
before training the negative pressure curve adaptation model, respectively carrying out numerical labeling on a wafer with uniform mass distribution, a wafer with non-uniform mass distribution and a negative pressure value set in the carrying process, taking wafer characteristic data as input of the negative pressure curve adaptation model, and taking the numerical labeling of the negative pressure value set as output of the negative pressure curve adaptation model; dividing sample data into a training set and a testing set, training the negative pressure curve adaptation model by using the training set to obtain an initial negative pressure curve adaptation model, and testing the initial negative pressure curve adaptation model by using the testing set to obtain a final negative pressure curve adaptation model meeting preset accuracy.
2. The automatic control method of a wafer handling robot based on machine learning of claim 1, wherein the method of preprocessing the wafer image comprises:
step S101, converting a wafer image into a wafer gray scale image by using an averaging method;
step S102, smoothing the image noise of the wafer by using a median filtering method.
3. The automatic control method of a wafer handling robot based on machine learning of claim 1, wherein the method of extracting the contour of the preprocessed wafer image comprises;
performing edge detection on the preprocessed wafer image;
performing binarization processing on the wafer image after edge detection, setting the point with the pixel value higher than the preset pixel threshold value as white, and setting the point with the pixel value lower than the preset pixel threshold value as black;
and (3) performing contour extraction by adopting a method of hollowing out internal points, if the target pixel in the wafer image is black and all 8 adjacent pixel points are black, deleting the target pixel point, namely setting the target pixel to be white, and traversing each pixel with the boundary removed to obtain an image only containing edge information, namely the contour of the wafer image.
4. The automatic wafer handling robot control method based on machine learning according to claim 1, wherein the wafer image is acquired by a high definition camera provided at an execution end of the robot, and the high definition camera mounting position does not interfere with the adsorption of the wafer at the execution end of the robot.
5. A machine learning-based wafer handling robot automatic control system for implementing the machine learning-based wafer handling robot automatic control method according to any one of claims 1 to 4, comprising:
the wafer image acquisition module is used for acquiring wafer images of wafers to be conveyed;
the wafer image preprocessing module is used for preprocessing the wafer image;
the contour extraction module is used for extracting the contour of the preprocessed wafer image;
the contour adding module is used for adding the contour of the extracted wafer image into the wafer image after pretreatment, and marking the obtained wafer image as a boundary wafer image;
the pixel value calculating module calculates the absolute value of the sum and difference value of pixel values of the corresponding pixel blocks at two sides of the boundary passing through the center of the boundary wafer image, and marks the absolute value as a difference pixel value;
the wafer type analysis module is used for judging the wafer type according to the difference pixel value, wherein the wafer type comprises a uniform mass distribution wafer or a non-uniform mass distribution wafer;
the adsorption area selection module is used for selecting a manipulator adsorption area according to the uniform mass distribution wafer and the non-uniform mass distribution wafer;
and the negative pressure curve self-adapting module is used for inputting the difference pixel value and the wafer type into a pre-constructed negative pressure curve adapting model to obtain negative pressure curves corresponding to the wafer with uniform mass distribution and the wafer with non-uniform mass distribution in the carrying process.
6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the machine learning based wafer handling robot automatic control method of any one of claims 1 to 4 when the computer program is executed by the processor.
7. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program when executed by a processor implements the automatic control method of the wafer handling robot based on machine learning as set forth in any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410060511.6A CN117565062B (en) | 2024-01-16 | 2024-01-16 | Automatic control method and system for wafer carrying manipulator based on machine learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410060511.6A CN117565062B (en) | 2024-01-16 | 2024-01-16 | Automatic control method and system for wafer carrying manipulator based on machine learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117565062A CN117565062A (en) | 2024-02-20 |
| CN117565062B true CN117565062B (en) | 2024-03-19 |
Family
ID=89886660
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410060511.6A Active CN117565062B (en) | 2024-01-16 | 2024-01-16 | Automatic control method and system for wafer carrying manipulator based on machine learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117565062B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009054933A (en) * | 2007-08-29 | 2009-03-12 | Ryusyo Industrial Co Ltd | Wafer transfer system |
| JP2021016922A (en) * | 2019-07-22 | 2021-02-15 | ファナック株式会社 | Three-dimensional data generator and robot control system |
| CN113129304A (en) * | 2021-05-18 | 2021-07-16 | 郑州轻工业大学 | Part detection method based on machine vision |
| CN114310920A (en) * | 2022-03-15 | 2022-04-12 | 广东若贝特智能机器人科技有限公司 | Intelligent sorting parallel robot and control system thereof |
| CN115229770A (en) * | 2022-09-23 | 2022-10-25 | 佛山隆深机器人有限公司 | Robot vision device and control method |
| CN116277037A (en) * | 2023-05-19 | 2023-06-23 | 泓浒(苏州)半导体科技有限公司 | Wafer handling mechanical arm control system and method |
-
2024
- 2024-01-16 CN CN202410060511.6A patent/CN117565062B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009054933A (en) * | 2007-08-29 | 2009-03-12 | Ryusyo Industrial Co Ltd | Wafer transfer system |
| JP2021016922A (en) * | 2019-07-22 | 2021-02-15 | ファナック株式会社 | Three-dimensional data generator and robot control system |
| CN113129304A (en) * | 2021-05-18 | 2021-07-16 | 郑州轻工业大学 | Part detection method based on machine vision |
| CN114310920A (en) * | 2022-03-15 | 2022-04-12 | 广东若贝特智能机器人科技有限公司 | Intelligent sorting parallel robot and control system thereof |
| CN115229770A (en) * | 2022-09-23 | 2022-10-25 | 佛山隆深机器人有限公司 | Robot vision device and control method |
| CN116277037A (en) * | 2023-05-19 | 2023-06-23 | 泓浒(苏州)半导体科技有限公司 | Wafer handling mechanical arm control system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117565062A (en) | 2024-02-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7352260B2 (en) | Robot system with automatic object detection mechanism and its operating method | |
| US10679379B1 (en) | Robotic system with dynamic packing mechanism | |
| JP5612916B2 (en) | Position / orientation measuring apparatus, processing method thereof, program, robot system | |
| JP6415066B2 (en) | Information processing apparatus, information processing method, position and orientation estimation apparatus, robot system | |
| CN112318485B (en) | Object sorting system and image processing method and device thereof | |
| CN110065068B (en) | Robot assembly operation demonstration programming method and device based on reverse engineering | |
| CN104915947A (en) | Image processing device, system, image processing method, and image processing program | |
| CN115830018B (en) | Carbon block detection method and system based on deep learning and binocular vision | |
| CN110598771A (en) | Visual target identification method and device based on deep semantic segmentation network | |
| CN108109154A (en) | A kind of new positioning of workpiece and data capture method | |
| CN111539927A (en) | Detection process and algorithm of automobile plastic assembly fastening buckle lack-assembly detection device | |
| CN114399463A (en) | Saw blade picking method and system based on digital image processing | |
| CN113172636B (en) | Automatic hand-eye calibration method and device and storage medium | |
| CN117565062B (en) | Automatic control method and system for wafer carrying manipulator based on machine learning | |
| CN113012181B (en) | Novel quasi-circular detection method based on Hough transformation | |
| CN112338898B (en) | Image processing method and device of object sorting system and object sorting system | |
| EP4285315A1 (en) | Systems and methods for paint defect detection using machine learning | |
| Gao et al. | An automatic assembling system for sealing rings based on machine vision | |
| CN116594351A (en) | Numerical control machining unit system based on machine vision | |
| JP2021039457A (en) | Image processing method, edge model creation method, robot system, and article manufacturing method | |
| CN113191997B (en) | Clamp spring measuring method based on machine vision | |
| CN116109701A (en) | Object grabbing method based on passive dual-purpose high light reflection | |
| CN114004812A (en) | Threaded hole detection method and system adopting guide filtering and neural network model | |
| CN112614172A (en) | Plane and/or curved surface dividing method and system based on three-dimensional vision | |
| Milecki et al. | The application of a vision system to detect trajectory points for soldering robot programming |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |