Summary of the invention
The technical problem to be solved in the present invention is: control method and control system that a kind of chip die bond machine is provided, by the mode of vision guide, die bond machine is controlled, there is good positioning effect, avoid occurring that the problem of the overall offset of shake and position location appears in mechanical platform motion.
The present invention solves the problems of the technologies described above taked technical scheme to be: a kind of control method of chip die bond machine, is characterized in that: it comprises the following steps:
Step 1, obtain the image of motion platform, in image, comprise wafer information;
Step 2, according to image determine wafer on motion platform with by the position relationship of operand:
2.1, the image of obtained motion platform is carried out to preliminary treatment;
2.2, pretreated imagery exploitation edge detection algorithm is obtained to edge image;
2.3, edge image extracts invariant moment features;
2.4,, according to invariant moment features, adopt sorting algorithm to obtain the three-dimensional coordinate of wafer; Sorting algorithm specifically comprises: the importance that adopts rough set theory to complete sample characteristics attribute is judged, obtains the attribute reduction of system, then according to the attribute after yojan, carries out SVM prediction classification;
2.5, set up Jacobian matrix model, the wafer under acquisition joint space coordinate on motion platform and the position relationship between its target location; The target location of wafer is on by operand;
Step 3, the position relationship that obtains according to step 2 are controlled end effector motion, make wafer movement on motion platform to its target location;
Step 4, control end effector carry out die bond.
Press such scheme, described preliminary treatment comprises carries out gray scale processing and binary conversion treatment to image.
Press such scheme, described edge detection algorithm selects canny boundary operator to calculate.
Press such scheme, described Jacobian matrix model is the image Jacobian matrix model based on BROYDEN model.
A control system for chip die bond machine, is characterized in that: it comprises with lower module:
Image collection module, for obtaining the image of motion platform, comprises wafer information in image;
Image is processed and target identification module, for according to image, determine wafer on motion platform with by the position relationship of operand, specifically comprise:
Pretreatment module, carries out preliminary treatment for the image of the motion platform to obtained;
Rim detection module, for obtaining edge image to pretreated imagery exploitation edge detection algorithm;
Characteristic extracting module, extracts invariant moment features for edge image;
Sort module, for according to invariant moment features, adopts sorting algorithm to obtain the three-dimensional coordinate of wafer; Sorting algorithm specifically comprises: the importance that adopts rough set theory to complete sample characteristics attribute is judged, obtains the attribute reduction of system, then according to the attribute after yojan, carries out SVM prediction classification;
Locating module, for setting up Jacobian matrix model, the wafer under acquisition joint space coordinate on motion platform and the position relationship between its target location; The target location of wafer is on by operand;
Motion-control module, controls end effector motion for process the position relationship obtaining with target identification module according to image, makes wafer movement on motion platform to its target location;
Die bond control module, carries out die bond for controlling end effector.
Beneficial effect of the present invention is: the mode by vision guide is controlled die bond machine, there is good positioning effect, there will not be mechanical platform motion to occur the problem of the overall offset of shake and position location, specifically, first with edge detection algorithm, obtain edge image, the data volume of participating in calculating sharply declines, thereby has greatly reduced amount of calculation; Bending moment does not have the statistical property of image, meets translation, gentry's contracting, rotates all constant consistency, therefore selects the invariant moment features of object that object is finally classified and identified; The importance that adopts rough set theory to complete sample characteristics attribute is judged, obtains the attribute reduction of system, then according to the attribute after yojan, carries out SVM prediction classification, avoids some unnecessary characteristic attribute to cause the classification erroneous judgement of system.
Embodiment
Below in conjunction with instantiation, the present invention will be further described.
A control method for chip die bond machine, comprises the following steps:
Step 1, obtain the image of motion platform, in image, comprise wafer information; In the present embodiment as shown in Figure 1;
Step 2, according to image determine wafer on motion platform with by the position relationship of operand:
2.1, the image of obtained motion platform is carried out to preliminary treatment; Preliminary treatment comprises carries out gray scale processing and binary conversion treatment to image;
2.2, pretreated imagery exploitation edge detection algorithm (selecting canny boundary operator to calculate in the present embodiment) is obtained to edge image; In the present embodiment as shown in Figure 2;
2.3, edge image extracts invariant moment features;
2.4,, according to invariant moment features, adopt sorting algorithm to obtain the three-dimensional coordinate of wafer; Sorting algorithm specifically comprises: the importance that adopts rough set theory to complete sample characteristics attribute is judged, obtains the attribute reduction of system, then according to the attribute after yojan, carries out SVM prediction classification;
2.5, set up Jacobian matrix model (being the image Jacobian matrix model based on BROYDEN model in the present embodiment), the wafer under acquisition joint space coordinate on motion platform and the position relationship between its target location; The target location of wafer is on by operand;
Step 3, the position relationship that obtains according to step 2 are controlled end effector motion, make wafer movement on motion platform to its target location;
Step 4, control end effector carry out die bond.
A control system for chip die bond machine, comprises with lower module:
Image collection module, for obtaining the image of motion platform, comprises wafer information in image;
Image is processed and target identification module, for according to image, determine wafer on motion platform with by the position relationship of operand, specifically comprise:
Pretreatment module, carries out preliminary treatment for the image of the motion platform to obtained;
Rim detection module, for obtaining edge image to pretreated imagery exploitation edge detection algorithm;
Characteristic extracting module, extracts invariant moment features for edge image;
Sort module, for according to invariant moment features, adopts sorting algorithm to obtain the three-dimensional coordinate of wafer; Sorting algorithm specifically comprises: the importance that adopts rough set theory to complete sample characteristics attribute is judged, obtains the attribute reduction of system, then according to the attribute after yojan, carries out SVM prediction classification;
Locating module, for setting up Jacobian matrix model, the wafer under acquisition joint space coordinate on motion platform and the position relationship between its target location; The target location of wafer is on by operand;
Motion-control module, controls end effector motion for process the position relationship obtaining with target identification module according to image, makes wafer movement on motion platform to its target location;
Die bond control module, carries out die bond for controlling end effector.
To attempting the acquisition process of information, be elaborated below.
In area of pattern recognition, the shape facility of image is the important object of feature extraction.Some the most basic two-dimensional shapes features all have direct relation with square, and bending moment does not have the statistical property of image, meet all constant consistency of translation, gentry's contracting, rotation, in field of image recognition, are widely used.Therefore, select the invariant moment features of object that object is finally classified and identified.
(p+q) rank square of image: given two-dimentional continuous function f (i, j), its (p+q) rank square M
pqcan be represented by formula (1).
M
pq=∫∫i
pj
qf(i,j)didj(p,q=0,1,2,...) (1),
P in formula, q, i, j is nonnegative integer, its span is 0,1,2....
In image calculation, the summation formula of general application (p+q) rank square, i.e. formula (2).
In formula (2), ii in formula, jj is natural number, its span is 1,2..., due to p and the desirable all nonnegative integers of q, therefore can generate the infinite set of a square, according to Pa Pulisi uniqueness theorem, this infinite set can be determined two dimensional image function f (i, j) itself.For bianry image, making its background value is 0, and the value in shape area is 1, and its zeroth order square represents the area of this shape area.
(p+q) center, rank square of image: in order to guarantee the location invariance of shape facility, must computer center square, the not bending moment value that the barycenter of object of take is initial point computed image.By zeroth order square and first moment, can be calculated the barycenter (i', j') of image, (p+q) center, rank square of this image array
computing formula is suc as formula shown in (3):
If center square is normalized according to shape area area, be about to
with
(wherein n=(p+q)/2+1) replaces, and the not bending moment obtaining so can meet yardstick independence.
In the current research about two dimension invariant moment, what mostly discuss is the extraction of overall intensity image object square.Obviously, will inevitably strengthen like this amount of calculation of system, affect the real-time of system.For this reason, we have proposed a kind of method of the Image Moment Invariants based on marginal information.First the method is applied canny boundary operator and is obtained edge image, and then edge image extracts its invariant moment features, and the not bending moment calculating so still can keep the region characteristic of square.Due to the effect of edge extracting, the data volume of participating in calculating sharply declines, thereby has greatly reduced amount of calculation.
The not bending moment of image is the function of 7 squares, meets translation, rotation and the yardstick consistency of shape, and it calculates suc as formula shown in (4).
Φ
1=m
20+m
02
Φ
2=(m
20-m
02)
2+4m
11
Φ
3=(m
30-3m
12)
2+(3m
21-m
03)
2
Φ
4=(m
30+m
12)
2+(m
21+m
03)
2
Φ
5=(m
30-3m
12)
2(m
30+m
12)[(m
30+m
12)
2-3(m
21+m
03)
2]
+(3m
21-m
03)(m
21-m
03)[3(m
30+m
12)
2-(m
21+m
03)
2]
Φ
6=(m
20-m
02)[(m
30+m
12)
2-3(m
21+m
03)
2]+
4m
11(m
30+m
12)(m
21+m
03)
Φ
7=(3m
12-m
30)
2(m
30+m
12)[(m
30+m
12)
2-3(m
21+m
03)
2]
-(m
03-3m
21)[3(m
30+m
12)
2-(m
21+m
03)
2] (4),
M in formula
ijfor ij square, i in formula, j is nonnegative integer, its span is 0,1,2....Φ
1-Φ
7for the function of 7 squares of bending moment not, the m in formula (4)
ijcan be by formula (3)
calculate.
Improved SVMs
(1) SVMs is theoretical
Support vector machine method is a kind of machine learning method on Statistical Learning Theory, it is based upon on VC theory and structural risk minimization principle basis, according to finite sample information, between the complexity of model and learning ability, seek best compromise effect, to obtaining better generalization ability.Its basic thought is exactly by a Nonlinear Mapping, in the data-mapping to of an input space high-dimensional feature space, then in this higher dimensional space, carries out linear classification.
If linear separability sample set is (x
i, y
i), (i=1 ..., n), x
i∈ R
d, i.e. x
id dimensional feature vector, y
i{ 1,1}, is classification number to ∈, and the general type of the linear discriminant function in its space is f (x
i)=wx
i+ b, wherein w is weight vector, b is classification thresholding.Classifying face equation is so:
w·x
i+b=0 (5),
If class m and class n linear separability in set, exist (w, b), make:
w·x
i+b>0,(x
i∈m)
w·x
i+b<0,(x
i∈n) (6),
Known according to formula (5), if w and b are exaggerated or dwindle simultaneously, constant by the definite classifying face of formula (5) so.Suppose that all samples in training set all meet | f (x
i)>=1|, even meet from the nearest sample of classifying face | f (x
i)=1|, the gap of classifying so just equals 2/||w||.Therefore gap maximum being equivalent to makes || and w|| is minimum.
Utilize lagrangian optimization method above-mentioned optimal classification face problem can be converted into dual problem, the decision function that solves the supported vector machine of the problems referred to above is:
f(v)=sign(∑t
iy
i(x
i,v)+b
*) (7),
B wherein
*for classification thresholds, t wherein
ifor Lagrange multiplier corresponding to each sample, y
i(x
i, v) being classification number, v is sample, sign () is sign function.For the situation of linearly inseparable, add a lax ξ
i>=0, so corresponding optimization problem becomes:
Wherein β is error punishment parameter.Suitably select the relevant parameter of SVM model to be of great importance to the effect of classification.
Provide several main training algorithm of SVMs below.The one, method of partition, it is the method that Vapnik proposes in order to solve fairly large SVM training problem; The 2nd, decomposition method, it is divided into working set and inoperative collection by training sample, and the number of samples of working set is much smaller than total number of samples, and algorithm is each only for the sample training in working set; In addition, solve in addition quadratic programming dual problem hidden lagrange formula algorithm, Incremental Learning Algorithm [, closest point algorithms etc.
Although SVMs has good classification performance, it can only be classified to two class samples, and in practical application, often needs a plurality of classifications to classify.Therefore, SVM need to be generalized in the problem of multicategory classification, support that Multiclass Classification has two kinds of modes, a kind of is that the sample of all categories is combined to the method for classifying; Another kind is decomposed and reconstituted method, is about to the method that multiclass problem is converted into a plurality of two class problems.For a plurality of micro parts identification problems in microoperation, the fuzzy support vector machine based on " one-to-many " method that we apply Taiwan's scholars Liu proposition carries out multi-object classify.Its basic thought is:
Decomposition strategy: be positive class from wherein selecting a class, remaining is negative class at every turn.
Combined strategy: the class that unknown sample is classified as to decision function value maximum.
(2) improved SVMs
Sample training for completion system, normally by after all characteristic attribute value normalization all for sample, will inevitably strengthen the amount of calculation of system like this, simultaneously because all properties all participates in computing, may cause because of some unnecessary characteristic attribute the classification erroneous judgement of system, therefore, a kind of importance method of discrimination of attributive character must be proposed.We introduce the importance judgement that rough set theory completes sample characteristics attribute, obtain the attribute reduction of system, then according to the attribute after yojan, carry out SVM prediction classification.
Paper rough set theory.Decision system S=(U, A, V, f), wherein U is universe, is a nonempty finite set; A=CUD, C and D are respectively conditional attribute collection and decision kind set; V is the codomain collection of attribute,
v
ait is the codomain of attribute a; F is information function, and f:U * A → V is right
, there is f (z, a) ∈ V in a ∈ A
a.
be a subset of conditional attribute set, claim that binary crelation Ind (B) is the undistinguishable relation of S=(U, A, V, f):
its indicated object θ and object λ are inseparable about the subset B of property set A.
The yojan of rough set attribute is under the prerequisite of loss of information not, to delete the attribute of redundancy, and the set R of the yojan collection of attribute can be expressed as
γ (R, D)=γ (C, D) }, wherein γ is Feature Dependence degree, so Feature Dependence degree equates can be used as the end condition of interative computation.
For completing the yojan of attribute, we have proposed a kind of based on can the didactic old attribute reduction algorithms of identification matrix, this algorithm application can identification matrix in the frequency that occurs of attribute as heuristic rule, thereby obtain the minimum Relative Reduced Concept of attribute.
Can by Skowron, be proposed by identification matrix, can identification entry of a matrix element c
ijbe defined as:
In formula (9), r (θ) is the value of object θ on attribute r, and D (θ) records the value of θ on D.Formula (9) represents: when decision attribute difference and conditional attribute are completely not identical yet, element value is mutually different combinations of attributes; When decision attribute is identical, element value is 0; When decision-making difference and conditional attribute when completely different, element value is-1.
If p (a) is the Importance of Attributes computing formula of attribute a, the frequency occurring according to attribute in can identification matrix so, we propose shape suc as formula the computing formula of (10):
In formula (10), ψ is general parameters, and obviously, the frequency that attribute occurs is larger, and its importance is also larger.Therefore, the heuristic rule of through type (10), first calculates the importance of each attribute, the attribute that then cancellation importance is less (less Importance of Attributes is got p (a) <0.6), thus obtain relative minimal reduction attribute.
Be below rough set can identification matrix heuristic old attribute reduction algorithms:
Input: decision table (U, AUD, V, f)
Output: Relative Reduced Concept
Algorithm steps:
(1) calculating can identification matrix;
(2) definite kernel attribute, and find out the community set that does not contain core attribute;
(3) obtain form D=∧ (∨ c of the conjunctive normal form of step (2) combinations of attributes
ij: (i=1,2,3...s; J=1,2,3...m)), then the community set obtaining is converted into the form of disjunctive normal form;
(4) according to formula (10), calculate and judge the importance of the attribute of gained;
(5) according to step (4), obtain the attribute of less importance, then deduct less importance attribute, obtain the attribute after yojan.
Characteristic attribute after yojan is sent in SVMs sample device and carried out model foundation.SVMs adopts gaussian kernel function
g wherein, h is general variance.Gaussian kernel function shows good learning performance in actual applications, its extrapolability weakens along with the increase of parameter σ, generally get σ=0.45, and error in formula (8) punishment parameter beta is to affect the Generalization Ability of study machine by controlling different mistakes minute rate, therefore, in application process, must consider the svm classifier device that two parameters just can obtain best performance.
Below, the self-calibration of the camera in the present invention is elaborated.
Image Jacobian matrix has provided the speed of end effector and the relation that characteristics of image changes.Consider q=[q
1, q
2... q
m]
rrepresent that robot is at the coordinate of task space, subscript R representing matrix transposition wherein, f=[f
1, f
2... f
m]
rbe robot at position coordinates corresponding to image space, can obtain so image Jacobian matrix defined matrix and be
In formula (11),
for the derivative of f,
for the derivative of q, J
q(q) be image Jacobian matrix
Q
1-q
mfor the coordinate of robot at task space
Image Jacobian matrix can obtain by demarcating the inside and outside parameter of robot and CCD sensor-based system.Yet, under dynamic or uncertain environment, obtain this demarcation of accurate system parameters normally impossible.Therefore, we adopt image Jacobian matrix to debate online the self-calibration visual servo method of knowledge, and application BROYDEN method is debated online and known image Jacobian matrix.
BROYDEN method can use formula (12) to represent
In formula (12), A
k+1for being the iteration amount of the k+1 time, A
kbe the iteration amount of the k time, y
(k)for function, s
(k)for independent variable.
Applying said method below comes construct image Jacobian matrix to debate knowledge model.
In formula (12), suppose that two sub-picture characteristic errors are e (q)=f
d-f
c, f wherein
dfor the characteristics of image of expectation, f
cfor current characteristics of image, e (q) Taylor expansion is obtained to formula (13)
Definition
for debating the N rank image Jacobian matrix of knowledge, have so
From formula (10) and (11), can obtain formula (15)
Therefore, if will
corresponding A, the corresponding y of Δ e, the corresponding s of Δ q, the corresponding k of n, can construct the image Jacobian matrix model based on BROYDEN model so, then according to BROYDEN algorithm, debates and knows image Jacobian matrix
image Jacobian matrix model based on BROYDEN model is as the formula (16):
Subscript T representing matrix transposition in formula (16), the self-calibration image Jacobian matrix that we have constructed BROYDEN method by formula (16) is debated knowledge model.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for a person skilled in the art, the present invention can have various modifications and variations.Within the spirit and principles in the present invention all, any modification of doing, be equal to replacement, improvement etc., within all should being included in protection scope of the present invention.