CN103325106A - Moving workpiece sorting method based on LabVIEW - Google Patents

Abstract

Provided is a moving workpiece sorting method based on LabVIEW. The moving workpiece sorting method based on the LabVIEW comprises the following steps of (1) an image processing algorithm for video streams, wherein the image processing algorithm for the video stream is used for detecting the positions and the postures of moving workpieces from collected video, (2) a calibration algorithm for a camera, wherein the calibration algorithm for the camera is used for converting pixel coordinates of the moving workpieces in images of the camera into physical coordinates of the workpieces in a world coordinate system, (3) a correction algorithm for radial distortion of the camera, wherein the correction algorithm for the radial distortion of the camera is used for eliminating radial distortion of images introduced by a wide-angle lens, (4) a Kalman prediction algorithm, wherein the Kalman prediction algorithm is used for carrying out prediction on the positions of the moving workpieces to obtain the workpiece positions without the time lag, (5) a workpiece type recognition algorithm, wherein when the workpieces are conveyed through a conveyor belt, the camera collects the images of the workpieces, recognition is carried out on the types of the workpieces, and classification is facilitated, and (6) a motion control method for a mechanical arm, wherein the motion control method for the mechanical arm comprises a track planning method and a track control method, and the mechanical arm can select an optimal path to rapidly sort the workpieces.

Description

Sorting method of moving workpiece based on LabVIEW

technical field

This patent relates to the technical field of sorting methods based on machine vision, in particular to a visual detection method and a product sorting method.

Background technique

As a new technology in the field of automatic control, the mechanical arm is gradually being recognized by people for its great role: first, the mechanical arm can replace some manual labor to complete some repetitive physical labor; The pre-set process can complete the work efficiently and quickly; thirdly, the mechanical arm has greatly improved the working conditions of workers, significantly increased labor productivity, and accelerated the pace of industrial production automation.

With the rapid development of the economy, the competition of enterprises is becoming more and more fierce. In order to improve efficiency and reduce production costs, conveyor belts have been widely used. Conveyor belts are widely used in industrial production systems. The application of conveyor belts not only saves labor, improves production efficiency, but also reduces production costs, and plays a huge role in industrial production. Whether it is handling, assembly, or workpiece quality inspection, the target needs to be grasped first. Therefore, there is a high demand for the classification and grasping function of the target on the conveyor belt.

Traditional industrial robots generally use teaching or offline programming to perform path planning and motion programming for processing tasks. During the processing process, they simply repeat the pre-programmed actions. Therefore, traditional industrial robot control technology cannot analyze the moving workpiece. pick.

Contents of the invention

One purpose of this patent is to provide a moving workpiece detection method with high detection accuracy and adaptable to changes in lighting conditions.

Another object of this patent is to provide a workpiece sorting method with a high degree of automation, which can realize the sorting of more kinds of workpieces and has high reliability.

The moving workpiece sorting method of this patent comprises the following steps:

1) The image processing algorithm of the video stream, which is used to detect the position and posture of the moving workpiece from the collected video;

2) The calibration algorithm of the camera, which is used to convert the pixel coordinates of the moving workpiece in the camera image into the physical coordinates of the workpiece in the world coordinate system.

3) The radial distortion correction algorithm of the camera is used to eliminate the radial distortion of the image introduced by the wide-angle lens.

4) The Kalman prediction algorithm is used to predict the position of the moving workpiece and obtain the workpiece position without time lag.

5) The type recognition algorithm of the workpiece. When the workpiece passes through the conveyor belt, the camera collects the image of the workpiece to identify the type of the workpiece, which is convenient for classification.

6) The motion control method of the manipulator, including the trajectory planning method and the trajectory control method, so that the manipulator can choose the optimal path and quickly sort the workpieces.

Among them, the algorithm of image processing includes the extraction of moving objects and the posture detection of moving workpieces. This patent adopts the background difference method to obtain the image coordinates of the moving workpiece. The background subtraction method is a method to detect moving objects by comparing the current frame in the image sequence with the background reference model, and its performance depends on the background modeling technology used. Since the background model is constantly updated, it can adapt to changes in light, while the general threshold method must change its threshold when the light environment changes. Define image(x,y) as the current frame, acc(x,y) as the background model, frimage(x,y) as the foreground frame, α as the background update rate, the range is 0-1, acc(x,y) Initialized to 0.

$\mathrm{<span\; class="notranslate">\; acc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; image</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; acc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; frimage</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; image</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; acc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>$

Finally, according to the separated foreground image, the position of the workpiece is detected. The attitude detection of the moving workpiece is first processed through gray scale, and then the points whose gradient value exceeds the threshold in the selected detection direction are found, and the straight line fitting is performed on these points to calculate the angle.

Among them, the calibration algorithm of the camera is mainly aimed at the calibration of the monocular two-dimensional vision system. Mount the camera perpendicular to the work plane. The world coordinate system is located on the working plane, the vertical plane of the Z axis is downward, the camera coordinate system and the world coordinate system have no rotation, only translation relationship exists, so there is a rotation matrix R=I (identity matrix) and a translation matrix P=[0 0 d ] ^T , d is the distance from the camera optical axis center point O _c to the working plane.

$\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c\; z\; c"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">c</figure-callout></span><figure-callout\; id="1"\; label="c\; z\; c"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">\; </figure-callout>}}& \\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{}{\mathrm{<span\; class="notranslate">\; c</span>}}_{}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cc}\mathrm{<span\; class="notranslate">\; <figure-callout\; id="0"\; label="R\; P"\; filenames="2013101293679100003DEST\_PATH\_IMAGE001.png"\; state="\{\{state\}\}">R</figure-callout></span><figure-callout\; id="0"\; label="R\; P"\; filenames="2013101293679100003DEST\_PATH\_IMAGE001.png"\; state="\{\{state\}\}">\; </figure-callout>}& \mathrm{<span\; class="notranslate">\; </span>}\mathrm{<span\; class="notranslate">P</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="w\; z\; w"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">w</figure-callout></span><figure-callout\; id="1"\; label="w\; z\; w"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">\; </figure-callout>}}& \\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{}{\mathrm{<span\; class="notranslate">\; w</span>}}_{}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; <figure-callout\; id="1"\; label="y\; w\; d"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">y</figure-callout></span><figure-callout\; id="1"\; label="y\; w\; d"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">\; </figure-callout>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}& \\ \mathrm{<span\; class="notranslate">\; d</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

In the formula: (x _c , y _c , z _c ) are the coordinates of the scene point in the camera coordinate system, and (x _w , y _w , z _w ) are the coordinates of the scene point in the world coordinate system. The coordinates of the scene point in the camera coordinate system can be obtained from the above formula, which is easy to get, x _c = x _w , y _c = y _w , and the coordinates of the moving workpiece on the working plane z _w =0.

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="u\; v"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">u</figure-callout></span><figure-callout\; id="1"\; label="u\; v"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">\; </figure-callout>}& \\ \mathrm{<span\; class="notranslate">\; </span>}\mathrm{<span\; class="notranslate">v</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; the\; <figure-callout\; id="0"\; label="y\; v"\; filenames="2013101293679100003DEST\_PATH\_IMAGE001.png"\; state="\{\{state\}\}">y</figure-callout></span><figure-callout\; id="0"\; label="y\; v"\; filenames="2013101293679100003DEST\_PATH\_IMAGE001.png"\; state="\{\{state\}\}">\; </figure-callout>}}& {\mathrm{<span\; class="notranslate">\; </span>}}_{}{\mathrm{<span\; class="notranslate">v</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="z\; c"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">z</figure-callout></span><figure-callout\; id="1"\; label="z\; c"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">\; </figure-callout>}}_{\mathrm{<span\; class="notranslate">\; </span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; xd</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}}\\ {\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; yd</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="v"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">v</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; <figure-callout\; id="1"\; label="y\; w"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">y</figure-callout></span><figure-callout\; id="1"\; label="y\; w"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">\; </figure-callout>}}_{\mathrm{<span\; class="notranslate">\; </span>}}}\end{array}\right.$

Where (u, v) is the image coordinate of the reference point; (u ₁ , v ₁ ) is the image coordinate of point P ₁ , (x _w1 , y _w1 ) is the two-dimensional world coordinate of point P ₁ ; (u ₂ , v ₂ ) is the image coordinates of point P ₂ , (x _w2 , y _w2 ) is the two-dimensional world coordinates of point P ₂ ; (u ₀ , v ₀ ) is the image coordinates of the center of the optical axis of the camera; k _xd =k _x d, k _yd = _ky d is the calibrated camera parameter.

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; wi</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="w"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">w</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; xd</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; wi</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; the\; <figure-callout\; id="1"\; label="y\; w"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">y</figure-callout></span><figure-callout\; id="1"\; label="y\; w"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">\; </figure-callout>}}_{\mathrm{<span\; class="notranslate">\; </span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; yd</span>}}\end{array}\right.$

In the formula: (u _i , v _i ) is the image coordinate of any point P _i , (x _wi , y _wi ) is the two-dimensional world coordinate of point P _i .

Among them, since the wide-angle camera will have distortion, and the radial distortion is the most obvious, we correct the radial distortion of the camera. In this patent, the radial distortion matrix of the camera is fitted by a linear regression (LR) method. Firstly, the distortion equation of the radial distortion is given:

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; u</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}\mathrm{<span\; class="notranslate">\; uv</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 20</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 02</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ {\mathrm{<span\; class="notranslate">\; v</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}\mathrm{<span\; class="notranslate">\; uv</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 20</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 02</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right.$

$\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; u</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\\ {\mathrm{<span\; class="notranslate">\; v</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cccccc}{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 20</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 02</span>}}\\ {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 20</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 02</span>}}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; 1</span>}\\ \mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; v</span>}\\ \mathrm{<span\; class="notranslate">\; uv</span>}\\ {\mathrm{<span\; class="notranslate">\; u</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ {\mathrm{<span\; class="notranslate">\; v</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right]$

In the formula: (u, v) is the image coordinate of the reference point, (u', v') is the image coordinate after distortion correction of the reference point, a _st and b _st are correction coefficients, s, t=0,1,2 .

Among them, the speed of the moving workpiece is unknown, and the position prediction adopts the Kalman prediction algorithm. Two frames of images are collected respectively, with an interval of 1s, and the change Δx of the workpiece position is the estimated velocity value. Establish a system model, and then use the two-dimensional world coordinates of the moving workpiece calculated by the above-mentioned calibration method as the position observation value, and at the same time use the estimated speed as the speed measurement value to form an observation vector, and calculate the Kalman filter for the workpiece motion The best predicted value of the state.

Among them, the type recognition of the workpiece needs to collect samples of various workpieces, perform feature detection on each sample, train the shape model, and establish a database of workpiece samples. Then detect the features of the target in the region of interest (ROI) of the real-time image, match it with the model in the database, calculate the matching value, and select the result with the highest matching value as the output of the classifier.

Among them, the motion control method of the manipulator includes trajectory planning, trajectory control of the mechanical arm, and grasping control of the electric gripper. The robotic arm controller plans the trajectory of the robotic arm according to the workpiece position information predicted by the host computer, and controls the robotic arm to move according to the expected trajectory, and at the same time adjusts the posture of the gripper to grab the moving workpiece on the conveyor belt. Finally, different types of workpieces are classified and placed according to requirements, and then the next round of workpiece sorting is carried out.

Description of drawings

Figure 1 is a schematic diagram of the background update of the difference method;

Fig. 2 is a schematic diagram of the workpiece classifier;

Fig. 3 is the functional block diagram of the Kalman prediction system;

Fig. 4 is the relationship diagram between predicted position and observed position;

Figure 5 is the error diagram of predicted position and observed position.

Figure 6 is a schematic diagram of trajectory planning for the servo control of the robotic arm.

Figure 7 is a flow chart of the servo control of the manipulator.

Detailed ways

The technical solution of this patent will be further described below in conjunction with the accompanying drawings and embodiments.

Example:

To extract the moving target by background subtraction method, the background should be modeled first. By continuously updating the background model, it is possible to adapt to changes in ambient light. Figure 1 is a schematic diagram of the background update of the system, and the background frame is iteratively updated according to the cycle in the schematic diagram.

$\mathrm{<span\; class="notranslate">\; acc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; image</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; acc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; frimage</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; image</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; acc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>$

Where image(x,y) is the current frame, acc(x,y) is the background model, frimage(x,y) is the foreground frame, α is the background update rate, select α=0.15, acc(x,y) is initialized as 0. Finally, according to the separated foreground image, the position of the workpiece is detected.

The posture detection of the workpiece first processes the color image in grayscale, measures the maximum distance occupied by the vertical and horizontal directions of the workpiece, and judges whether the posture of the workpiece tends to be vertical or horizontal. If it tends to a vertical posture, it detects the edge from left to right; if it tends to a horizontal posture, it detects the edge from top to bottom. Edge detection needs to detect points where the gradient value exceeds the threshold in the selected direction, and then perform straight line fitting on these points to calculate the angle.

Workpiece recognition on the assembly line requires classification results to be given in a short time. The shape, texture and other features of the workpiece are relatively simple. Therefore, model matching is performed directly in the top view. The schematic diagram of the classifier is shown in Figure 2.

First, input samples of various workpieces in the pattern classify program of LabVIEW, perform feature detection on each sample, and train the shape model to establish a database of workpiece samples. Then detect the features of the target in the region of interest (ROI) of the real-time image, match it with the model in the database, calculate the matching value, and select the result with the highest matching value as the output of the classifier.

Distortion correction implementation process: In order to calculate the 12 parameters in the camera radial distortion matrix, at least 6 image coordinates before and after correction are required. Collect a picture of the calibration board, set the threshold so that the reference point can be detected, we select 12 points in the required correction area, and record their image coordinates (u _i , v _i ) and (u _i ', v _i ' ), i=1~12, see the table, form the equation shown in the following formula, and solve the parameters a and b.

$\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; u</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\\ {\mathrm{<span\; class="notranslate">\; v</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cccccc}{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 20</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 02</span>}}\\ {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 20</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 02</span>}}\end{array}\right]\left[\begin{array}{cccc}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& & \mathrm{<span\; class="notranslate">\; 1</span>}\\ {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& & {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}\\ {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& <span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span>& {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}\\ {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& <span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span>& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}\\ {{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="u"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">u</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}& {{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}& & {{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ {{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="v"\; filenames="127906DEST\_PATH\_IMAGE002.png"\; state="\{\{state\}\}">v</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}& {{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}& & {{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right]$

point sequence 1 2 3 4 5 6 7 8 9 10 11 12 u(pixel) 538.1 626.67 536.94 625.82 537.88 627.1 538.59 627.4 569.88 599.41 598.97 569.74 v(pixel) 330.81 325.28 190.68 192.61 225.62 225.48 296.06 292.15 260.09 259.49 225.48 294.81 u'(pixel) 550.9 653.71 551.16 653.94 551.22 653.83 551.14 653.87 585.58 620.01 620.11 585.55 v'(pixel) 348.46 347.99 210.82 210.58 244.95 245.01 314.02 314.02 279.49 279.76 244.98 314.12

The obtained distortion correction matrix is:

$\left[\begin{array}{cccccc}{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 20</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 02</span>}}\\ {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 20</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 02</span>}}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cccccc}\mathrm{<span\; class="notranslate">\; 253.137</span>}& \mathrm{<span\; class="notranslate">\; 0.11401</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.168791</span>}& \mathrm{<span\; class="notranslate">\; 3.66461</span>}\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 5</span>}& \mathrm{<span\; class="notranslate">\; 0.00088723</span>}\mathrm{<span\; class="notranslate">\; 3</span>}& \mathrm{<span\; class="notranslate">\; 0.00026485</span>}\\ \mathrm{<span\; class="notranslate">\; 94.3187</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.131471</span>}& \mathrm{<span\; class="notranslate">\; 0.666474</span>}& \mathrm{<span\; class="notranslate">\; 0.00060478</span>}\mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 4.54589</span>}\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 6</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1.97698</span>}\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 5</span>}\end{array}\right]$

The implementation process of monocular two-dimensional vision system calibration: select the corrected image coordinates and world coordinates of two reference points on the calibration board, point P ₁ : (u ₁ ', v ₁ ')=(454.62,194.04), (x ₁ ,y ₁ )=(280.70,2.41); point P ₂ : (u ₂ ',v ₂ ')=(324.29,280.64), (x ₂ ,y ₂ )=(69.89,142.48). According to the principle of the monocular two-dimensional vision system mentioned above, the calibration of the camera parameters is realized by programming.

The implementation process of the Kalman prediction algorithm: first establish a system model, and then use the above-mentioned calibration method to calculate the two-dimensional world coordinates of the moving workpiece as the position observation value, and at the same time use the estimated speed as the speed measurement value to form an observation vector. Then the optimal prediction value of the Kalman filter is obtained. Figure 3 is a block diagram of the prediction system. Model the following system:

$\mathrm{<span\; class="notranslate">\; dx</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; dt</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right]\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right]\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right]\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span>$

In this model, the first equation is the state equation of the system, x(t) is the state vector composed of the position x of the workpiece in the X direction, the velocity v _x , and the acceleration a _x , and u(t) is the control quantity of the motion system. If u(t)=0, the acceleration of the system is 0, which is a uniform motion, and w(t) is the system noise, which is generally set as Gaussian white noise with a mean value of zero. The second equation is the output equation, which defines the output value of the system. The output value y(t) of this system is defined as a vector composed of position and velocity (the measurement value must be consistent with the output value type), and v(t) is The measurement noise is also set to Gaussian white noise with zero mean.

The programming implementation of Kalman prediction algorithm: collect two frames of images respectively, with an interval of 1s, and the change Δx of the workpiece position is the estimated velocity value, which is input into the Kalman filter as the initial velocity v ₀ . According to the system model, the observation vector composed of position and velocity, the Kalman filter calculates the optimal estimated value of the motion state. FIG. 4 is a graph showing the relationship between the predicted position value X (solid line) and the observed position value X′ (dashed line). As shown in the figure, due to a certain deviation in the speed estimate, there is a certain deviation between X and X', but due to the continuous update of the Kalman prediction model, the predicted value X gradually approaches X', and the error between the two gradually converges to zero. Figure 5 shows the error between the predicted value X and the observed value X'. The dots in the figure are messy because of the observation noise, but it is easy to see that the error value tends to zero.

Grabbing control of SMC LEFH32K2-32-R16N3 electric gripper: Write the gripping control program through the programming software of the gripper, and set action 0 and action 1. Action 0 is grab and action 1 is release. Then send the coded signal to the SMC controller through the I/O port of the EPSON robotic arm controller to realize the control of the gripper. The "Out1,208" instruction of the robotic arm programming software sends binary code 11010000 from port 1 to reset the gripper. The "Out1,144" command sends binary code 10010000 and writes action 0. The "Out1,176" command sends the binary code 10110000, sets the drive bit to 1, and executes the grabbing action. Similarly, "Out1, 145", write action 1, "Out1, 177", execute release action.

Servo control of the EPSON SCARA-G6 robotic arm: the PC establishes a connection with the controller of the robotic arm through the TCP/IP protocol, and the IP address of the network port #201 of the controller is set in the same subnet as the PC, and then use ""OpenNet#201" command opens the corresponding network port, "Read#201,data1$,12" reads the received motion information including X, Y, V, deg (angle), and "MOVE" command controls the movement of the robotic arm. The system does not use real-time tracking because the robotic arm will block the moving workpiece. The system sets a specified position x ₀ =211 (refer to the coordinate system of the manipulator). When the x coordinate of the workpiece reaches x ₀ , it plans the trajectory of the manipulator, controls the movement of the manipulator according to the desired trajectory, and then commands the gripper to grab the workpiece , grab the workpiece to the desired position according to the requirements, and then send a signal to release the workpiece, and carry out the next round of grabbing. The whole servo control schematic diagram is shown in Figure 6, and the servo control flow chart is shown in Figure 7.

Claims (6)

1. workpiece method for sorting based on machine vision technique may further comprise the steps:

1) image processing algorithm of video flowing detects position and the attitude of workpiece the motion in the middle of the video that gathers with this algorithm;

2) calibration algorithm of video camera is converted into the pixel coordinate of Moving Workpieces in camera review with this calibration algorithm the physical coordinates of workpiece in the world coordinate system.

3) the Lens Distortion Correction algorithm of video camera is in order to eliminate the radial distortion of the image of introducing because of wide-angle lens.

4) Kalman Prediction algorithm is predicted in order to the position to Moving Workpieces, obtains the location of workpiece without time lag.5) the type identification algorithm of workpiece, when workpiece from the travelling belt through out-of-date, the camera acquisition workpiece image is identified the workpiece type, is convenient to classification.

6) motion control method of mechanical arm comprises method for planning track, and method for controlling trajectory makes mechanical arm can select optimal path, sorts rapidly workpiece.

2. the image processing algorithm of video flowing according to claim 1 is characterized in that detecting the background subtraction point-score of Moving Workpieces position and the method for detection workpiece attitude, may further comprise the steps:

1) video flowing that gathers is carried out background modeling:

acc(x,y)=α*image(x,y)+(1-α)*acc(x,y)

Wherein image (x, y) is present frame, and acc (x, y) is background model (being initialized as 0), and α is the context update rate, and its scope is 0-1, and system chooses α=0.15.

2) separate the prospect two field picture, detect the location of workpiece:

frimage(x,y)=image(x,y)-acc(x,y)

Wherein frimage (x, y) is the prospect frame.

3) colour picture being carried out gray scale processes.

4) with detecting workpiece vertical direction and the shared ultimate range of horizontal direction, judge that the attitude of workpiece is tending towards vertically or level.Then choose from left to right rim detection if be tending towards vertical attitude; Then choose from top to bottom rim detection if be tending towards horizontal attitude.Detection Grad on selected direction surpasses the point of threshold value, and these points are carried out fitting a straight line, calculates angle.

3. the Lens Distortion Correction algorithm of video camera according to claim 1 is characterized in that the method with linear regression (LR) has simulated the radial distortion matrix of video camera.May further comprise the steps:

1) sets up the Lens Distortion Correction model.

$\left\{\begin{array}{c}{u}^{\&prime;}=a_{00}+a_{10}u+a_{01}v+a_{11}\mathrm{uv}+a_{20}{u}^2+a_{02}{v}^2\\ v^{\&prime;}=b_{00}+b_{10}u+b_{01}v+b_{11}\mathrm{uv}+b_{20}{u}^2+b_{02}{v}^2\end{array}\right.$

In the formula: (u, v) is the image coordinate of reference point, and (u', v') is the image coordinate behind the reference point distortion correction, a _St, b _StBe correction coefficient, s, t=0,1,2.

2) obtain the image coordinate that 12 reference point are proofreaied and correct front and back, ask for the radial distortion matrix.

4. Kalman Prediction algorithm according to claim 1 is characterized in that need not to detect by scrambler the speed of travelling belt, and its prediction may further comprise the steps:

1) estimates workpiece motion s speed.

2) calculate optimum state estimation value according to observation signal and model estimated signal.

5. workpiece kind identification method according to claim 1 is characterized in that can realizing classification than the multiple types workpiece based on the workpiece sorter of LabVIEW visual development module (VDM), and directly the 3D workpiece sample that gathers is carried out the sorter training.May further comprise the steps:

1) sample of the various workpiece of input in the pattern of LabVIEW classify program carries out feature detection to each sample, and the training shapes model, sets up the workpiece sample database.

2) clarification of objective in the area-of-interest (ROI) of realtime graphic is detected, and mate with model in the database, calculate matching value, choose the highest result of matching value as the output of sorter.

6. workpiece automatic sorting method according to claim 1 is characterized in that EPSON SCARA-G6 mechanical arm trajectory planning and track following, makes its continuously grabbing workpiece.May further comprise the steps:

1) according to the Work position information of host computer prediction, mechanical arm is carried out trajectory planning.

2) the control mechanical arm moves according to desired trajectory.

3) adjust the jaw attitude, the Moving Workpieces on the crawl travelling belt.Last as requested dissimilar workpiece the classification puts, and then makes manipulator motion to holding point, waits for the sorting of next round workpiece.

Images