CN105303564A - Tower type crane load stereo pendulum angle vision detection method - Google Patents

Abstract

The invention provides a tower type crane load stereo pendulum angle vision detection method. The method comprises steps of left and right camera calibration, image acquisition, image conversion for gray scale images, filtering and denoising, image binaryzation, edge detection, lifting rope linear detection, white mark point plane coordinate point calculation, white mark point three-dimensional coordinate calculation and three-dimensional swing angle calculation. Through the method, an acceleration gauge and a gyroscope having expensive prices are prevented from being used, and high detection precision and relatively good application prospects are realized.

Description

The visible detection method of the three-dimensional pivot angle of a kind of tower crane load

Technical field

The present invention relates to a kind of visible detection method, especially a kind of visible detection method being directed to the three-dimensional pivot angle of tower crane load.

Background technology

Along with the development of yield-power, tower crane, as a kind of main handling machinery, in commercial production and covil construction, plays more and more important effect.Tower crane is ubiquity load swing problem in operation, can limit load derricking speed and speed of gyration, reduces load precision in place, too increases the labour intensity of operating personnel simultaneously, load can be caused time serious to break off relations, accidents caused.Therefore to eliminate or control overhead swings raising tower crane work efficiency, to reduce loading and unloading operation production hidden danger significant.

At present, the three-dimensional pivot angle of tower crane load Antisway Control System detects and realizes pivot angle detection primarily of the sensor such as accelerometer, gyroscope composition sensor assembly, and hardware cost is higher and signal transacting complicated.

Summary of the invention

The technical problem to be solved in the present invention is that existing three-dimensional pivot angle detects and mainly relies on various kinds of sensors, and cost is high and there is larger error.

In order to solve the problems of the technologies described above, the invention provides the visible detection method of the three-dimensional pivot angle of a kind of tower crane load, comprising the steps:

Step 1, demarcates respectively two cameras that left-right balance is installed, obtains the relative geometry position relation between two cameras;

Step 2, utilizes calibrated left and right camera to carry out image acquisition to the weight that tower crane carries;

Step 3, converts the left and right two width original color image collected to gray level image;

Step 4, carries out Wiener filtering to left and right two width gray level image and removes noise;

Step 5, maximizes inter-class variance to left and right two width denoising imagery exploitation OTSU global threshold and carries out region segmentation, obtains left and right two width bianry images;

Step 6, utilizes canny edge detection operator to carry out object edge detection to left and right two width bianry image, obtains left and right two breadths edge contour curve images;

Step 7, detects to left and right two breadths edge imagery exploitation Hough transform the straight line lifting rope hanging loads in tower crane image, and utilizes pixel value to detect to be arranged on the white marking point at lifting rope two ends;

Step 8, is designated as the origin coordinates point of image, is partitioned into lifting rope linearity region by the white marking point of the lifting rope head end detected;

Step 9, detects the ranks coordinate of the white marking point of lifting rope linearity region tail end in the two width images of left and right respectively;

Step 10, adopts the mid point of different surface beeline common vertical line to approach the white marking point three-dimensional coordinate of the lifting rope tail end that will recover, thus obtains the three-dimensional coordinate of the white marking point for recovering lifting rope tail end;

Step 11, calculates the three-dimensional pivot angle of this some distance lifting rope head end white marking basic point according to the three-dimensional coordinate for recovering lifting rope tail end white marking point.

Employing is demarcated respectively to the inner parameter of two cameras and external parameter, effectively can improve the precision of detection, calculates basis accurately for three-dimensional coordinate below calculates to establish; Utilize the white marking point of lifting rope head end, conveniently can be partitioned into lifting rope linearity region from image, improve the accuracy of lifting rope identification; Utilize the white marking point of lifting rope linearity region tail end in two width images to calculate the three-dimensional coordinate of the white marking point for recovering lifting rope tail end, thus calculate three-dimensional pivot angle further, complete the vision-based detection of the three-dimensional pivot angle of load.

As further restriction scheme of the present invention, in step 1, respectively timing signal is carried out to two cameras, specifically comprise the steps:

Step 1-1, adopts black and white chess pattern as calibrating template;

Step 1-2, is placed in the dead ahead of left and right camera by calibrating template, the planimetric position of conversion calibrating template and the anglec of rotation, gather N picture group sheet with left and right camera to the calibrating template of same position simultaneously;

Step 1-3, chooses about same group two width images, and determines the unique point on the upper left corner of all corresponding black and white grid in every width image, the upper right corner, four borders in the lower left corner and the lower right corner respectively;

Step 1-4, calculates inner parameter and the external parameter of two cameras in left and right respectively according to projection imaging planimetric coordinates (u, v) and the rectangular coordinate (X, Y) of unique point on calibrating template of character pair point.

The demarcation of two cameras is bases of vision-based detection, and this step is directly connected to the precision of whole vision inspection process, accurately demarcates the accuracy just guaranteeing vision-based detection result.

As further restriction scheme of the present invention, the concrete steps calculating the inner parameter of two cameras, external parameter and relative geometry position relation in step 1-4 are:

Step 1-41, is solved by least square method pointwise $\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=H\left[\begin{array}{c}X\\ Y\\ 1\end{array}\right]=\[\begin{array}{ccc}{h}_1& h_2& h_3\end{array}\]\left[\begin{array}{c}X\\ Y\\ 1\end{array}\right]=\left[\begin{array}{ccc}{h}_{11}& h_{12}& h_{13}\\ h_{21}& h_{22}& h_{23}\\ h_{31}& h_{32}& h_{33}\end{array}\right]\left[\begin{array}{c}X\\ Y\\ 1\end{array}\right]$ In homography matrix H, the i-th row vector of homography matrix H is h _i=[h _i1h _i2h _i3], the jth column vector of homography matrix H is h _j=[h _1jh _2jh _3j] ^t;

Step 1-42, makes symmetric matrix $B=A^{-T}{A}^{-1}=\left[\begin{array}{ccc}{B}_{11}& B_{12}& B_{13}\\ B_{21}& B_{22}& B_{23}\\ B_{31}& B_{32}& B_{33}\end{array}\right],$ Wherein, A is inner parameter matrix, $A=\left[\begin{array}{ccc}\mathrm{\α}& \mathrm{\γ}& u_0\\ 0& \mathrm{\β}& v_0\\ 0& 0& 1\end{array}\right],$ α, β, u ₀and v ₀for the inner parameter of camera, γ is the out of plumb factor between camera imaging plane diaxon, the i.e. angle of pixel ranks, and α is the horizontal direction focal length of camera, and β is the vertical direction focal length of camera, u ₀and v ₀for the principal point coordinate of camera, in desirable camera model, (u ₀, v ₀) be positioned at image central authorities, α=beta, gamma=0 or 90 °;

Step 1-43, then make v _ij=[h _i1h _1jh _i1h _2j+ h _i2h _1jh _i2h _2jh _i3h _1j+ h _i1h _3jh _i3h _2j+ h _i2h _3jh _i3h _3j] ^t, b=[B ₁₁b ₁₂b ₂₂b ₁₃b ₂₃b ₃₃] ^-T, gather N width image, row are write N number of $\left[\begin{array}{c}{v}_{12}^T\\ {(v_{11}-v_{22})}^T\end{array}\right]b=0$ Equation, the i.e. equation of Vb=0;

Step 1-44, solves V ^tv minimal eigenvalue characteristic of correspondence vector is the solution of b, inverts to b, utilizes Choleski to decompose the computing formula obtaining inner parameter to be:

$\left\{\begin{array}{c}\mathrm{\λ}=B_{33}-\[B_{13}^2+v_0\left(B_{12}{B}_{13}-B_{11}{B}_{23}\right)\]/B_{11}\\ \mathrm{\α}=\sqrt{\mathrm{\λ}/B_{11}}\\ \mathrm{\β}=\sqrt{\mathrm{\λ}/B_{11}/\left(B_{11}{B}_{22}-B_{12}^2\right)}\\ \mathrm{\γ}=-B_{12}{\mathrm{\α}}^2\mathrm{\β}/\mathrm{\λ}\\ v_0=\left(B_{12}{B}_{13}-B_{11}{B}_{23}\right)/\left(B_{11}{B}_{22}-B_{12}^2\right)\\ u_0={\mathrm{\γv}}_0/\mathrm{\β}-B_{13}{\mathrm{\α}}^2/\mathrm{\λ}\end{array}\right.---\left(3\right)$

Step 1-45, solves H=[h ₁h ₂h ₃]=λ A [r ₁r ₂t], the computing formula obtaining external parameter R and T of camera is:

$\left\{\begin{array}{c}{r}_1={\mathrm{\λA}}^{-1}{h}_1\\ r_2={\mathrm{\λA}}^{-1}{h}_2\\ r_3=r_1\×r_2\\ T={\mathrm{\λA}}^{-1}{h}_3\end{array}\right.---\left(4\right)$

Wherein, λ is scale factor, r _i(i=1,2,3) represent i-th column vector of 3 × 3 rotation matrix R, and namely i-th row vector of rotation matrix R is R _i=[r _i1r _i2r _i3], the jth column vector of rotation matrix R is R _j=[r _1jr _2jr _3j] ^t, T is the translation matrix of 3 × 1;

Step 1-46, then according to formula $\left\{\begin{array}{c}r=R_1{R}_2^{-1}\\ t=T_1-R_1{R}_2^{-1}{T}_2\end{array}\right.$ Calculate the relative geometry position relation between two cameras, wherein, r is relative rotation matrices, and t is relative translation matrix, R ₁and R ₂represent the rotation matrix of two cameras in left and right respectively, T ₁and T ₂be respectively the translation matrix of two cameras in left and right.

The inner parameter of two cameras and external parameter are the bases of the relative geometry position relation between calculating two cameras, after the geometry site determining two cameras, just can carry out vision-based detection.

As further restriction scheme of the present invention, the concrete steps obtaining the three-dimensional coordinate of the white marking point of lifting rope tail end in step 10 are:

Step 10-1, solves the m in following equation _x, m _y, m _z, n _x, n _yand n _z,

$\left[\begin{array}{cccccc}{a}_{11}& a_{12}& a_{13}& 0& 0& 0\\ a_{21}& a_{22}& a_{23}& 0& 0& 0\\ 0& 0& 0& a_{31}& a_{32}& a_{33}\\ 0& 0& 0& a_{41}& a_{42}& a_{43}\\ v_{L1}& v_{L2}& v_{L3}& -v_{L1}& -v_{L2}& -v_{L3}\\ v_{R1}& v_{R2}& v_{R3}& -v_{R1}& -v_{R2}& -v_{R3}\end{array}\right]\left[\begin{array}{c}{m}_x\\ m_y\\ m_z\\ n_x\\ n_y\\ n_z\end{array}\right]=\left[\begin{array}{c}{b}_1\\ b_2\\ b_3\\ b_4\\ 0\\ 0\end{array}\right]---\left(5\right)$

Wherein, the lifting rope straight line O of left camera imaging _lp _lwith the lifting rope straight line O of right camera imaging _rp _rdirection vector be respectively with lifting rope straight line O _lp _lthe normal vector of two planes at place is lifting rope straight line O _rp _rthe normal vector of two planes at place is ${\stackrel{\→}{v}}_L=\left(\begin{array}{ccc}{v}_{L1}& v_{L2}& v_{L3}\end{array}\right)={\stackrel{\→}{n}}_1\×{\stackrel{\→}{n}}_2,{\stackrel{\→}{v}}_R=\left(\begin{array}{ccc}{v}_{R1}& v_{R2}& v_{R3}\end{array}\right)={\stackrel{\→}{n}}_3\×{\stackrel{\→}{n}}_4,\left[\begin{array}{c}{b}_1\\ b_2\\ b_3\\ b_4\end{array}\right]=\left[\begin{array}{c}{m}_{14}^L-u_L{m}_{34}^L\\ m_{24}^L-v_L{m}_{34}^L\\ m_{14}^R-u_R{m}_{34}^R\\ m_{24}^R-v_R{m}_{34}^R\end{array}\right],$ (u _l, v _l) and (u _r, v _r) be the coordinate on left images, m ₁₄, m ₂₄and m ₃₄for calibrating parameters $M_{3\×4}=\left[\begin{array}{cccc}{m}_{11}& m_{12}& m_{13}& m_{14}\\ m_{21}& m_{22}& m_{23}& m_{24}\\ m_{31}& m_{32}& m_{33}& m_{34}\end{array}\right]=A\left[\begin{array}{cc}R& T\end{array}\right]$ The coefficient that matrix is corresponding;

Step 10-2, calculates the mid point P' of final common vertical line MN, and the three-dimensional coordinate of the white marking point namely on lifting rope tail end is: ${P_x}^{\′}=\frac{m_x+n_x}{2},{P_y}^{\′}=\frac{m_y+n_y}{2}$ And ${P_z}^{\′}=\frac{m_z+n_z}{2},$ Wherein (m _x, m _y, m _z) and (n _x, n _y, n _z) be respectively the lifting rope straight line O of left camera imaging _lp _lwith the lifting rope straight line O of right camera imaging _rp _rthe three-dimensional coordinate of common vertical line MN summit M and N point.

Beneficial effect of the present invention is: (1) employing is demarcated respectively to the inner parameter of two cameras and external parameter, effectively can improve the precision of detection, calculates basis accurately for three-dimensional coordinate below calculates to establish; (2) utilize the white marking point of lifting rope head end, conveniently can be partitioned into lifting rope linearity region from image, improve the accuracy of lifting rope identification; (3) utilize the white marking point of lifting rope linearity region tail end in two width images to calculate the three-dimensional coordinate of the white marking point for recovering lifting rope tail end, thus calculate three-dimensional pivot angle further, complete the vision-based detection of the three-dimensional pivot angle of load.

Accompanying drawing explanation

Fig. 1 is the visible detection method process flow diagram of the three-dimensional pivot angle of tower crane load of the present invention;

Fig. 2 is calibrating template schematic diagram in the visible detection method of the three-dimensional pivot angle of tower crane load of the present invention;

Fig. 3 is that the visible detection method binocular stereo vision of the three-dimensional pivot angle of tower crane load of the present invention detects schematic diagram;

Fig. 4 is common vertical line mid-point computation schematic diagram in the visible detection method of the three-dimensional pivot angle of tower crane load of the present invention.

Embodiment

As Figure 1-4, the visible detection method of the three-dimensional pivot angle of a kind of tower crane load of the present invention, comprises the steps:

Step 1, demarcate respectively two cameras that left-right balance is installed, obtain the relative geometry position relation between two cameras, concrete steps are:

Step 1-1, adopt black and white chess pattern as calibrating template, wherein black and white grid size is 20mm*20mm;

Step 1-2, is placed in the dead ahead of left and right camera by calibrating template, the planimetric position of conversion calibrating template and the anglec of rotation, gather N picture group sheet with left and right camera to the calibrating template of same position, preferred acquisition 20 picture group sheet of the present invention simultaneously;

Step 1-3, chooses about same group two width images, and determines the unique point on the upper left corner of all corresponding black and white grid in every width image, the upper right corner, four borders in the lower left corner and the lower right corner respectively;

Step 1-4, according to the projection imaging planimetric coordinates (u of character pair point, v) with the rectangular coordinate (X of unique point on calibrating template, Y) calculate inner parameter and the external parameter of two cameras in left and right respectively, the concrete steps of inner parameter, external parameter and relative geometry position relation are:

Step 1-41, is solved by least square method pointwise $\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=H\left[\begin{array}{c}X\\ Y\\ 1\end{array}\right]=\[\begin{array}{ccc}{h}_1& h_2& h_3\end{array}\]\left[\begin{array}{c}X\\ Y\\ 1\end{array}\right]=\left[\begin{array}{ccc}{h}_{11}& h_{12}& h_{13}\\ h_{21}& h_{22}& h_{23}\\ h_{31}& h_{32}& h_{33}\end{array}\right]\left[\begin{array}{c}X\\ Y\\ 1\end{array}\right]$ In homography matrix H, the i-th row vector of homography matrix H is h _i=[h _i1h _i2h _i3], the jth column vector of homography matrix H is h _j=[h _1jh _2jh _3j] ^t;

Step 1-42, makes symmetric matrix $B=A^{-T}{A}^{-1}=\left[\begin{array}{ccc}{B}_{11}& B_{12}& B_{13}\\ B_{21}& B_{22}& B_{23}\\ B_{31}& B_{32}& B_{33}\end{array}\right],$ Wherein, A is inner parameter matrix, $A=\left[\begin{array}{ccc}\mathrm{\α}& \mathrm{\γ}& u_0\\ 0& \mathrm{\β}& v_0\\ 0& 0& 1\end{array}\right],$ α, β, u ₀and v ₀for the inner parameter of camera, γ is the out of plumb factor between camera imaging plane diaxon, the i.e. angle of pixel ranks, and α is the horizontal direction focal length of camera, and β is the vertical direction focal length of camera, u ₀and v ₀for the principal point coordinate of camera, in desirable camera model, (u ₀, v ₀) be positioned at image central authorities, α=beta, gamma=0 or 90 °;

Step 1-43, then make v _ij=[h _i1h _1jh _i1h _2j+ h _i2h _1jh _i2h _2jh _i3h _1j+ h _i1h _3jh _i3h _2j+ h _i2h _3jh _i3h _3j] ^t, b=[B ₁₁b ₁₂b ₂₂b ₁₃b ₂₃b ₃₃] ^-T, gather 20 width images, row write 20 $\left[\begin{array}{c}{v}_{12}^T\\ {(v_{11}-v_{22})}^T\end{array}\right]b=0$ Equation, namely comprises the system of equations Vb=0 of 120 equations;

Step 1-44, solves V ^tv minimal eigenvalue characteristic of correspondence vector is the solution of b, inverts to b, utilizes Choleski to decompose the computing formula obtaining inner parameter to be:

$\left\{\begin{array}{c}\mathrm{\λ}=B_{33}-\[B_{13}^2+v_0\left(B_{12}{B}_{13}-B_{11}{B}_{23}\right)\]/B_{11}\\ \mathrm{\α}=\sqrt{\mathrm{\λ}/B_{11}}\\ \mathrm{\β}=\sqrt{\mathrm{\λ}/B_{11}/\left(B_{11}{B}_{22}-B_{12}^2\right)}\\ \mathrm{\γ}=-B_{12}{\mathrm{\α}}^2\mathrm{\β}/\mathrm{\λ}\\ v_0=\left(B_{12}{B}_{13}-B_{11}{B}_{23}\right)/\left(B_{11}{B}_{22}-B_{12}^2\right)\\ u_0={\mathrm{\γv}}_0/\mathrm{\β}-B_{13}{\mathrm{\α}}^2/\mathrm{\λ}\end{array}\right.---\left(3\right)$

Step 1-45, solves H=[h ₁h ₂h ₃]=λ A [r ₁r ₂t], the computing formula obtaining external parameter R and T of camera is:

$\left\{\begin{array}{c}{r}_1={\mathrm{\λA}}^{-1}{h}_1\\ r_2={\mathrm{\λA}}^{-1}{h}_2\\ r_3=r_1\×r_2\\ T={\mathrm{\λA}}^{-1}{h}_3\end{array}\right.---\left(4\right)$

Wherein, λ is scale factor, r _i(i=1,2,3) represent i-th column vector of 3 × 3 rotation matrix R, and namely i-th row vector of rotation matrix R is R _i=[r _i1r _i2r _i3], the jth column vector of rotation matrix R is R _j=[r _1jr _2jr _3j] ^t, T is the translation matrix of 3 × 1;

Step 1-46, then according to formula $\left\{\begin{array}{c}r=R_1{R}_2^{-1}\\ t=T_1-R_1{R}_2^{-1}{T}_2\end{array}\right.$ Calculate the relative geometry position relation between two cameras, wherein, r is relative rotation matrices, and t is relative translation matrix, R ₁and R ₂represent the rotation matrix of two cameras in left and right respectively, T ₁and T ₂be respectively the translation matrix of two cameras in left and right;

Step 2, utilizes calibrated left and right camera to carry out image acquisition to the weight that tower crane carries;

Step 3, converts the left and right two width original color image collected to gray level image;

Step 4, carries out Wiener filtering to left and right two width gray level image and removes noise;

Step 5, maximizes inter-class variance to left and right two width denoising imagery exploitation OTSU global threshold and carries out region segmentation, obtains left and right two width bianry images;

Step 6, utilizes canny edge detection operator to carry out object edge detection to left and right two width bianry image, obtains left and right two breadths edge contour curve images;

Step 7, detects to left and right two breadths edge imagery exploitation Hough transform the straight line lifting rope hanging loads in tower crane image, and utilizes pixel value to detect to be arranged on the white marking point at lifting rope two ends;

Step 8, is designated as the origin coordinates point of image, is partitioned into lifting rope linearity region by the white marking point of the lifting rope head end detected;

Step 9, detects the ranks coordinate of the white marking point of lifting rope linearity region tail end in the two width images of left and right respectively;

Step 10, adopts the mid point of different surface beeline common vertical line to approach the white marking point three-dimensional coordinate of the lifting rope tail end that will recover, thus obtains the three-dimensional coordinate of the white marking point for recovering lifting rope tail end, and concrete steps are:

Step 10-1, solves the m in following equation _x, m _y, m _z, n _x, n _yand n _z,

$\left[\begin{array}{cccccc}{a}_{11}& a_{12}& a_{13}& 0& 0& 0\\ a_{21}& a_{22}& a_{23}& 0& 0& 0\\ 0& 0& 0& a_{31}& a_{32}& a_{33}\\ 0& 0& 0& a_{41}& a_{42}& a_{43}\\ v_{L1}& v_{L2}& v_{L3}& -v_{L1}& -v_{L2}& -v_{L3}\\ v_{R1}& v_{R2}& v_{R3}& -v_{R1}& -v_{R2}& -v_{R3}\end{array}\right]\left[\begin{array}{c}{m}_x\\ m_y\\ m_z\\ n_x\\ n_y\\ n_z\end{array}\right]=\left[\begin{array}{c}{b}_1\\ b_2\\ b_3\\ b_4\\ 0\\ 0\end{array}\right]---\left(5\right)$

Wherein, the lifting rope straight line O of left camera imaging _lp _lwith the lifting rope straight line O of right camera imaging _rp _rdirection vector be respectively with lifting rope straight line O _lp _lthe normal vector of two planes at place is lifting rope straight line O _rp _rthe normal vector of two planes at place is ${\stackrel{\→}{v}}_L=\left(\begin{array}{ccc}{v}_{L1}& v_{L2}& v_{L3}\end{array}\right)={\stackrel{\→}{n}}_1\×{\stackrel{\→}{n}}_2,{\stackrel{\→}{v}}_R=\left(\begin{array}{ccc}{v}_{R1}& v_{R2}& v_{R3}\end{array}\right)={\stackrel{\→}{n}}_3\×{\stackrel{\→}{n}}_4,\left[\begin{array}{c}{b}_1\\ b_2\\ b_3\\ b_4\end{array}\right]=\left[\begin{array}{c}{m}_{14}^L-u_L{m}_{34}^L\\ m_{24}^L-v_L{m}_{34}^L\\ m_{14}^R-u_R{m}_{34}^R\\ m_{24}^R-v_R{m}_{34}^R\end{array}\right],$ (u _l, v _l) and (u _r, v _r) be the coordinate on left images, m ₁₄, m ₂₄and m ₃₄for calibrating parameters $M_{3\×4}=\left[\begin{array}{cccc}{m}_{11}& m_{12}& m_{13}& m_{14}\\ m_{21}& m_{22}& m_{23}& m_{24}\\ m_{31}& m_{32}& m_{33}& m_{34}\end{array}\right]=A\left[\begin{array}{cc}R& T\end{array}\right]$ The coefficient that matrix is corresponding;

Step 10-2, calculates the mid point P' of final common vertical line MN, and the three-dimensional coordinate of the white marking point namely on lifting rope tail end is: ${P_x}^{\′}=\frac{m_x+n_x}{2},{P_y}^{\′}=\frac{m_y+n_y}{2}$ And ${P_z}^{\′}=\frac{m_z+n_z}{2},$ Wherein (m _x, m _y, m _z) and (n _x, n _y, n _z) be respectively the lifting rope straight line O of left camera imaging _lp _lwith the lifting rope straight line O of right camera imaging _rp _rthe three-dimensional coordinate of common vertical line MN summit M and N point;

Step 11, calculates the three-dimensional pivot angle of this some distance lifting rope head end white marking basic point according to the three-dimensional coordinate for recovering lifting rope tail end white marking point.

Employing is demarcated respectively to the inner parameter of two cameras and external parameter, effectively can improve the precision of detection, calculates basis accurately for three-dimensional coordinate below calculates to establish; Utilize the white marking point of lifting rope head end, conveniently can be partitioned into lifting rope linearity region from image, improve the accuracy of lifting rope identification; Utilize the white marking point of lifting rope linearity region tail end in two width images to calculate the three-dimensional coordinate of the white marking point for recovering lifting rope tail end, thus calculate three-dimensional pivot angle further, complete the vision-based detection of the three-dimensional pivot angle of load; The demarcation of two cameras is bases of vision-based detection, and this step is directly connected to the precision of whole vision inspection process, accurately demarcates the accuracy just guaranteeing vision-based detection result; The inner parameter of two cameras and external parameter are the bases of the relative geometry position relation between calculating two cameras, after the geometry site determining two cameras, just can carry out vision-based detection.

The visible detection method of the three-dimensional pivot angle of tower crane load of the present invention's design, the lengthy and tedious rough detection operation control procedure that Traditional Man is visual can be replaced, automatically the three-dimensional oscillating scope of lifting rope weight relative to equilibrium position is detected accurately, operate relative to traditional manual detection, the method of the present invention's design has that automaticity is high, accuracy of detection is high, speed is fast, reliability is high, is easy to realize, and with low cost, that applicability is strong advantage.

To sum up, by setting up and implementing the visible detection method of the three-dimensional pivot angle of tower crane load of the present invention's design, can realize lifting rope slinging weight moment or have the detection of the non-equilibrium state produced during external interference, provide three-dimensional pivot angle, for follow-up robotization anti-swing control establishes important foundation, there is wide market application foreground and economic worth.

By reference to the accompanying drawings embodiments of the present invention are explained in detail above, but the present invention is not limited to above-mentioned embodiment, in the ken that those of ordinary skill in the art possess, can also makes a variety of changes under the prerequisite not departing from present inventive concept.

Claims (4)

1. a visible detection method for the three-dimensional pivot angle of tower crane load, is characterized in that, comprise the steps:

Step 1, demarcates respectively two cameras that left-right balance is installed, obtains the relative geometry position relation between two cameras;

Step 2, utilizes calibrated left and right camera to carry out image acquisition to the weight that tower crane carries;

Step 3, converts the left and right two width original color image collected to gray level image;

Step 4, carries out Wiener filtering to left and right two width gray level image and removes noise;

Step 5, maximizes inter-class variance to left and right two width denoising imagery exploitation OTSU global threshold and carries out region segmentation, obtains left and right two width bianry images;

Step 6, utilizes canny edge detection operator to carry out object edge detection to left and right two width bianry image, obtains left and right two breadths edge contour curve images;

Step 7, detects to left and right two breadths edge imagery exploitation Hough transform the straight line lifting rope hanging loads in tower crane image, and utilizes pixel value to detect to be arranged on the white marking point at lifting rope two ends;

Step 8, is designated as the origin coordinates point of image, is partitioned into lifting rope linearity region by the white marking point of the lifting rope head end detected;

Step 9, detects the ranks coordinate of the white marking point of lifting rope linearity region tail end in the two width images of left and right respectively;

Step 10, adopts the mid point of different surface beeline common vertical line to approach the white marking point three-dimensional coordinate of the lifting rope tail end that will recover, thus obtains the three-dimensional coordinate of the white marking point for recovering lifting rope tail end;

Step 11, calculates the three-dimensional pivot angle of this some distance lifting rope head end white marking basic point according to the three-dimensional coordinate for recovering lifting rope tail end white marking point.

2. the visible detection method of the three-dimensional pivot angle of tower crane load according to claim 1, is characterized in that, carry out timing signal respectively, specifically comprise the steps: in step 1 to two cameras

Step 1-1, adopts black and white chess pattern as calibrating template;

Step 1-2, is placed in the dead ahead of left and right camera by calibrating template, the planimetric position of conversion calibrating template and the anglec of rotation, gather N picture group sheet with left and right camera to the calibrating template of same position simultaneously;

Step 1-3, chooses about same group two width images, and determines the unique point on the upper left corner of all corresponding black and white grid in every width image, the upper right corner, four borders in the lower left corner and the lower right corner respectively;

Step 1-4, calculates inner parameter and the external parameter of two cameras in left and right respectively according to projection imaging planimetric coordinates (u, v) and the rectangular coordinate (X, Y) of unique point on calibrating template of character pair point.

3. the visible detection method of the three-dimensional pivot angle of tower crane load according to claim 2, it is characterized in that, the concrete steps calculating the inner parameter of two cameras, external parameter and relative geometry position relation in step 1-4 are:

Step 1-41, is solved by least square method pointwise $\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=H\left[\begin{array}{c}X\\ Y\\ 1\end{array}\right]=\[\begin{array}{ccc}{h}_1& h_2& h_3\end{array}\]\left[\begin{array}{c}X\\ Y\\ 1\end{array}\right]=\left[\begin{array}{ccc}{h}_{11}& h_{12}& h_{13}\\ h_{21}& h_{22}& h_{23}\\ h_{31}& h_{32}& h_{33}\end{array}\right]\left[\begin{array}{c}X\\ Y\\ 1\end{array}\right]$ In homography matrix H, the i-th row vector of homography matrix H is h _i=[h _i1h _i2h _i3], the jth column vector of homography matrix H is h _j=[h _1jh _2jh _3j] ^t;

Step 1-42, makes symmetric matrix $B=A^{-T}{A}^{-1}=\left[\begin{array}{ccc}{B}_{11}& B_{12}& B_{13}\\ B_{21}& B_{22}& B_{23}\\ B_{31}& B_{32}& B_{33}\end{array}\right],$ Wherein, A is inner parameter matrix, $A=\left[\begin{array}{ccc}\mathrm{\α}& \mathrm{\γ}& u_0\\ 0& \mathrm{\β}& v_0\\ 0& 0& 1\end{array}\right],$ α, β, u ₀and v ₀for the inner parameter of camera, γ is the out of plumb factor between camera imaging plane diaxon, the i.e. angle of pixel ranks, and α is the horizontal direction focal length of camera, and β is the vertical direction focal length of camera, u ₀and v ₀for the principal point coordinate of camera, in desirable camera model, (u ₀, v ₀) be positioned at image central authorities, α=beta, gamma=0 or 90 °;

Step 1-43, then make v _ij=[h _i1h _1jh _i1h _2j+ h _i2h _1jh _i2h _2jh _i3h _1j+ h _i1h _3jh _i3h _2j+ h _i2h _3jh _i3h _3j] ^t, b=[B ₁₁b ₁₂b ₂₂b ₁₃b ₂₃b ₃₃] ^-T, gather N width image, row are write N number of $\left[\begin{array}{c}{v}_{12}^T\\ {(v_{11}-v_{22})}^T\end{array}\right]b=0$ Equation, the i.e. equation of Vb=0;

Step 1-44, solves V ^tv minimal eigenvalue characteristic of correspondence vector is the solution of b, inverts to b, utilizes Choleski to decompose the computing formula obtaining inner parameter to be:

$\left\{\begin{array}{c}\mathrm{\λ}=B_{33}-\[B_{13}^2+v_0(B_{12}{B}_{13}-B_{11}{B}_{23})\]/B_{11}\\ \mathrm{\α}=\sqrt{\mathrm{\λ}/B_{11}}\\ \mathrm{\β}=\sqrt{\mathrm{\λ}/B_{11}/(B_{11}{B}_{22}-B_{12}^2)}\\ \mathrm{\γ}=-B_{12}{\mathrm{\α}}^2\mathrm{\β}/\mathrm{\λ}\\ v_0=(B_{12}{B}_{13}-B_{11}{B}_{23})/(B_{11}{B}_{22}-B_{12}^2)\\ u_0={\mathrm{\γv}}_0/\mathrm{\β}-B_{13}{\mathrm{\α}}^2/\mathrm{\λ}\end{array}\right.---\left(3\right)$

Step 1-45, solves H=[h ₁h ₂h ₃]=λ A [r ₁r ₂t], the computing formula obtaining external parameter R and T of camera is:

$\left\{\begin{array}{c}{r}_1={\mathrm{\λA}}^{-1}{h}_1\\ r_2={\mathrm{\λA}}^{-1}{h}_2\\ r_3=r_1\×r_2\\ T={\mathrm{\λA}}^{-1}{h}_3\end{array}\right.---\left(4\right)$

Wherein, λ is scale factor, r _i(i=1,2,3) represent i-th column vector of 3 × 3 rotation matrix R, and namely i-th row vector of rotation matrix R is R _i=[r _i1r _i2r _i3], the jth column vector of rotation matrix R is R _j=[r _1jr _2jr _3j] ^t, T is the translation matrix of 3 × 1;

Step 1-46, then according to formula $\left\{\begin{array}{c}r=R_1{R}_2^{-1}\\ t=T_1-R_1{R}_2^{-1}{T}_2\end{array}\right.$ Calculate the relative geometry position relation between two cameras, wherein, r is relative rotation matrices, and t is relative translation matrix, R ₁and R ₂represent the rotation matrix of two cameras in left and right respectively, T ₁and T ₂be respectively the translation matrix of two cameras in left and right.

4. the visible detection method of the three-dimensional pivot angle of tower crane load according to claim 1, it is characterized in that, the concrete steps obtaining the three-dimensional coordinate of the white marking point of lifting rope tail end in step 10 are:

Step 10-1, solves the m in following equation _x, m _y, m _z, n _x, n _yand n _z,

$\left[\begin{array}{cccccc}{a}_{11}& a_{12}& a_{13}& 0& 0& 0\\ a_{21}& a_{22}& a_{23}& 0& 0& 0\\ 0& 0& 0& a_{31}& a_{32}& a_{33}\\ 0& 0& 0& a_{41}& a_{42}& a_{43}\\ v_{L1}& v_{L2}& v_{L3}& -v_{L1}& -v_{L2}& -v_{L3}\\ v_{R1}& v_{R2}& v_{R3}& -v_{R1}& -v_{R2}& -v_{R3}\end{array}\right]\left[\begin{array}{c}{m}_x\\ m_y\\ m_z\\ n_x\\ n_y\\ n_z\end{array}\right]=\left[\begin{array}{c}{b}_1\\ b_2\\ b_3\\ b_4\\ 0\\ 0\end{array}\right]---\left(5\right)$

Wherein, the lifting rope straight line O of left camera imaging _lp _lwith the lifting rope straight line O of right camera imaging _rp _rdirection vector be respectively with lifting rope straight line O _lp _lthe normal vector of two planes at place is ${\stackrel{\→}{n}}_1=\left(\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\end{array}\right),$ ${\stackrel{\→}{n}}_3=\left(\begin{array}{ccc}{a}_{21}& a_{22}& a_{23}\end{array}\right),$ Lifting rope straight line O _rp _rthe normal vector of two planes at place is ${\stackrel{\→}{n}}_3=\left(\begin{array}{ccc}{a}_{31}& a_{32}& a_{33}\end{array}\right),$ ${\stackrel{\→}{n}}_4=\left(\begin{array}{ccc}{a}_{41}& a_{42}& a_{43}\end{array}\right),$ ${\stackrel{\→}{v}}_L=\left(\begin{array}{ccc}{v}_{L1}& v_{L2}& v_{L3}\end{array}\right)={\stackrel{\→}{n}}_1\×{\stackrel{\→}{n}}_2,$ ${\stackrel{\→}{v}}_R=\left(\begin{array}{ccc}{v}_{R1}& v_{R2}& v_{R3}\end{array}\right)={\stackrel{\→}{n}}_3\×{\stackrel{\→}{n}}_4,$ $\left[\begin{array}{c}{b}_1\\ b_2\\ b_3\\ b_4\end{array}\right]=\left[\begin{array}{c}{m}_{14}^L-u_L{m}_{34}^L\\ m_{24}^L-v_L{m}_{34}^L\\ m_{14}^R-u_R{m}_{34}^R\\ m_{24}^R-v_R{m}_{34}^R\end{array}\right],$ (u _l, v _l) and (u _r, v _r) be the coordinate on left images, m ₁₄, m ₂₄and m ₃₄for calibrating parameters $M_{3\×4}=\left[\begin{array}{cccc}{m}_{11}& m_{12}& m_{13}& m_{14}\\ m_{21}& m_{22}& m_{23}& m_{24}\\ m_{31}& m_{32}& m_{33}& m_{34}\end{array}\right]=A\[\begin{array}{cc}R& T\end{array}\]$ The coefficient that matrix is corresponding;

Step 10-2, calculates the mid point P' of final common vertical line MN, and the three-dimensional coordinate of the white marking point namely on lifting rope tail end is: ${P_x}^{\′}=\frac{m_x+n_x}{2},$ ${P_y}^{\′}=\frac{m_y+n_y}{2}$ And ${P_z}^{\′}=\frac{m_z+n_z}{2},$ Wherein (m _x, m _y, m _z) and (n _x, n _y, n _z) be respectively the lifting rope straight line O of left camera imaging _lp _lwith the lifting rope straight line O of right camera imaging _rp _rthe three-dimensional coordinate of common vertical line MN summit M and N point.