CN103559703B - Crane barrier based on binocular vision is monitored and method for early warning and system - Google Patents

Abstract

The invention discloses a kind of crane barrier monitoring based on binocular vision and method for early warning and system, its method includes: step 1, structure binocular vision model, initializes relevant parameter;Step 2, real-time image acquisition, and carry out pretreatment;Step 3, detection object edge, extract characteristics of image;Step 4, complete left and right image registration, split public territory;Step 5, iterate, distinguish prospect and background;Step 6, reconstruction three-dimensional information, identify prospect and barrier;Step 7, ask for kinematic parameter, analyze and export early warning information.The present invention is based on adopting binocular vision system, monitor the working environment of crane in real time, efficiently solve a difficult problem for the pre-anticollision of crane, have efficiency height, precision is suitable, system structure is simple, low cost and other advantages, is very suitable for manufacturing on-the-spot online, noncontact Product checking and quality control.

Description

Crane barrier based on binocular vision is monitored and method for early warning and system

Technical field

The present invention relates to a kind of crane barrier monitoring based on binocular vision and method for early warning and system.

Background technology

The industries such as hoisting machinery industry is China's metallurgy, mine, railway, shipbuilding, transportation, hydraulic and electric engineering provide various hoisting machineries equipment, are industries indispensable in the development of the national economy.Crane belongs to the one of hoisting machinery, be a kind of circulate, the machinery of intermittent movement, be widely applied to now in the industry-by-industry of people's life, provided more facility for people.

At present, the real-time of crane and safety are then one of hot issues of widely paying close attention to.Crane is when operation, at crane under the traction of arm, weight and hang thing and have the motion of certain speed, also have barrier such as pedestrian, the UFO etc. of motion in working site simultaneously, once sash weight collides with barrier with hanging thing, probably crane, barrier are caused very big impact, even make crane have the probability toppled.It addition, the work of crane also can be interfered by the external environments such as wind, this has also greatly threatened the safety that crane runs.

Safety problem for crane; people further investigate; crane is mounted with various protection device; as prevented the load limiter of overload crane; prevent the limiter of moment that load moment is excessive; the restriction rising extreme position limiter of boom position, falling-threshold position limiter and operational limit position limiter etc., but these devices all can not solve above-mentioned problem.Generally, operator by the vision of oneself, sensation or with assist personnel exchange the ambient conditions understanding around crane, crane and barrier can not be analyzed exactly, and barrier itself also has uncertainty and randomness, is only difficult to avoid that collision accident with existing device and manpower.

Therefore, it is necessary to design a kind of brand-new crane barrier monitoring and method for early warning and system.

Summary of the invention

The technical problem to be solved is to provide a kind of crane barrier monitoring based on binocular vision and method for early warning and system, it is somebody's turn to do the crane barrier monitoring based on binocular vision and method for early warning and system adopts binocular vision system and image processing techniques, barrier is judged and identifies, detection based on barrier is implemented to report to the police, and safety is high.

The technical solution of invention is as follows:

A kind of crane barrier based on binocular vision is monitored and method for early warning, crane is provided with run-in index binocular vision system, photographic head in run-in index binocular vision system straight down, and is fixed on the top of crane, for the image of Real-time Collection depression angle;Comprise the following steps:

Step 1: build binocular vision model;

Step 2: real-time image acquisition, and the image gathered is carried out pretreatment;

Step 3: detect the object edge in pretreated image, extracts characteristics of image;

Selection opertor carries out rim detection, extracts the profile information of target;Polygon approach is carried out, using polygonal summit as characteristic point by the curve that edge point set is constituted;

Step 4: complete left and right image registration;

According to this characteristic of Edge Feature Points ordered arrangement, adopt the Eigenvector that characteristic point is sequentially connected, calculating the total curvature of its line segment, matching characteristic line segment, the Eigenvector after coupling being launched into characteristic point form, thus realizing the coupling of characteristic point;

Step 5: by iterative operation, distinguish the prospect in image and background;

The motion feature different with background according to prospect, so that prospect and background show this feature of different position relationships in consecutive frame image, by the method iterated, distinguishes foreground and background;

Step 6: based on described binocular vision model, rebuilds three-dimensional information, identifies prospect and barrier;

Step 7: ask for kinematic parameter, analyzes and exports early warning information, and return to step 2.

Binocular vision model is characterized by following formula:

For space any point A, its absolute coordinate is (X, Y, Z), the respectively (x of the three-dimensional coordinate in left and right video camera_L, y_L, z_L)、(x_R, y_R, z_R), the respectively (u of the coordinate in left and right imaging plane_L, v_L)、(u_R, v_R), the respectively (U of the coordinate in pixel planes_L, V_L)、(U_R, V_R), and the parameter of two video cameras is identical, has:

$\mathrm{\μ}\left[\begin{array}{c}{U}_L\\ V_L\\ 1\end{array}\right]=\left[\begin{array}{ccc}1/d_x& s^{\′}& U_0\\ 0& 1/d_y& V_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& t\\ 0_T& 1\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right];$

${\mathrm{\μ}}^{\′}\left[\begin{array}{c}{U}_R\\ V_R\\ 1\end{array}\right]=\left[\begin{array}{ccc}1/d_x& s^{\′}& U_0\\ 0& 1/d_y& V_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}{R}^{\′}& t^{\′}\\ 0_T& 1\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right];$

Wherein, μ, μ ' are 2 factors, have μ=z_L, μ '=z_R, d_xAnd d_yRespectively each pixel physical size on imaging plane coordinate system u axle and v direction of principal axis, (U₀, V₀) for the initial point (midpoint) in image imager coordinate face corresponding to the coordinate of the point on pixel coordinate face, s ' represents because of the obliquity factor of the video camera imaging mutually mutually non-orthogonal extraction of plane coordinates axle;[Rt] is video camera external parameter, and f is the focal length of video camera, 0^T=(0,0,0).(adquisitiones of these parameters is the demarcation by video camera, and camera marking method has a variety of)

Described pretreatment includes gray processing and filtering, and reduces the luminance difference of left and right image pair, strengthens edge and the details of image;[being prior art]

Following formula is adopted to carry out gray processing, I (i, j)=[ω_rR (i, j)+ω_gG (i, j)+ω_bB (i, j)]/3,

R (i, j), G (i, j) and B (i, j) respectively pixel (i, j) corresponding red, green and blue colouring component;Wherein weights ω_r=0.30, ω_g=0.59, ω_r=0.11;

Described is filtered into medium filtering: first determine the window W of a 3X3 pixel, after each pixel is queued up by gray scale size in window, replaces former f (i with the gray value of intermediate position, j) gray value, become window center gray value g (i, j), namely

G (i, j)=Med{f (x-k, y-l), (k, l ∈ W) },

F (x-k, y-l) is the gray value of pixel each in window W.

In step 4, the identification based on weight carries out registration.

In steps of 5, described prospect includes crane self key operation part and hangs thing, and described crane self key operation part includes suspension hook, weight and lifting rope；Background refers to other objects except prospect in visual field；

The method of differentiation prospect and background:

Respectively the left and right image of several two field pictures adjacent after doing rotation transformation is done poor shadow, if the poor shadow of certain scenery is less than the threshold value set, then it is assumed that be prospect, otherwise it is assumed that be background, the position of record prospect and background respectively.

In step 6, calculate the absolute-coordinate three-dimensional of characteristic point based on binocular vision model, then approach the surface of target with three-dimensional grid, so that it is determined that each object dimensional information；Three-dimensional information includes position and shape information；

The recognition methods of barrier:

With the horizontal plane of the minimum point B of current time weight excessively for interfering picture；

With the public view field region of binocular camera shooting system in the subregion of the interference side of drawing for interference region；

Interfering the subregion below picture for noninterference region with the public view field region of binocular camera shooting system；

If suspicion thing is positioned partially or entirely in interference region, then this suspicion thing is barrier.

In step 7,

The moving situation of prospect is obtained by crane self；The break the barriers change of position of the kinematic parameter of barrier is obtained；

According to the dependent thresholds set, it is determined that advanced warning grade, analyze the safe condition of crane, it was predicted that collision situation, by final early warning information output, start under dangerous situation and report to the police.

A kind of crane barrier based on binocular vision is monitored and early warning system, crane is provided with run-in index binocular vision system, photographic head in run-in index binocular vision system straight down, and is fixed on the top of crane, for the image of Real-time Collection depression angle；Adopt the aforesaid crane barrier based on binocular vision to monitor and method for early warning implementation barrier thing is monitored and early warning.

Beneficial effect:

The crane barrier based on binocular vision of the present invention is monitored and method for early warning and system, based on adopting binocular vision system, monitor the working environment of crane in real time, according to principle of parallax, rebuild scene, accurately identify out the dynamic barrier under dynamic background and be accurately positioned, the kinestate of acquired disturbance thing, analyze the safety of crane, provide early warning information to operator in real time, improve the gentle safety in operation of Automated water of crane system.The method efficiently solves a difficult problem for the pre-anticollision of crane, has efficiency height, precision is suitable, system structure is simple, low cost and other advantages, is very suitable for manufacturing on-the-spot online, noncontact Product checking and quality control.

The present invention adopts run-in index binocular vision system, overlook from two parallel angle in real time, gather the ambient image of crane working site, be available for operator's real-time monitored, simultaneously by the process of the image to synchronization diverse location, according to principle of parallax, recover the three-dimensional on-the-spot of three-dimensional from two two-dimentional flat picture shapes, accurately identify and barrier around positioning crane, and determine its movable information such as speed, acceleration, feed back to operator in time, it is prevented that the generation of collision accident.

The method is based on computer binocular vision technology and sets up, and the image information of acquisition is intuitively accurate, it is possible to analyze the potential potential safety hazard at scene qualitatively.By the process of the image information to different angles and analysis, it is accurately determined and hangs the position of thing, weight and barrier etc., shape and motion feature, understand the situation of crane working site exactly, take effectively to prevent pre-measure in time, it is to avoid security incident.The method and system have efficiency height, precision is suitable, system structure is simple, low cost and other advantages, are very suitable for manufacturing on-the-spot online, noncontact Product checking and quality control.

Accompanying drawing explanation

Fig. 1 is crane barrier provided by the invention monitoring and method for early warning flow chart.

Fig. 2 is the coordinate model figure of binocular vision system provided by the invention.

Fig. 3 is identification weight schematic diagram provided by the invention.

Fig. 4 is the computation model of total curvature provided by the invention.

The illustraton of model of Fig. 5 cognitive disorders thing provided by the invention.

Detailed description of the invention

Below with reference to the drawings and specific embodiments, the present invention is described in further details:

Embodiment 1:

Fig. 1 is the flow chart of crane barrier provided by the invention monitoring and method for early warning.

Crane barrier based on binocular vision provided by the invention is monitored and method for early warning, including following step:

Step 101: build binocular vision model, initializes relevant parameter.For the status information of accurate description, analysis space object, binocular vision coordinate model need to be set up, mainly have world coordinate system, camera coordinate system, imaging coordinate system and pixel coordinate system;Simultaneously, reply related data initializes, namely obtain the inner parameter of video camera by camera calibration, pass through to measure or preset acquisition baseline distance, initialize world coordinate system by other devices (alignment system such as bridge crane), so that it is determined that the transformational relation between each coordinate system.

Step 102: real-time image acquisition, and carry out pretreatment.Directly over the arm of crane, gather crane job visual field present image along two parallel directions vertically downward, be designated as left image, right image, by raw image storage;Meanwhile, the image obtained is done pretreatment, mainly gray processing and filtering, reduces the distortion of effect of noise and image, reduces the luminance difference of left and right image pair, strengthen edge and the details of image.

Step 103: detection object edge, extracts characteristics of image.Select suitable operator to carry out rim detection, extract the profile information of target;Polygon approach is carried out, using polygonal summit as characteristic point by the curve that edge point set is constituted.

Step 104: complete left and right image registration, splits public territory.According to this characteristic of Edge Feature Points ordered arrangement, adopt the Eigenvector that characteristic point is sequentially connected, calculate the total curvature of its line segment, matching characteristic line segment, by the Eigenvector after coupling is launched into characteristic point form according to certain rule, thus realizing the coupling of characteristic point.While coupling, with weight for important references, and note the coordinate relation between match point according to principle of parallax.

Step 105: iterate, distinguishes foreground and background.The motion feature different with background according to prospect, so that prospect and background show this feature of different position relationships in consecutive frame image, is selected the method iterated, is distinguished foreground and background.

Step 106: rebuild three-dimensional information, identifies prospect and barrier.According to the camera coordinates model set up, obtain the absolute coordinate space of the point of each Feature point correspondence, approach the surface of target with three-dimensional grid;According to the prospect that step 5 is distinguished, it is thus achieved that the position of prospect and shape;Relative position according to suspicion thing Yu interference surface, identifies the barrier in background, and records its information such as shape and position.

Step 107: ask for kinematic parameter, analyzes and exports early warning information.Calculate the relative distance between each barrier and prospect;By the process to continuous print multiple image, the change according to Obstacle Position, ask for the kinematic parameter of each barrier;Relative distance according to weight and barrier and movement velocity, acceleration etc., carry out prediction of collision;By relevant early warning information output, and report to the police.

Fig. 2 is the specific embodiment that the present invention proposes, and in an embodiment, the present invention installs run-in index binocular vision system on crane, and this system photographic head vertically downward, and is fixed on the top of crane, Real-time Collection overhead view image.Fig. 2 illustrates the coordinate model of the run-in index binocular vision system in the present invention.

The absolute coordinate of 1 A in hypothesis space is (X, Y, Z), the respectively (x of the three-dimensional coordinate in left and right video camera_L, y_L, z_L)、(x_R, y_R, z_R), the respectively (u of the coordinate in left and right imaging plane_L, v_L)、(u_R, v_R), the respectively (U of the coordinate in pixel planes_L, V_L)、(U_R, V_R), and assume that the parameter of two video cameras is identical.

The transformational relation of left image pixel coordinates system (208) and left imaging plane coordinate system (204) is:

$\left[\begin{array}{c}{U}_L\\ V_L\\ 1\end{array}\right]=\left[\begin{array}{ccc}1/d_x& s^{\′}& U_0\\ 0& 1/d_y& V_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{u}_L\\ v_L\\ 1\end{array}\right]=M\left[\begin{array}{c}{u}_L\\ v_L\\ 1\end{array}\right]$ ... ... 1.,

The transformational relation of right image pixel coordinates system (209) and right imaging plane coordinate system (205) is:

$\left[\begin{array}{c}{U}_R\\ V_R\\ 1\end{array}\right]=\left[\begin{array}{ccc}1/d_x& s^{\′}& U_0\\ 0& 1/d_y& V_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{u}_R\\ v_R\\ 1\end{array}\right]=M\left[\begin{array}{c}{u}_R\\ v_R\\ 1\end{array}\right]$ ... ... 2.,

1., 2. in formula, d_xAnd d_yRespectively each pixel physical size on imaging plane coordinate system u axle and v direction of principal axis, (U₀, V₀) for the initial point (midpoint) in image imager coordinate face corresponding to the coordinate of the point on pixel coordinate face, s ' represents because of the obliquity factor of the video camera imaging mutually mutually non-orthogonal extraction of plane coordinates axle.

The transformational relation of left camera coordinate system (206) and left imaging plane coordinate system (204) is:

$\mathrm{\μ}\left[\begin{array}{c}{u}_L\\ v_L\\ 1\end{array}\right]=\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{x}_L\\ y_L\\ z_L\\ 1\end{array}\right]=N\left[\begin{array}{c}{x}_L\\ y_L\\ z_L\\ 1\end{array}\right]$ ... ... 3.,

The transformational relation of right camera coordinate system (207) and right imaging plane coordinate system (205) is:

${\mathrm{\μ}}^{\′}\left[\begin{array}{c}{u}_R\\ v_R\\ 1\end{array}\right]=\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{x}_R\\ y_R\\ z_R\\ 1\end{array}\right]=N\left[\begin{array}{c}{x}_R\\ y_R\\ z_R\\ 1\end{array}\right]$ ... ... 4.,

3., 4. in, f is the focal length of left and right video camera, and μ, μ ' are invariant.

$\left[\begin{array}{c}{x}_L\\ y_L\\ z_L\\ 1\end{array}\right]=\left[\begin{array}{cc}R& t\\ 0^T& 1\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]$ ... ... 5.,

In 5. formula, spin matrix R and translation vector t can be obtained by camera calibration, and 0^T=(0,0,0).

The transformational relation of right camera coordinate system (207) and world coordinate system (201) is:

$\left[\begin{array}{c}{x}_R\\ y_R\\ z_R\\ 1\end{array}\right]=\left[\begin{array}{cc}{R}^{\′}& t^{\′}\\ 0^T& 1\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]$ ... ... 6.,

In 6. formula, spin matrix R ' and translation vector t ' can be obtained by camera calibration, and 0^T=(0,0,0).

Obtained by 1., 3. and 5. formula,

$\begin{array}{c}\mathrm{\μ}\left[\begin{array}{c}{U}_L\\ V_L\\ 1\end{array}\right]=\left[\begin{array}{ccc}{1/d}_x& s^{\′}& U_0\\ 0& {1/d}_y& V_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& t\\ 0_T& 1\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]\\ =\left[\begin{array}{ccc}\mathrm{\α}& s& U_0\\ 0& \mathrm{\β}& V_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cc}R& t\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]\\ =K\·\left[\begin{array}{cc}R& t\end{array}\right]\·T=S\·T\end{array}=$ ... ... 7.,

Wherein, α=f/d_x, β=f/d_y, s=s ' f, $K=\left[\begin{array}{ccc}\mathrm{\α}& s& U_0\\ 0& \mathrm{\β}& V_0\\ 0& 0& 1\end{array}\right],$ S=K [Rt],

T=(X, Y, Z, 1)^T.K is camera intrinsic parameter matrix, and only relevant with video camera internal structure, wherein α, β respectively left video camera is at U_LAxle and V_LScale factor on axle, s is the parameter being described as picture and image coordinate axle inclined degree;[Rt] is video camera external parameter, is determined relative to the position of world coordinate system by left video camera completely;S is called projection matrix, and namely world coordinates is tied to the transition matrix of image coordinate system.Intrinsic parameters of the camera matrix K can be passed through camera calibration is obtained, and video camera external parameter matrix [Rt] can be determined according to the length of crane arm, inclination angle and moving situation.

In like manner, also have:

$\begin{array}{c}{\mathrm{\μ}}^{\′}\mathrm{\μ}\left[\begin{array}{c}{U}_R\\ V_R\\ 1\end{array}\right]=\left[\begin{array}{ccc}{1/d}_x& s^{\′}& U_0\\ 0& {1/d}_y& V_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}{R}^{\′}& t^{\′}\\ 0_T& 1\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]\\ =\left[\begin{array}{ccc}\mathrm{\α}& s& U_0\\ 0& \mathrm{\β}& V_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cc}{R}^{\′}& t^{\′}\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]\\ =K\·\left[\begin{array}{cc}{R}^{\′}& t^{\′}\end{array}\right]\·T=S^{\′}\·T\end{array}=$ ... ... 8.,

In this part, give real-time image acquisition the method doing pretreatment.

In the present invention, with the parallel visual system real-time image acquisition of binocular, and the original image that will gather preserves.

To the current frame image obtained, first should by its gray processing, the preferred weighted mean method of the present invention, namely

I (i, j)=[ω_rR (i, j)+ω_gG (i, j)+ω_bB (i, j)]/3,

Wherein weights ω_r=0.30, ω_g=0.59, ω_rWhen=0.11, the most rational gray level image can be obtained.

The image quality of image can be produced large effect owing to the environment at crane job scene has a lot of uncertainty, noise and light.Therefore must select suitable wave filter, noise jamming can be efficiently reduced, the positional information of each scenery in image can be kept again to greatest extent not to be damaged, be the key issue to solve of pretreatment.

Passing through in the theory of various filtering methods and the basis of experimentation, the preferred median filter of the present invention reduces noise jamming.Medium filtering is the nonlinear smoothing filter processing method of a kind of modal removal noise, can retain again the edge details of image while removing impulsive noise, salt-pepper noise.

The algorithm principle of medium filtering is, it is first determined the window W of an odd pixel, after in window, each pixel is queued up by gray scale size, (i, j) gray value, become the gray value g (i of window center to replace former f with the gray value of intermediate position, j), namely

G (i, j)=Med{f (x-k, y-l), (k, l ∈ W) },

W is the size (might as well be taken as 3X3) of selected window, the grey scale pixel value that f (x-k, y-l) is window W.Mainly comprising the following steps of medium filtering:

1) template is roamed in the drawings, and template is overlapped with certain location of pixels in figure;

2) gray value of each respective pixel under template is read;

3) grey scale pixel value corresponding for template is sorted from small to large;

4) gray value of 1 pixel coming centre in grey-level sequence is chosen;

5) this intermediate value is assigned to the pixel gray value as pixel of corresponding templates center.

This part gives detection object edge, the method extracting characteristics of image:

(1) detection of object edge

In the present invention, it is preferable that adopt the optimum operator Canny operator of rim detection, detect the edge of target.It is implemented as follows:

Use 2D signal f^(k)(x, y) represents the gray value of image, note

$(G_x,G_y)=(\frac{{\∂f}^{\left(k\right)}(x,y)}{\∂x},\frac{{\∂f}^{\left(k\right)}(x,y)}{\∂y}).$

Definition:

$d^{\left(k\right)}(x,y)=\sqrt{G_x^2+G_y^2}$

For f^(k)(x, gradient (filter window is of a size of 3*3) y)；

$w^{\left(k\right)}(x,y)=\exp (-\frac{{\left(d^{\left(k\right)}(x,y)\right)}^2}{{2h}^2})$

For the reflection successional weight coefficient of image picture elements gray value, wherein h is constant parameter, is called that edge retains range coefficient.

The process that realizes of self-adaptive smooth filtering is:

1) making k=1, iterations is N, and arranges edge reservation range coefficient h；

2) Difference Calculation gradient is utilized(x, y) and(x, y):

$G_x^{\left(k\right)}(x,y)=\frac{1}{2}[f^{\left(k\right)}(x+1,y)-f^{\left(k\right)}(x-1,y)],$

$G_y^{\left(k\right)}(x,y)=\frac{1}{2}[f^{\left(k\right)}(x,y+1)-f^{\left(k\right)}(x,y-1)];$

3) the weight coefficient w of wave filter is calculated^(k)(x, y):

$w^{\left(k\right)}(x,y)=\exp \{-\frac{{\left[G_x^{\left(k\right)}(x,y)\right]}^2+{\left[G_x^{\left(k\right)}(x,y)\right]}^2}{{2h}^2}\};$

4) filtering, to f^(k)(x, y) is weighted on average:

$f^{(k+1)}(x,y)=\frac{1}{N}\underset{i=-1}{\overset{1}{\mathrm{\Σ}}}\underset{j=-1}{\overset{1}{\mathrm{\Σ}}}{f}^{\left(k\right)}(x+i,y+j)w^{\left(k\right)}(x+i,y+j),$

Wherein, $N=\underset{i=-1}{\overset{1}{\mathrm{\Σ}}}\underset{j=-1}{\overset{1}{\mathrm{\Σ}}}{w}^{\left(k\right)}(x+i,y+j).$

5) judge that whether k is equal to N, if k=N, terminates filtering, otherwise k=k+1, returns 2).

After successive ignition, the output image of wave filter is made up of some uniform strength regions, and asking of these regions exists good edge.

(2) extraction of target characteristic point

After edge extracting, it is possible to obtaining a series of edge point set orderly, that represent target integrity profile, these point sets describe the shape facility of object edge.If using this point set as identifying clarification of objective point, then the amount of calculation of recognizer is very huge.In order to reach the purpose of compression point set, in the present invention, carry out polygon approach by the curve that edge point set is constituted, using polygonal summit as characteristic point.

By each edge point set P={p_i(x_i, y_i) | i=1,2 ..., first S (x in N}_s, y_s) and last point E (x_e, y_e) as two end points of a virtual line segment, calculate this point and concentrate other p_jDistance to this line segment

$d_k=\frac{|x_j(y_s-y_e)+y_j(x_s-x_e)+y_e{x}_s-y_s{x}_e|}{\sqrt{{(x_s-x_e)}^2+{(y_s-y_e)}^2}},j=\mathrm{2,3},...,N-1---\left(a\right),$

In formula, N is the marginal point number that dummy line is intersegmental.Note

d_max=max (d_j), j=2,3 ..., N-1 (b).

Corresponding point is designated as M (x_max, y_max).If d_max< d_th(threshold values), then be decided to be polygonal one by this virtual line segment;Otherwise, a M is set to end points, i.e. (an x of virtual line segment_e, y_e)←(x_max, y_max), another end points remains unchanged, and utilizes formula (a) to calculate the some distance d to virtual line segment of the marginal point concentration put between S and some E_k, then judge d again_maxAnd d_thMagnitude relationship, if d_max< d_th, then two end points that the 1st S is one article of limit of polygon of newly-generated M and marginal point concentration.Using the M starting point as virtual line segment, i.e. (x_s, y_s)←(x_max, y_max), using marginal point concentrate last 1 as the terminal of virtual line segment, proceed said process;Otherwise, newly-generated some M is as the terminal of virtual line segment, and starting point remains unchanged, and utilizes formula (b) to calculate d_max, until d_max< d_thTill.

One edge point set, after above-mentioned process, can obtain the ordered set Q of the polygon form point after matching, and these points can serve as clarification of objective point.

This part gives the method for left images registration:

(1) characteristic analysis of the image that this biocular systems collects

In the present invention, the left and right image that this binocular vision system collects should have following characteristics:

1) according to principle of parallax, under camera coordinates, only there is a difference in space certain point, the coordinate in x, z-axis equal in the y-axis direction.Therefore any one matching double points in the pixel faces of left and right, it should meet: U_L=U_R、V_L-V_R=l, l are relevant with the depth of field of this spatial point.Therefore, if this feature can be considered, the accurateness of Feature Points Matching can be effectively improved.

2) information such as crane weight shape determine that, known, know according to priori, as viewed from surface, weight is circular, therefore the projection of shape in left and right image of weight should be circular (or connecing rotund ellipse), and weight should by some region paracentral.

Therefore, if the weight that can first identify according to shape facility in image, and using the weight in left images as the important references mated, in conjunction with feature 1), then can be effectively improved the accurateness of coupling.

(2) weight is identified

As it is shown on figure 3, the boundary line (301) to a certain closing, for judging that whether it is the contour line of weight, this polygon geometric properties need to be studied, i.e. girth, mean pole distance and area.

At this, first polygon is made normalized (302).If this polygonal feature point set is Q={q_i(x_i, y_i) | i=1,2 ..., n}, the centre of form coordinate of point set Q is, then

$\stackrel{\&OverBar;}{x}=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{x}_i,\stackrel{\&OverBar;}{y}=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{y}_i---\left(c\right)$

Definition normalization factor

$D=\max |q_i-C|=\max \left(\sqrt{{(x_i-\stackrel{\&OverBar;}{x})}^2+{(y_i-\stackrel{\&OverBar;}{y})}^2}\right)---\left(d\right)$

Characteristic point q after normalization_i′(x_i', y_i') with normalization before characteristic point q_i(x_i, y_i) corresponding relation be

${x_i}^{\&prime;}=(x_i-\stackrel{\&OverBar;}{x})/D,{y_i}^{\&prime;}=(y_i-\stackrel{\&OverBar;}{y})/D,i=\mathrm{1,2},...,n---\left(e\right)$

After normalization, ask for the polygonal girth of normalization, mean pole distance and area:

The polygonal girth of normalization is defined as

$L=\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{l}_i=\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}\sqrt{{({x_i}^{\&prime;}-{x_{i+1}}^{\&prime;})}^2+{({y_i}^{\&prime;}-{y_{i+1}}^{\&prime;})}^2}---\left(f\right)$

Wherein l_iFor characteristic point q_i′(x_i', y_i') and q_i+1′(x_i+1', y_i+1') between Euclidean distance, take i+1=1 as i=M.

The polygonal mean pole distance of normalization is

$\stackrel{\&OverBar;}{d}=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{d}_i=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}\sqrt{{x_i}^{\&prime;2}+{y_i}^{\&prime;2}}---\left(g\right)$

Wherein, d_iFor characteristic point q_i′(x_i', y_i') to the distance between centre of form C ' (0,0).

The polygonal area of normalization

$S=\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{\mathrm{\&Delta;S}}_i=\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}\sqrt{s(s-d_i)(s-d_{i+1})(s-l_i)}---\left(h\right)$

Wherein s=(d_i+d_i+1+l_i)/2, Δ S_iIt is by characteristic point q_i' and q_i+1' the area of triangle that constitutes with centre of form C ', takes i+1=1 as i=M.

If objective contour is a standard round, then should be a unit circle after normalization.Using the ratio of the target geometric properties after normalization and the geometric properties of its unit circle (radius r=1) as characteristic parameter, it may be assumed that

Girth compares: c₁=L/ (2 π r)=L/ (2 π),

Radius ratio: $c_2=\stackrel{\&OverBar;}{d}/r=\stackrel{\&OverBar;}{d},$

Area ratio: c₃=S/ (π r²)=S/ π.

If according to the c that above formula is obtained₁、c₂And c₃Meet c simultaneously₁> T₁、c₂> T₂、c₃> T₃(wherein T₁、T₂And T₃Respectively c₁、c₂And c₃Threshold values), then target is circular, it is believed that this target is weight.

(3) characteristic matching

In the present invention, according to this characteristic of Edge Feature Points ordered arrangement, the Eigenvector that characteristic point is sequentially connected is adopted, calculate the total curvature of its line segment, matching characteristic line segment, by being launched into characteristic point form to the Eigenvector after coupling according to certain rule, thus realizing the coupling of characteristic point.

Left and right two width images are never from angle shot, and same target exists very big-difference on geometric properties, but there is radiation transformation relationship between the target image of different angles, and total curvature of line segment is similar invariants.Fig. 4 gives the computation model of total curvature.

Wherein, l₁It is a p₁And p₂Between distance, l₂It is a p₂And p₃Between distance, l₃It is a p₃And p₄Between distance, l₁₃It is a p₁And p₃Between distance, l₂₄It is a p₂And p₄Between distance,It is line segment p₁p₂And p_xp₃Between angle,It is line segment p₂p₃And p₃p₄Between angle.

According to the cosine law, Ying You

${\cos \&PartialD;}_1=\frac{l_1^2+l_2^2-l_{13}^2}{2l_1{l}_2},{\&PartialD;}_1=\arccos \frac{l_1^2+l_2^2-l_{13}^2}{2l_1{l}_2},$

Then put p₁The curvature at place is

$c_1=\frac{{\&PartialD;}_1}{l_1+l_2},$

In like manner can obtain a p₂The curvature c at place₂。

Therefore, some p₁With a p₂Between total curvature be

C₁₂=l₂*|c₁-c₂|。

The specific algorithm of matching characteristic line segment is as follows:

1) calculate characteristic point to be matched in left and right two figure and be sequentially connected with total curvature of the Eigenvector obtained.

2) in the set of the total curvature of the line segment from left figure, value compares with all of total curvature in right figure successively, if the absolute value of both differences is in a given scope, is considered as the two Eigenvector and is mutually matched.

3) repeat 2) work, obtain whole matching result.

4) circular order that the Eigenvector main manifestations in Statistic analysis result goes out, removes the characteristic matching result not meeting this circulation.

5) result removing error hiding later is launched into characteristic point form according to certain rule, it is possible to obtain the matching result of characteristic point.

This part gives the method distinguishing prospect and background by iterating.

In the present invention, described prospect is primarily referred to as crane self key operation part (suspension hook, weight, lifting rope etc.) and hangs thing, and described background is primarily referred to as other objects in visual field.

In the present invention, described prospect and background have following characteristics:

Cycle owing to gathering image is shorter, and crane is when operation, and for camera coordinate system, the motion of prospect is very slow, and position in the picture is basically unchanged;The motion of background is relatively obvious, and the change in location in the image that consecutive frame gathers is obvious.

Based on characteristic features described above, the present invention a kind of preferably method iterated distinguishes the prospect in visual field and background.Owing to target location is not different in the same time, therefore before the iteration, with current frame image for benchmark, according to parameters such as crane brachium and inclination angles, former two field pictures are done rotation transformation.Concrete iterative process is: respectively the left and right image of several two field pictures adjacent after doing rotation transformation is done poor shadow, if the poor shadow of certain scenery is less than the threshold values set, then it is assumed that be prospect, otherwise it is assumed that be background, and record position respectively.

This part gives the method rebuilding crane visual field three-dimensional information cognitive disorders thing.

(1) three-dimensional coordinate of each object is calculated

According to the camera coordinates model set up, using left and right pixel image as it is known that obtain the absolute coordinate of point corresponding to each feature point pairs.Implement process as follows:

Assuming that A (X, Y, Z) is for the arbitrary characteristics point in space, it is followed successively by the coordinate of left and right imaging plane and pixel planes: (u_L, v_L)、(u_R, v_R)、(U_L, V_L) and (U_R, V_R)。

By 7. formula and 8. formula obtain:

μ(U_L, V_L, 1)^T=K [Rt] (X, Y, Z, 1)^T¨ ¨ ¨ ¨ ¨ ¨ 9.,

μ′(U_R, V_R, 1)^T=K' t ' " (X, Y, Z, 1)^T¨ ¨ ¨ ¨ ¨ ¨ 10.,

9. and 10. simultaneous formula can uniquely determine an A (X, Y, Z).

By methods described above, it is possible to obtain the absolute-coordinate three-dimensional of arbitrary characteristics point, then approach the surface of target with three-dimensional grid, thus the three-dimensional informations such as the position of each object, shape are also just uniquely determined.

(2) identification of barrier

For prospect, specific embodiment 6 provides recognition methods, directly according to methods described above, asked for its three-dimensional coordinate.For background, then will according to each object location, it is judged that whether it barrier.

Identification to barrier, it is necessary to first determine interference surface, thus delimit interference region and noninterference region, then judge this suspicion thing whether barrier according to the relative position of suspicion thing Yu interference surface.

The recognition methods of barrier is described below in conjunction with Fig. 5.

Fig. 5 is the areal model figure of cognitive disorders thing provided by the invention.In Fig. 5, B point is the minimum point of current time weight, H₀Vertical distance for B point distance video camera photocentre, region 501 is the public view field region (i.e. region 218 in Fig. 2) of binocular camera shooting system, plane 502 was horizontal plane and the interference surface of B point, region 503 is the part and the interference region that are positioned at interference surface (502) top in public view field region (501), region 504 is the part and the noninterference region that are positioned at interference surface (502) lower section in public view field region (501), and suspicion thing 505,506 is the object that shape, color, speed etc. are unknown.

Wherein, suspicion thing 505 some be positioned at interference region (502), it is possible to the safe operation of crane being threatened, therefore this suspicion thing should be considered barrier;Suspicion thing 506 is fully located in noninterference region (503), and temporarily without influence on the safe operation of crane, therefore this suspicion thing should be considered non-barrier.

This part is to asking for kinematic parameter, analysis and exporting early warning information and sat brief elaboration.

On the one hand, the moving situation of prospect can be by what crane self obtained;On the other hand, for the kinematic parameter of barrier, it is possible to the change of the position that breaks the barriers is obtained.It addition, arbitrarily barrier is also observable with the current distance of prospect.Therefore, it can these parameters are sent into specialist system, according to the related threshold set, it is determined that advanced warning grade, analyze the safe condition of crane, it was predicted that collision situation, by final early warning information output, Realtime Alerts under dangerous situation.

The above, be only presently preferred embodiments of the present invention, and the present invention not does any pro forma restriction.All those of ordinary skill in the art, without departing from, under technical solution of the present invention ambit, may be by the method for the disclosure above and technology contents and technical solution of the present invention is made rational improvement, or change the Equivalent embodiments of equivalent variations into.Therefore, every content without departing from technical solution of the present invention, the technical spirit of the foundation present invention, to any simple modification made for any of the above embodiments, equivalent variations and modification, all still falls within the scope of the technology of the present invention protection.

Claims (6)

1. the crane barrier based on binocular vision is monitored and method for early warning, it is characterized in that, being provided with run-in index binocular vision system on crane, the photographic head in run-in index binocular vision system is straight down, and it is fixed on the top of crane, for the image of Real-time Collection depression angle；Comprise the following steps:

Step 1: build binocular vision model；

Step 2: real-time image acquisition, and the image gathered is carried out pretreatment；

Step 3: detect the object edge in pretreated image, extracts characteristics of image；

Selection opertor carries out rim detection, extracts the profile information of target；Polygon approach is carried out, using polygonal summit as characteristic point by the curve that edge point set is constituted；

Step 4: complete left and right image registration；

According to this characteristic of Edge Feature Points ordered arrangement, adopt the Eigenvector that characteristic point is sequentially connected, calculating the total curvature of its line segment, matching characteristic line segment, the Eigenvector after coupling being launched into characteristic point form, thus realizing the coupling of characteristic point；

Step 5: by iterative operation, distinguish the prospect in image and background；

The motion feature different with background according to prospect, so that prospect and background show this feature of different position relationships in consecutive frame image, by the method iterated, distinguishes foreground and background；

Step 6: based on described binocular vision model, rebuilds three-dimensional information, identifies prospect and barrier；

Step 7: ask for kinematic parameter, analyzes and exports early warning information, and return to step 2；

In step 6, calculate the absolute-coordinate three-dimensional of characteristic point based on binocular vision model, then approach the surface of target with three-dimensional grid, so that it is determined that each object dimensional information；Three-dimensional information includes position and shape information；

The recognition methods of barrier:

With the horizontal plane of the minimum point B of current time weight excessively for interference surface；

With the public view field region of binocular camera shooting system, the subregion above interference surface is for interference region；

With the public view field region of binocular camera shooting system, the subregion below interference surface is for noninterference region；

If suspicion thing is positioned partially or entirely in interference region, then this suspicion thing is barrier；

In step 7,

The moving situation of prospect is obtained by crane self；The break the barriers change of position of the kinematic parameter of barrier is obtained；

According to the dependent thresholds set, it is determined that advanced warning grade, analyze the safe condition of crane, it was predicted that collision situation, by final early warning information output, start under dangerous situation and report to the police.

2. the crane barrier based on binocular vision according to claim 1 is monitored and method for early warning, it is characterised in that binocular vision model is characterized by following formula:

For space any point A, its absolute coordinate is (X, Y, Z), the respectively (x of the three-dimensional coordinate in left and right video camera_L,y_L,z_L)、(x_R,y_R,z_R), the respectively (u of the coordinate in left and right imaging plane_L,v_L)、(u_R,v_R), the respectively (U of the coordinate in pixel planes_L,V_L)、(U_R,V_R), and the parameter of two video cameras is identical, has:

$\mathrm{\&mu;}\left[\begin{array}{c}{U}_L\\ V_L\\ 1\end{array}\right]=\left[\begin{array}{ccc}1/d_x& s^{\&prime;}& U_0\\ 0& 1/d_y& V_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& t\\ 0^T& 1\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right];$

${\mathrm{\&mu;}}^{\&prime;}\left[\begin{array}{c}{U}_R\\ V_R\\ 1\end{array}\right]=\left[\begin{array}{ccc}1/d_x& s^{\&prime;}& U_0\\ 0& 1/d_y& V_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}{R}^{\&prime;}& t^{\&prime;}\\ 0^T& 1\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right];$

Wherein, and μ, μ ' it is 2 factors, there is μ=z_L, μ '=z_R, d_xAnd d_yRespectively each pixel physical size on imaging plane coordinate system u axle and v direction of principal axis, (U₀,V₀) for the initial point in image imager coordinate face corresponding to the coordinate of the point on pixel coordinate face, s' represents because of the obliquity factor of the video camera imaging mutually mutually non-orthogonal extraction of plane coordinates axle；R and R ' is the spin matrix of left and right video camera, and t and t ' is the translation vector of left and right video camera, and f is the focal length of video camera, 0^T=(0,0,0).

3. the crane barrier based on binocular vision according to claim 2 is monitored and method for early warning, it is characterised in that described pretreatment includes gray processing and filtering, and reduces the luminance difference of left and right image pair, strengthens edge and the details of image；

Following formula is adopted to carry out gray processing, I (i, j)=[ω_rR(i,j)+ω_gG(i,j)+ω_bB (i, j)]/3, R (i, j), G (i, j) and B (i, j) respectively pixel (i, j) corresponding red, green and blue colouring component；

Wherein weights ω_r=0.30, ω_g=0.59, ω_r=0.11；

Described is filtered into medium filtering: first determine the window W of a 3X3 pixel, after each pixel is queued up by gray scale size in window, replaces former f (i with the gray value of intermediate position, j) gray value, become window center gray value g (i, j), namely

G (i, j)=Med{f (i-k, j-l), (k, l ∈ W) },

F (i-k, j-l) is the gray value of pixel each in window W.

4. the crane barrier based on binocular vision according to claim 3 is monitored and method for early warning, it is characterised in that in step 4, the identification based on weight carries out registration.

5. the crane barrier based on binocular vision according to claim 4 is monitored and method for early warning, it is characterized in that, in steps of 5, described prospect includes crane self key operation part and hangs thing, and described crane self key operation part includes suspension hook, weight and lifting rope；Background refers to other objects except prospect in visual field；

The method of differentiation prospect and background:

Respectively the left and right image of several two field pictures adjacent after doing rotation transformation is done poor shadow, if the poor shadow of certain scenery is less than the threshold value set, then it is assumed that be prospect, otherwise it is assumed that be background, the position of record prospect and background respectively.

6. the crane barrier based on binocular vision is monitored and early warning system, it is characterized in that, being provided with run-in index binocular vision system on crane, the photographic head in run-in index binocular vision system is straight down, and it is fixed on the top of crane, for the image of Real-time Collection depression angle；Adopt the crane barrier based on binocular vision described in claim 1 to monitor and method for early warning implementation barrier thing is monitored and early warning.