CN106371453A - Visual servo method for use in the automatic line inspection by multi-rotor unmanned plane - Google Patents

Abstract

The line inspection by multi-rotor unmanned plane can greatly reduce cost, reduce the work intensity of people and resolve the safety problem posed by human line inspection. The navigating and positioning of a conventional multi-rotor unmanned plane mainly employs the combination of IMUs (inertia Measurement Unit) and satellites (GPS, Beidou, and etc.), which has some limitations. The precision of the prior non-differential positioning GPS navigation system is around 2.5 meters. And the inertial navigation system, due to its cumulative integration error, can only serve as a means of auxiliary navigation. As the transmission voltage across the transmission line is high, usually in the tens to hundreds of kilovolts. Such a high voltage level makes a very complex electromagnetic field around the transmission line, in which the existing navigation system, especially the geomagnetic sensor in the inertial navigation is likely to dysfunction due to interference, and ultimately air crash might occur. With the method of the invention, the probability of air crash by current unmanned aerial vehicles due to interference can be greatly reduced.

Description

A kind of visual servo method in the autonomous line walking of many rotor wing unmanned aerial vehicles

Technical field

The invention belongs to the technical field of UAV Flight Control and machine vision mixing together is and in particular to a kind of be based on straightway Many rotor wing unmanned aerial vehicles Visual servoing control method of intersection point azimuth determination.

Background technology

In recent years, many rotor wing unmanned aerial vehicles have obtained quick development, have flight stability, easily manipulation, low noise, safety hazard The features such as property is little, obtains increasingly wider application in multiple fields such as aeroplane photography, electric inspection process, the condition of a disaster investigation, military surveillances.

Using many rotor wing unmanned aerial vehicles line walking, can be good at reduces cost, the amount of lightening one's labor and solve what artificial line walking was brought Safety problem.The navigator fix of traditional many rotor wing unmanned aerial vehicles mainly adopts imu (inertia assembly) and satellite (such as gps, Big Dipper etc.) Compound mode, has certain limitation.The gps navigation system precision of front non-differential positioning about at 2.5 meters, inertial navigation System, due to there being accumulative integral error, is only used as the means of assisting navigation.Because the transmission voltage of transmission line of electricity is higher, typically All arrive hundreds of kilovolt tens, so high electric pressure causes the electromagnetic field environment of transmission line of electricity periphery more complicated, existing Geomagnetic sensor in navigation system, especially inertial navigation is easily interfered and failure, ultimately results in the generation of air crash accident

In order to improve the intelligent and reliability of the autonomous line walking of many rotor wing unmanned aerial vehicles, reduce the workload of manual intervention it is necessary to increase The ability of the autonomous Visual servoing control of many rotor wing unmanned aerial vehicles by force.

Content of the invention

It is an object of the invention to making up the deficiency of the autonomous line walking technology of existing many rotor wing unmanned aerial vehicles, a kind of straightway that is based on is provided to hand over The Visual servoing control method of point azimuth determination, the method is without manual intervention, and need not carry out edge during detection electric wire Detection, strong robustness and being easily achieved, computational efficiency is better than existing algorithm.

For reaching above-mentioned purpose, the present invention employs the following technical solutions and is achieved:

Visual servo method in a kind of autonomous line walking of many rotor wing unmanned aerial vehicles, comprises the following steps:

1) image acquisition: using the monocular camera being mounted in below many rotor wing unmanned aerial vehicles, the region comprising electric wire is acquired, obtains To image i.Wherein camera intrinsic parameter is demarcated, focal length f, x and y direction single pixel physical size d_xAnd d_yIt is known；

2) line segment detection: using the line Segment Detection growing based on gradient, line segment detection is carried out to image i, and will grow Degree is ignored less than the straightway of threshold value, and the straightway detecting is expressed as { l₁, l₂..., l_n}；

3) Straight-line segments mergence: end-point distances and angle are asked for any two in the straightway set detecting, to eligible Straightway merge, obtain straightway set { l₁, l₂..., l_m}；

4) straightway screening: to the straightway set { l after merging treatment₁, l₂..., l_mIn each bar straight line judged, sieve Select qualified two straightway l_iAnd l_j；

5) straightway intersection point solves and azimuthal angle calculation: asks for straightway l_iAnd l_jIntersection point p (x, y), and according to (x, y) Coordinate asks for azimuth angle alpha and the height angle beta that this intersection point is relative to unmanned plane；

6) visual servo scheme: azimuth angle alpha and height angle beta are input in intelligent controller, output control variable, control Rotor wing rotation direction and speed, so that unmanned plane is along electric wire autonomous flight, execute line walking task.

The present invention is further improved by:

Described step 3) in, the concrete grammar of Straight-line segments mergence is as follows:

To { l₁, l₂..., l_nIn any two straightway { l_i, l_j, two end points are respectively (x_i1, y_i1) and (x_i2, y_i2)、 (x_j1, y_j1) and (x_j2, y_j2), it is respectively θ with the angle of x-axis₁And θ₂；Calculate (x_i1, y_i1) and (x_j1, y_j1)、(x_j2, y_j2) Between Euclidean distance d₁₁And d₁₂, (x_i2, y_i2) and (x_j1, y_j1)、(x_j2, y_j2) between Euclidean distance be d₂₁And d₂₂；d_min For d₁₁、d₁₂、d₂₁And d₂₂In minima, obtain l simultaneously_iAnd l_jBetween angle theta₁₂=abs (θ₁-θ₂)；Calculate straight Line l_iTwo end points (x_i1, y_i1) and (x_i2, y_i2) arrive straight line l_jTwo vertical line sections, note intersection point be respectively t₁, t₂, vertical line segment length Degree is respectively d₁₁, d₁₂；Calculate straight line l_jTwo end points (x_j1, y_j1) and (x_j2, y_j2) arrive straight line l_iTwo vertical line sections, remember intersection point It is respectively t₃, t₄, vertical line segment length is respectively d₂₁, d₂₂.Defining criterion one is: d_minLess than 10 pixels；Define criterion two For: θ₁₂Less than 5 degree；Defining criterion three is: b₁||b₂||b₃||b₄, wherein b₁=t₁∈l_j&&d₁₁＜ threshold value t_p, b₂=t₂ ∈l_j&&d₁₂＜ threshold value t_p, b₃=t₃∈l_j&&d₂₁＜ threshold value t_p, b₄=t₄∈l_i&&d₂₂＜ threshold value t_p.

For any two straightway l_iAnd l_j, when there being a satisfaction in criterion one, two, three.Then merging two straightways is one Bar.Straightway set { l is obtained after all straightways are all processed₁, l₂..., l_m}.

Described step 4) in, straightway screening method particularly includes:

Calculate straightway { l₁, l₂..., l_mLength and slope, choose the maximum straightway of length, and obtain and its length difference Value is less than other straightways of threshold value with angle.Choose two of angle maximum as l in the straightway solving_iAnd l_j.

Described step 5) in, straightway intersection point solves and azimuthal angle calculation method particularly includes:

l_iAnd l_jTwo end points be respectively (i_x1, i_y1)、(i_x2, i_y2) and (j_x1, j_y1)、(j_x2, j_y2), l_iAnd l_jIntersection point be p (x, y).By (i_x1, i_y1)、(i_x2, i_y2), (j_x1, j_y1)、(j_x2, j_y2) (ij can be obtained according to the following formula_x, ij_y):

X=

(i_x1*j_x1*i_y2-i_x2*j_x1*i_y1-i_x1*j_x2*i_y2+i_x2*j_x2*i_y1-i_x1*j_x1*j_y2+i_x1*j_x2*j_y1+i_x2*j_x1*j_y2-i_x2*j_x2*j_y1)/(i_x1*j_y1-j_x1*i_y1-i_x1*j_y2-i_x2*j_y1+j_x1*i_y2+j_x2*i_y1+i_x2*j_y2-j_x2*i_y2)

Y=

(i_x1*i_y2*j_y1-i_x2*i_y1*j_y1-i_x1*i_y2*j_y2+i_x2*i_y1*j_y2-j_x1*i_y1*j_y2+j_x2*j_y1*j_y1+j_x1*i_y2*j_y2-j_x2*i_y2*j_y1)/(i_x1*j_y1-j_x1*i_y1-i_x1*j_y2-i_x2*j_y1+j_x1*i_y2+j_x2*i_y1+i_x2*j_y2-j_x2*i_y2)

Then azimuth angle alpha and height angle beta are obtained by p (x, y).

Described step 6) in, visual servo scheme method particularly includes:

The controling power of azimuth and the angle of site is f_α、f_β.Controller is controlled using the control of the pd with dead band, controling power meter Calculation such as following formula:

Wherein, α₀、β₀It is azimuth controlling dead error respectively.It is controlled by azimuthal, the angle of site, can reach inclined to position The control of difference.

Compared with prior art, the method have the advantages that

The present invention using the line Segment Detection based on gradient growth, thus rim detection need not be carried out to image, it is to avoid binaryzation The impact that threshold value is chosen.Method using Straight-line segments mergence, it is to avoid the interference of short and small straightway in small, broken bits, reduces calculating simultaneously Amount.Using the property that the straightway of less parallel line is intersecting at a distance, ask for the azimuth of intersection point, the angle of site as control deviation, Realize the line walking function under the autonomous characteristic condition of circuit.Finally, the present invention can be used for many rotor wing unmanned aerial vehicles to power circuit from In main tour task.

Brief description

Fig. 1 is the application scenarios schematic diagram of the present invention；

Fig. 2 is the breakpoint detection schematic diagram to power circuit for the present invention.

Fig. 3 is the algorithm overall block flow diagram of the present invention；

Specific embodiment

Below in conjunction with the accompanying drawings the present invention is described in further detail:

Fig. 1 is typical application scenarios of the present invention, and many rotor wing unmanned aerial vehicles are autonomous by a point along electric lines of force 1,2,3 rectilinear direction Move to b point.Fig. 2 illustrates the present invention and to the detection of electric lines of force 1,2,3 and solves intersection point schematic diagram.

Referring to Fig. 3, the present invention comprises the following steps:

1st, image acquisition

I is obtained after target image is acquired.Picture size width is m, highly for n.

2nd, line segment detection

Using the line Segment Detection growing based on gradient, image i is carried out with line segment detection, and length is less than the straight line of 0.5 × n Section is ignored, and the straightway detecting is expressed as { l₁, l₂..., l_n}.

Wherein with regard to above-mentioned steps how concrete operations, it is known in the art technology, refers to document 1 " Qin Xunhui, Ma Rong, Fu Wei Flat, etc. a kind of line segment detection algorithm [j] based on gradient. photon journal, 2012,41 (2), 205-209. ", herein no longer Repeat.

3rd, Straight-line segments mergence

To { the l obtaining in previous step₁, l₂..., l_nIn any two straightway { l_i, l_jIt is known that two end points are respectively (x_i1, y_i1) and (x_i2, y_i2)、(x_j1, y_j1) and (x_j2, y_j2), it is respectively θ with the angle of x-axis₁And θ₂；Calculate (x_i1, y_i1) and (x_j1, y_j1)、(x_j2, y_j2) between Euclidean distance d₁₁And d₁₂, (x_i2, y_i2) and (x_j1, y_j1)、(x_j2, y_j2) between Euclidean distance be d₂₁And d₂₂；d_minFor d₁₁、d₁₂、d₂₁And d₂₂In minima, obtain l simultaneously_iAnd l_jBetween angle theta₁₂=abs (θ₁-θ₂)； Calculate straight line l_iTwo end points (x_i1, y_i1) and (x_i2, y_i2) arrive straight line l_jTwo vertical line sections, note intersection point be respectively t₁, t₂, hang down Line segment length is respectively d₁₁, d₁₂；Calculate straight line l_jTwo end points (x_j1, y_j1) and (x_j2, y_j2) arrive straight line l_iTwo vertical line sections, Note intersection point is respectively t₃, t₄, vertical line segment length is respectively d₂₁, d₂₂.Defining criterion one is: d_minLess than 10 pixels；Definition Criterion two is: θ₁₂Less than 5 degree；Defining criterion three is: b₁||b₂||b₃||b₄, wherein b₁=t₁∈l_j&&d₁₁＜ threshold value t_p, b₂=t₂∈l_j&&d₁₂＜ threshold value t_p, b₃=t₃∈l_i&&d₂₁＜ threshold value t_p, b₄=t₄∈l_i&&d₂₂＜ threshold value t_p.

For any two straightway l_iAnd l_j, when there being a satisfaction in criterion one, two, three.Then merge two straightways and be one. Straightway set { l is obtained after all straightways are all processed₁, l₂..., l_m}.

4th, straightway screening:

Calculate straightway { l₁, l₂..., l_mLength and slope, choose the maximum straightway of length, and obtain with its length difference and Angle is less than other straightways of threshold value.Choose two of angle maximum as l in the straightway solving_iAnd l_j.

5th, straightway intersection point solves and azimuthal angle calculation

l_iAnd l_jTwo end points be respectively (i_x1, i_y1)、(i_x2, i_y2) and (j_x1, j_y1)、(j_x2, j_y2), l_iAnd l_jIntersection point is p (x, y). By (i_x1, i_y1)、(i_x2, i_y2), (j_x1, j_y1)、(j_x2, j_y2) (ij can be obtained according to the following formula_x, ij_y):

X=

(i_x1*j_x1*i_y2-i_x2*j_x1*i_y1-i_x1*j_x2*i_y2+i_x2*j_x2*i_y1-i_x1*j_x1*j_y2+i_x1*j_x2*j_y1+i_x2*j_x1*j_y2-i_x2*j_x2*j_y1)/(i_x1*j_y1 -j_x1*i_y1-i_x1*j_y2-i_x2*j_y1+j_x1*i_y2+j_x2*i_y1+i_x2*j_y2-j_x2*i_y2)

Y=

(i_x1*i_y2*j_y1-i_x2*i_y1*j_y1-i_x1*i_y2*j_y2+i_x2*i_y1*j_y2-j_x1*i_y1*j_y2+j_x2*j_y1*j_y1+j_x1*i_y2*j_y2-j_x2*i_y2*j_y1)/(i_x1*j_y1 -j_x1*i_y1-i_x1*j_y2-i_x2*j_y1+j_x1*i_y2+j_x2*i_y1+i_x2*j_y2-j_x2*i_y2)

Then azimuth angle alpha and height angle beta are obtained by p (x, y), acquiring method is known in the art technology, referred to document 2 " once Moral is virtuous, Dong Xurong, Li Rui. spacecraft relative angle measuring method [c]. Chinese intelligent automation meeting, and Lanzhou, 2007 August 24-27 day in year. "

6th, visual servo scheme

Controller is controlled using the pd with dead band.The controling power of azimuth and the angle of site is f_α、f_β.Controling power is calculated as follows formula:

Wherein, α₀、β₀It is azimuth controlling dead error respectively.It is controlled by azimuthal, the angle of site, can reach inclined to position The control of difference.

Above content technological thought only to illustrate the invention, it is impossible to limit protection scope of the present invention with this, every carries according to the present invention The technological thought going out, any change done on the basis of technical scheme, each fall within claims of the present invention protection domain it Interior.

Claims (5)

1. the visual servo method in a kind of autonomous line walking of many rotor wing unmanned aerial vehicles is it is characterised in that comprise the following steps:

1) image acquisition: using the monocular camera being mounted in below many rotor wing unmanned aerial vehicles, the region comprising electric wire is acquired, obtains To image i, wherein camera intrinsic parameter is demarcated, focal length f, x and y direction single pixel physical size d_xAnd d_yIt is known；

2) line segment detection: using the line Segment Detection growing based on gradient, line segment detection is carried out to image i, and will grow Degree is ignored less than the straightway of threshold value, and the straightway detecting is expressed as { l₁, l₂..., l_n}；

3) Straight-line segments mergence: end-point distances and angle are asked for any two in the straightway set detecting, to eligible Straightway merge, obtain straightway set { l₁, l₂..., l_m}；

4) straightway screening: to the straightway set { l after merging treatment₁, l₂..., l_mIn each bar straight line judged, filter out Qualified two straightway l_iAnd l_j；

5) straightway intersection point solves and azimuthal angle calculation: asks for straightway l_iAnd l_jIntersection point p (x, y), and asked according to (x, y) coordinate Take azimuth angle alpha and the height angle beta that this intersection point is relative to unmanned plane；

6) visual servo scheme: azimuth angle alpha and height angle beta are input in intelligent controller, output control variable, control rotation Wing direction of rotation and speed are so that unmanned plane is along electric wire autonomous flight.

2. according to claim 1 based on merge lbp and gray feature description Algorism of Matching Line Segments method it is characterised in that Described step 3) in, the concrete grammar of Straight-line segments mergence is as follows:

To { l₁, l₂..., l_nIn any two straightway { l_i, l_j, two end points are respectively (x_i1, y_i1) and (x_i2, y_i2)、(x_j1, y_j1) (x_j2, y_j2), it is respectively θ with the angle of x-axis₁And θ₂；Calculate (x_i1, y_i1) and (x_j1, y_j1)、(x_j2, y_j2) between Euclidean distance d₁₁ And d₁₂, (x_i2, y_i2) and (x_j1, y_j1)、(x_j2, y_j2) between Euclidean distance be d₂₁And d₂₂；d_minFor d₁₁、d₁₂、d₂₁And d₂₂ In minima, obtain l simultaneously_iAnd l_jBetween angle theta₁₂=abs (θ₁-θ₂)；Calculate straight line l_iTwo end points (x_i1, y_i1) and (x_i2, y_i2) arrive straight line l_jTwo vertical line sections, note intersection point be respectively t₁, t₂, vertical line segment length is respectively d₁₁, d₁₂；Calculate straight line l_j Two end points (x_j1, y_j1) and (x_j2, y_j2) arrive straight line l_iTwo vertical line sections, note intersection point be respectively t₃, t₄, vertical line segment length is respectively d₂₁, d₂₂.Defining criterion one is: d_minLess than 10 pixels；Defining criterion two is: θ₁₂Less than 5 degree；Defining criterion three is: b₁||b₂||b₃||b₄, wherein b₁=t₁∈l_j&&d₁₁＜ threshold value t_p, b₂=t₂∈l_j&&d₁₂＜ threshold value t_p, b₃=t₃∈l_i&&d₂₁＜ Threshold value t_p, b₄=t₄∈l_i&&d₂₂＜ threshold value t_p；For any two straightway l_iAnd l_j, when having one in criterion one, two, three Individual satisfaction, then merge two straightways and be one, obtain straightway set { l after all straightways are all processed₁, l₂..., l_m}.

3. according to claim 1 based on merge lbp and gray feature description Algorism of Matching Line Segments method it is characterised in that Described step 4) in, straightway screening method particularly includes:

Calculate straightway { l₁, l₂..., l_mLength and slope, choose the maximum straightway of length, and obtain with its length difference and Angle is less than other straightways of threshold value, chooses two of angle maximum as l in the straightway solving_iAnd l_j.

4. according to claim 1 based on merge lbp and gray feature description Algorism of Matching Line Segments method it is characterised in that Described step 5) in, straightway intersection point solves and azimuthal angle calculation method particularly includes:

l_iAnd l_jTwo end points be respectively (i_x1, i_y1)、(i_x2, i_y2) and (j_x1, j_y1)、(j_x2, j_y2), l_iAnd l_jIntersection point is p (x, y), by (i_x1, i_y1)、(i_x2, i_y2), (j_x1, j_y1)、(j_x2, j_y2) (ij can be obtained according to the following formula_x, ij_y):

X=

(i_x1*j_x1*i_y2-i_x2*j_x1*i_y1-i_x1*j_x2*i_y2+i_x2*j_x2*i_y1-i_x1*j_x1*j_y2+i_x1*j_x2*j_y1+i_x2*j_x1*j_y2-i_x2*j_x2*j_y1)/(i_x1*j_y1-j_x1 *i_y1-i_x1*j_y2-i_x2*j_y1+j_x1*i_y2+j_x2*i_y1+i_x2*j_y2-j_x2*i_y2)

Y=

(i_x1*i_y2*j_y1-i_x2*i_y1*j_y1-i_x1*i_y2*j_y2+i_x2*i_y1*j_y2-j_x1*i_y1*j_y2+j_x2*i_y1*j_y1+j_x1*i_y2*j_y2-j_x2*i_y2*j_y1)/(i_x1*j_y1-j_x1 *i_y1-i_x1*j_y2-i_x2*j_y1+j_x1*i_y2+j_x2*i_y1+i_x2*j_y2-j_x2*i_y2)

Then azimuth angle alpha and height angle beta are obtained by p (x, y).

5. according to claim 1 based on merge lbp and gray feature description Algorism of Matching Line Segments method it is characterised in that Described step 6) in, visual servo scheme method particularly includes:

The controling power of azimuth and the angle of site is f_α、f_β.Controller is controlled using the control of the pd with dead band, controling power meter Calculation such as following formula:

Wherein, α₀、β₀It is azimuth controlling dead error respectively, is controlled by azimuthal, the angle of site, can reach inclined to position The control of difference.