TWI720510B - Aircraft line-following control method and flight control method using the method - Google Patents
Aircraft line-following control method and flight control method using the method Download PDFInfo
- Publication number
- TWI720510B TWI720510B TW108120559A TW108120559A TWI720510B TW I720510 B TWI720510 B TW I720510B TW 108120559 A TW108120559 A TW 108120559A TW 108120559 A TW108120559 A TW 108120559A TW I720510 B TWI720510 B TW I720510B
- Authority
- TW
- Taiwan
- Prior art keywords
- image
- following
- target
- flight
- center
- Prior art date
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
本發明提供一種飛行器線條跟隨控制方法,主要係先將一影像畫面中央的分割影像進行再分割產生多個再分割影像,根據各該再分割影像出現跟隨目標的機率值決定一飛行方向,並計算飛行方向中跟隨目標的中心點以根據該中心點決定一虛擬飛行座標;而飛行控制方法則係先將影像畫面分割為多個分割影像,根據各分割影像出現跟隨目標的機率值決定一飛行方向區域,若飛行方向區域在畫面中央,則執行上述線條跟隨控制方法,進而產生一飛行指令;本發明透過先判斷跟隨目標所在之飛行方向,再進一步計算精確的飛行座標,達到高精準度的線條跟隨飛行控制。The present invention provides a line-following control method of an aircraft, which is mainly to first subdivide a segmented image in the center of an image frame to generate multiple subdivided images, determine a flight direction according to the probability value of each subdivided image appearing to follow a target, and calculate Follow the center point of the target in the flight direction to determine a virtual flight coordinate based on the center point; while the flight control method first divides the image screen into multiple segmented images, and determines a flying direction according to the probability of each segmented image following the target. If the flight direction area is in the center of the screen, the line following control method is executed to generate a flight instruction; the present invention first determines the flight direction of the following target, and then further calculates the precise flight coordinates to achieve high-precision lines Follow the flight control.
Description
一種飛行器線條跟隨控制方法及運用該方法的飛行控制方法,尤指一種藉由虛擬目標方位網格機率模型以實現高精度線路跟隨的飛行控制方法。 A line-following control method for an aircraft and a flight control method using the method, in particular to a flight control method that implements high-precision line following by using a virtual target azimuth grid probability model.
無人飛行器近年來被廣泛運用於人員難以抵達的空間進行影像擷取或環境勘查等用途,除了由人員手動操作飛行方向之外,令無人飛行器根據特定條件自動執行飛行任務的控制方法能大力節省人力並降低人為判斷失誤。其中,令無人飛行器跟隨一跟隨目標沿線飛行是一種經常使用的飛行方法,例如沿著一河流進行影像擷取及勘察。現有的飛行器線條跟隨控制並不要求良好的沿線精準度,也就是說只要求控制該飛行器沿著線條延伸的大略方向前進即可。舉例來說,現有的控制方法是先擷取目標跟隨線條所在之影像,根據該影像計算該目標跟隨線條的延伸方向,並控制飛行器沿該目標跟隨線條延伸的方向前進。當判斷線條方向改變時,則根據新的延伸方向,控制該飛行器朝向一轉向角進行轉向,以沿著新的延伸方向飛行。然而,根據擷取影像判斷線條延伸方向若因計算誤差與該跟隨目標實際延伸的方向並不完全相符,當飛行器朝向該計算出來的延伸方向飛行時,該飛行器即會偏離該跟隨目標所在位置,即使根據下一次的計算再次校正飛行方向,該飛行器也已經偏離跟隨目標之所在位置,並且因此產生飛行搖擺或蛇行之不穩定情況。如果上述偏離跟隨目標之幅度過大,飛行器則可能完全失去目標。 In recent years, unmanned aerial vehicles have been widely used in difficult-to-reach spaces for image capture or environmental surveys. In addition to manual operation of the flight direction by personnel, the control method that enables unmanned aerial vehicles to automatically perform flight tasks according to specific conditions can greatly save manpower. And reduce human error of judgment. Among them, making the unmanned aerial vehicle fly along a following target is a frequently used flying method, such as image capturing and surveying along a river. The existing line following control of the aircraft does not require good accuracy along the line, that is to say, it is only required to control the aircraft to advance along the approximate direction of the line extension. For example, the existing control method is to first capture the image where the target follows the line, calculate the extension direction of the target follow line based on the image, and control the aircraft to move along the direction in which the target follows the line. When it is judged that the direction of the line changes, the aircraft is controlled to steer toward a steering angle according to the new extension direction, so as to fly along the new extension direction. However, if the direction of line extension is not completely consistent with the actual extension direction of the following target due to calculation errors based on the captured images, when the aircraft flies toward the calculated extension direction, the aircraft will deviate from the position of the following target. Even if the flight direction is corrected again according to the next calculation, the aircraft has already deviated from the position of the following target, and as a result, the unstable situation of flying sway or snaking occurs. If the above deviation from the following target is too large, the aircraft may completely lose the target.
此外,藉由轉向以改變跟隨方向的方法主要是針對位於飛行器下方平面之線條進行跟隨。若所需跟隨之線條係位於一前方垂直面上,例如跟隨目標係一大樓牆面之待掃描裂痕、高壓電塔檢測,摩天樓外牆設備安檢或是風力發電機葉片檢測等等任務時,由於這些設備或是建物的高度較為高聳,對於任務執行人員來說具有一定程度的風險,因此使用無人機進行任務是較為安全的。在這些任務中,由於必須沿一垂直面飛行移動,飛行器即無法藉由平面轉向以改變前進方向,因為如此會使得飛行器失去前方正在跟隨的線條,或是轉向已經跟隨過的線條部分。由上述可知,現有技術藉由偵測線條延伸方向控制飛行器進行一水平面的轉向以改變飛行方向的控制方法並不適用於跟隨垂直面上線條之應用環境,因此,現有技術的飛行器線條跟隨控制方法勢必須進行進一步改良,並謀求可行的解決方案。 In addition, the method of changing the following direction by turning is mainly to follow the lines on the plane below the aircraft. If the line to be followed is located on a front vertical plane, for example, the following target is a crack to be scanned on the wall of a building, high-voltage electrical tower inspection, equipment security inspection of skyscraper exterior walls, or wind turbine blade inspection, etc. , Because the height of these equipment or buildings is relatively high and poses a certain degree of risk to the mission executors, it is safer to use drones for missions. In these tasks, since it must move along a vertical plane, the aircraft cannot change the forward direction by turning the plane, because this will cause the aircraft to lose the line it is following in front of it, or to turn to the part of the line it has already followed. It can be seen from the above that the prior art control method of controlling the aircraft to turn on a horizontal plane by detecting the extension direction of the line to change the flying direction is not suitable for the application environment of following the line on the vertical plane. Therefore, the prior art line follow control method of the aircraft It is necessary to make further improvements and seek feasible solutions.
有鑑於現有的飛行器線條跟隨的飛行控制精準度低,發生偵測判斷誤差時輕微則飛行搖擺蛇行,嚴重時會產生偏移或失去跟隨目標,因此本發明提供一種可提升精準度的飛行器線條跟隨控制方法,為達成前述目的採取的主要技術手段係令前述方法由一飛行器的控制器執行,該飛行器線條跟隨控制方法包含以下步驟:對一影像畫面中央的一分割影像進行再分割,以產生多數的再分割影像;判斷各該再分割影像中出現一跟隨目標的機率值;根據各該再分割影像的機率值決定一飛行方向,並計算該飛行方向中之該跟隨目標的一中心點;根據該跟隨目標的中心點決定一虛擬飛行座標。 In view of the low accuracy of the flight control of the existing aircraft line following, when the detection and judgment error occurs, the flight will sway and snake, and when the error is serious, it will shift or lose the target. Therefore, the present invention provides an aircraft line following that can improve the accuracy. The control method. The main technical means adopted to achieve the foregoing objective is to make the foregoing method be executed by an aircraft controller. The aircraft line-following control method includes the following steps: subdividing a segmented image in the center of an image frame to generate a majority Determine the probability value of a following target in each of the subdivided images; determine a flight direction according to the probability value of each subdivided image, and calculate a center point of the following target in the flight direction; according to The center point of the following target determines a virtual flight coordinate.
根據上述線條跟隨控制方法,係將該跟隨目標置於影像畫面中央,也就是該飛行器此時所對準之中心位置,並經該影像畫面中央的分割影像進行再分割,產生多個更小的再分割影像。進一步的,根據該影像畫面中央多個再分割影像中個別的跟隨目標的機率值決定飛行方向,計算該飛行方中該跟隨目標的中心點,並且據以決定一虛擬飛行座標。 According to the above-mentioned line-following control method, the following target is placed in the center of the image frame, that is, the center position the aircraft is aimed at at this time, and the split image in the center of the image frame is subdivided to produce multiple smaller ones. Divide the image again. Further, the flight direction is determined according to the probability values of the individual following targets in the multiple subdivided images in the center of the image frame, the center point of the following target in the flying party is calculated, and a virtual flight coordinate is determined accordingly.
也就是說,透過該跟隨目標在畫面中央的多個再分割影像出現的機率值決定一飛行方向,使得飛行器能夠確實沿著跟隨目標所在的方向前進,並且維持該跟隨目標在飛行器拍攝的影像畫面中央;進一步的,由於該虛擬飛行座標是根據該跟隨目標的一中心點決定,也就是該中心點勢必在該跟隨目標所在之範圍內,因此該飛行器能精準的朝向該虛擬飛行座標前進,而確保飛行器不會偏離跟隨目標所在之面積範圍,達到高精準度的線條跟隨。 In other words, the probability value of the multiple subdivided images of the following target in the center of the screen determines a flight direction, so that the aircraft can actually move in the direction of the following target, and maintain the image of the following target on the aircraft. Center; further, because the virtual flight coordinates are determined according to a center point of the following target, that is, the center point is bound to be within the range of the following target, so the aircraft can accurately advance toward the virtual flight coordinates, and Ensure that the aircraft does not deviate from the area of the following target, achieving high-precision line tracking.
此外,本發明另外還提供一種應用上述線條跟隨控制方法的飛行器控制方法,係由一飛行器執行,包含以下步驟:擷取一影像畫面,將該影像畫面劃分為多數個分割影像;判斷各該分割影像中出現一跟隨目標的機率值;根據各該分割影像中的跟隨目標的機率值決定一飛行方向區域;當該飛行方向區域位在該影像畫面中央的一分割影像內,對該分割影像進行再分割,以產生多數的再分割影像;判斷各該再分割影像中出現該跟隨目標的機率值;根據各該再分割影像的機率值決定一飛行方向,並計算該飛行方向中之該跟隨目標的一中心點;根據該跟隨目標的中心點決定一虛擬飛行座標;根據該虛擬飛行座標產生一飛行指令。 In addition, the present invention also provides an aircraft control method using the above-mentioned line-following control method, which is executed by an aircraft and includes the following steps: capturing an image frame, dividing the image frame into a plurality of segmented images; judging each segment A probability value of following the target appears in the image; a flying direction area is determined according to the probability value of following the target in each segmented image; when the flying direction area is located in a segmented image in the center of the image frame, the segmented image is performed Subdivide to generate a majority of subdivided images; determine the probability value of the following target in each subdivided image; determine a flight direction according to the probability value of each subdivided image, and calculate the following target in the flight direction Determine a virtual flight coordinate based on the center point of the following target; generate a flight command based on the virtual flight coordinate.
也就是說,本發明應用上述飛行器線條跟隨控制方法的飛行器控制方法通過二階段的判斷決定飛行器實際飛行的方向及精確目的座標。首先將影像畫面切分為複數分割影像,判斷各該分割影像中跟隨目標的機率值以大略的判斷該跟隨目標就飛行器現在所在位置而言的相對位置,以決定一飛行方向區域,再從該方向區域中精確的計算該跟隨目標所在區域之中心點,使得該飛行器確實的朝向該中心點前進,確保該飛行器的飛行方向及擷取的影像畫面所在位置確實是沿著該跟隨目標前進。 That is to say, the aircraft control method of the present invention applying the above-mentioned aircraft line following control method determines the actual flight direction and precise target coordinates of the aircraft through a two-stage judgment. First, the image frame is divided into multiple segmented images, and the probability value of following the target in each segmented image is judged to roughly determine the relative position of the following target with respect to the current position of the aircraft to determine a flight direction area. Accurately calculate the center point of the area where the following target is located in the direction area, so that the aircraft will advance toward the center point, ensuring that the flying direction of the aircraft and the location of the captured image frame are indeed moving along the following target.
也就是說,由於該飛行器並非簡單判斷跟隨目標的延伸方向並朝該概略的方向轉向進行飛行,而是準確地朝向該跟隨目標所在之方向及精確的一座標點飛行,因此不會因線條延伸方向偵測錯誤而偏離或失去跟隨目標,從而可維持該飛行器高準確地沿著跟隨目標所在之位置前進。 That is to say, because the aircraft does not simply judge the extension direction of the following target and turn to the rough direction to fly, but accurately flies toward the direction of the following target and a precise punctuation, so it will not be affected by the extension direction of the line. Detecting errors and deviating or losing the following target can maintain the aircraft to move along the position of the following target with high accuracy.
10:影像畫面 10: Video screen
11:跟隨目標 11: Follow the goal
110:重心 110: center of gravity
101’:最近目標寬度中點 101’: Midpoint of the nearest target width
G0~G8:分割影像 G 0 ~ G 8 : Split image
G01~G08:再分割影像 G 01 ~ G 08 : Divide the image again
圖1係本發明飛行器線條跟隨控制方法的流程圖。 Fig. 1 is a flow chart of the method for controlling the line following of the aircraft of the present invention.
圖2係本發明飛行器線條跟隨控制方法的應用示意圖。 Figure 2 is a schematic diagram of the application of the aircraft line following control method of the present invention.
圖3係本發明飛行器線條跟隨控制方法第一較佳實施例的流程圖。 Fig. 3 is a flowchart of the first preferred embodiment of the aircraft line following control method of the present invention.
圖4係本發明飛行器線條跟隨控制方法第一較佳實施例的應用示意圖。 Fig. 4 is an application schematic diagram of the first preferred embodiment of the aircraft line following control method of the present invention.
圖5係本發明飛行器線條跟隨控制方法第二較佳實施例的流程圖。 Fig. 5 is a flowchart of the second preferred embodiment of the aircraft line following control method of the present invention.
圖6A~6D係本發明飛行器線條跟隨控制方法第二較佳實施例的應用示意圖。 6A to 6D are schematic diagrams of the application of the second preferred embodiment of the aircraft line following control method of the present invention.
圖7係本發明應用飛行器線條控制方法的飛行控制方法的流程圖 Figure 7 is a flow chart of the flight control method applying the aircraft line control method of the present invention
圖8係本發明應用飛行器線條控制方法的飛行控制方法的應用示意圖 Fig. 8 is a schematic diagram of the application of the flight control method using the aircraft line control method of the present invention
圖9係本發明應用飛行器線條控制方法的飛行控制方法第三較佳實施例的流程圖。 Fig. 9 is a flowchart of a third preferred embodiment of a flight control method using the aircraft line control method of the present invention.
圖10係本發明應用飛行器線條控制方法的飛行控制方法第三較佳實施例的應用示意圖。 FIG. 10 is a schematic diagram of the application of the third preferred embodiment of the flight control method using the aircraft line control method of the present invention.
圖11係本發明應用飛行器線條控制方法的飛行控制方法第四較佳實施例的流程圖。 FIG. 11 is a flowchart of a fourth preferred embodiment of a flight control method using the aircraft line control method of the present invention.
關於本發明的一較佳實施例,係在提供一種飛行器線條跟隨控制方法,其由一飛行器的一控制器執行,請參閱圖1所示,其包含以下步驟:對一影像畫面中央的一分割影像進行再分割,以產生多數的再分割影像(S101);判斷各該再分割影像中出現一跟隨目標的機率值(S102);根據各該再分割影像的機率值決定一飛行方向,並計算該飛行方向中之該跟隨目標的一中心點(S103);根據該跟隨目標的中心點決定一虛擬飛行座標(S104)。 Regarding a preferred embodiment of the present invention, an aircraft line following control method is provided, which is executed by a controller of an aircraft, as shown in FIG. 1, which includes the following steps: a segmentation in the center of an image frame The image is subdivided to generate a plurality of subdivided images (S101); it is determined that a probability value of following the target appears in each subdivided image (S102); a flight direction is determined according to the probability value of each subdivided image, and calculated A center point of the following target in the flight direction (S103); a virtual flight coordinate is determined according to the center point of the following target (S104).
請參閱圖2所示,圖2係該影像畫面10的示意圖。在步驟S101中,將該影像畫面10中央的一分割影像G0進一步分割為多個再分割影像G00~G08。其中,各該再分割影像的編號G00~G08標示於圖中各該再分割影像的右下角,而各該再分割影像中出現該跟隨目標11的機率值標示於各該再分割影像的右上角。在本實施例中,該分割影像G0計分割為九個再分割影像G00~G08,九個再分割影像作三行三列對正,也就是一般俗稱的九宮格。其中,可設定該再分割影像G00位於該影像畫面10的最中央,亦即該飛行器此時正對之所在位置,且設定其餘再分割影像G02~G08由正上方依序逆時針排列。
Please refer to FIG. 2, which is a schematic diagram of the
在步驟S102中,計算多個再分割影像中該跟隨目標11出現的機率值p(G 0i ),而該跟隨目標11出現的機率值較佳地係該跟隨目標11在該再分割影像中之面積比例計算得到。舉例來說,各該再分割影像中的機率值p(G 0i )是根據以下公式計算:
p(G 0i )=a(G 0i )/A p( G 0 i ) = a ( G 0 i )/ A
其中,a(G 0i )係指該跟隨目標11在各該再分割影像G0i中的面積,A為該跟隨目標11在各該再分割影像中的總面積。
Wherein, a ( G 0 i ) refers to the area of the following
請參閱圖3所示,在本發明的一第一較佳實施例中,在步驟S102中,當計算得到各該再分割影像中的原始機率值後(S1021),還進一步進行以下步驟以得到各該再分割影像中出現一跟隨目標11的機率值:判斷該飛行器的一飛行慣性反方向和一飛行慣性方向(S1022);當位於該飛行慣性反方向上的再分割影像的機率值大於0,將該再分割影像的機率值設定為「0」(S1023);當位於該飛行慣性方向上的再分割影像的機率值大於0,將該再分割影像的機率值設定為「1」(S1024)。
Referring to FIG. 3, in a first preferred embodiment of the present invention, in step S102, after the original probability value in each subdivided image is calculated (S1021), the following steps are further performed to obtain A probability value of following
請一併參閱圖2及4所示,圖2係該影像畫面10中各該再分割影像內根據面積計算公式所得到之跟隨目標11的原始機率值。為方便說明,定義該影像畫面10的一Y軸方向及一Z軸方向,假設該飛行器係由該影像畫面10右側言該跟隨目標11向左側飛行,該飛行器的一飛行慣性係一-y方向。因此,該飛行慣性反方向係一+y方向,也就是在該再分割影像G00之右側。根據步驟S1021及S2022,當判斷得到該飛行慣性反方向後,將該飛行慣性反方向上之再分割影像的機率值設定為「0」,在圖4A的例子中,該飛行慣性反方向上的再分割影
像為G07。此外,較佳的,由於該再分割影像G00為該飛行器目前對準之位置,不屬於待選擇的飛行方向,因此可將G00的機率值也設定為「0」。
Please refer to FIGS. 2 and 4 together. FIG. 2 is the original probability value of following the
上述步驟係用以排除飛行器前一個時刻所在方向的再分割影像,避免根據各該再分割影像的機率值決定之飛行方向導致飛行器反覆回頭之狀況發生。 The above steps are used to eliminate the subdivided images in the direction of the aircraft at the previous moment, so as to avoid the situation that the flying direction determined according to the probability value of each subdivided image causes the aircraft to turn back repeatedly.
經過上述調整步驟後,各該再分割影像的機率值如圖4所示,其進一步將飛行慣性方向上的再分割影像G0i的機率值設定為「1」,例如圖4中的該再分割影像G03。如此一來,在步驟S103中,根據各該再分割影像G00~G08的機率值,即能判斷具有最大機率值的該再分割影像G03之方向為該飛行器應該行進之飛行方向,並接著計算該飛行方向中之該跟隨目標11的一中心點。在圖4所示的例子中,也就是該再分割影像G03中之該跟隨目標11的中心點。
After the above adjustment steps, the probability value of each subdivided image is shown in Figure 4, which further sets the probability value of the subdivided image G 0i in the flight inertia direction to "1", such as the subdivided image in Figure 4 Image G 03 . In this way, in step S103, according to the probability values of each of the subdivided images G 00 ~ G 08 , it can be determined that the direction of the subdivided image G 03 with the maximum probability value is the flight direction in which the aircraft should travel, and Then, a center point of the following
請參閱圖5及圖6A所示,較佳的,前述跟隨目標11的中心點係根據以下步驟得到:計算該跟隨目標11在該再分割影像中之重心110,並判斷該重心110是否在該跟隨目標11所在之範圍內(S1031);當該重心在該在跟隨目標11所在之範圍內,設定該重心110為該跟隨目標11的中心點(S1032);當該重心不在該跟隨目標11所在之範圍內,計算該重心110的一最近目標寬度中點,設定該最近目標寬度中點110’為該跟隨目標11的中心點(S1033)。
Please refer to FIGS. 5 and 6A. Preferably, the center point of the following
其中,步驟S1031所述重心的位置座標(l y ,l z )可根據以下公式計算得到:
其中,(y i ,z i )為該再分割影像中為該跟隨目標11涵蓋之每一像素座標,M為該再分割影像中該跟隨目標11涵蓋之總像素數量。
Among them, ( y i , z i ) is the coordinate of each pixel covered by the following
請參閱圖6A~6D所示,根據幾何原理可知,經上述公式計算所得之重心可能不在該跟隨目標11所在之範圍內。因此,如圖6A及6B所示,當該重心是在該跟隨目標11所在之範圍內時,則決定該重心即為步驟S103中所需之中心點,圖中該白點110即為計算得到位於該再分割影像G0i中的跟隨目標11的重心110,而該重心110在該跟隨目標11的範圍內,因此決定該重心110即為該跟隨目標11在該再分割影像中的中心點;如圖6C及6D所示,當判斷該重心110不在該跟隨目標11所在之範圍內時,則進行一修正計算,計算離該重心110所在位置距離最近的該跟隨目標11的一寬度中點,並以該最近目標寬度中點110’作為中心點,其中該白點為經上述公式計算得到該跟隨目標11在該再分割影像中的重心110,而該黑點為經修正所得到之中心點。該最近目標寬度中點可根據以下計算方法得到:判斷該重心在其Y軸方向及Z軸方向上距離該跟隨目標較近的其中一方向,設定該方向為一修正中心方向;判斷該跟隨目標在該修正中心方向的一寬度,並將該中心點設定為該寬度的一中點。
Please refer to Figures 6A to 6D. According to geometric principles, the center of gravity calculated by the above formula may not be within the range where the following
以圖6C的例子說明,該再分割影像G0i中,該重心110是在該Z軸方向上與該跟隨目標11距離較近,因此設定Z軸方向為該修正中心方向。進一步的,判斷該跟隨目標在該重心110的Z軸方向上的一寬度,並將該跟隨目標在該寬度的中點設定為該中心點。該寬度中點的座標(l y ',l z ')可根據以下計算公式得到:
其中,l z_max 及l z_min 分別為該跟隨目標11在該Z軸方向距離該重心110較遠的一邊界點Z軸座標值,及距離該重心110較近的一邊界點Z軸座標值。
Wherein, l z_max and l z_min are respectively a Z-axis coordinate value of a boundary point farther from the center of
若該重心110是在一Y軸方向上距離該跟隨目標11較近,則設定Y軸方向為該修正中心方向,並判斷該跟隨目標在該重心的Y軸方向上的一寬度,並將該跟隨目標11在該Y軸方向寬度的中點設定為該中心點110’。計算方法與上述計算一Z軸方向上的寬度中點方法近似,在此不加以贅述。
If the center of
如此一來,將位在該跟隨目標範圍外的重心平移至該最近目標寬度中點,確保根據前述步驟S1031所得到之中心點會位在該跟隨目標11內,而且不是位於該飛行器前一個時刻所在之方向上,因此根據該中心點決定之該虛擬飛行座標可確實使得飛行器沿著該跟隨目標11飛行,且不會偏離該跟隨目標11所在之位置。
In this way, the center of gravity located outside the range of the following target is translated to the midpoint of the nearest target width to ensure that the center point obtained according to the foregoing step S1031 will be located within the following
本發明提供的第二個較佳實施例,是一種應用了上述線條跟隨控制方法的飛行控制方法,請參閱圖7所示,包含以下步驟:擷取一影像畫面,將該影像畫面劃分為多數個分割影像(S701);判斷各該分割影像中出現一跟隨目標的機率值(S702);根據各該分割影像中的跟隨目標的機率值決定一飛行方向區域(S703);當該飛行方向區域在該影像畫面中央(S704),執行上述線條跟隨控制方法:對該影像畫面中央的分割影像進行再分割,以產生多數的再分割影像(S101);判斷各該再分割影像中出現該跟隨目標的機率值(S102);根據各該再分割影像的機率值決定一飛行方向,並計算該飛行方向中之該跟隨目標的一中心點(S103);根據該跟隨目標的中心點決定一虛擬飛行座標(S104) 根據該虛擬飛行座標產生一飛行指令(S705)。 The second preferred embodiment provided by the present invention is a flight control method using the above-mentioned line following control method. Please refer to FIG. 7, which includes the following steps: capturing an image frame and dividing the image frame into a plurality of Segmented images (S701); determine the probability value of a following target in each segmented image (S702); determine a flying direction area according to the probability value of following the target in each segmented image (S703); when the flying direction area In the center of the image frame (S704), execute the above-mentioned line-following control method: subdivide the segmented image in the center of the image frame to generate a majority of subdivided images (S101); determine that the following target appears in each subdivided image The probability value of (S102); determine a flight direction according to the probability value of each subdivided image, and calculate a center point of the following target in the flight direction (S103); determine a virtual flight according to the center point of the following target Coordinates (S104) A flight instruction is generated according to the virtual flight coordinates (S705).
該飛行控制方法由一飛行器執行,包含一攝影模組及一控制器,該攝影模組用以擷取一影像畫面,且一般來說是朝向該飛行器的前方。請一併參閱圖8所示,當該攝影模組擷取該影像畫面10後,係先將該影像畫面10分割為多個分割影像G0~G8(S701),其中G0為位於畫面中央之分割影像,G1~G8由影像畫面10正上方開始為逆時針排列。其中,各該分割影像的編號G0~G8標示於各該分割影像的左下角,而各該分割影像中出現該跟隨目標11的機率值標示於各該分割影像的右上角。在本實施例中,該影像畫面10計分割為九個分割影像G0~G8,九個分割影像G0~G8作三行三列對正,亦即俗稱的九宮格。
The flight control method is executed by an aircraft and includes a camera module and a controller. The camera module is used to capture an image frame and generally faces the front of the aircraft. Please also refer to Figure 8. After the camera module captures the
接著計算各該分割影像G0~G8中該出現跟隨目標11之機率值p(G i )(S702)。該機率值的計算方法與步驟S102中計算各該再分割影像中出現該跟隨目標11的機率值的計算方法類似,也就是計算該跟隨目標11在各該分割影像G0~G8中之面積比例,其計算公式如下:
p(G i )=a'(G i )/A' p( G i ) = a' ( G i )/ A'
其中,a'(G i )係指該跟隨目標11在各該分割影像G0~G8中的面積,A'為該跟隨目標11在該影像畫面10中的總面積。
Wherein, a '(G i) of the means to follow the
在步驟S703中,根據上一步驟計算得到的各該分割影像的機率值決定一飛行方向區域。其中,上一步驟計算得到的各該分割影像的機率值係代表了各該分割影像中包含的該跟隨目標11比例,當其中一分割影像Gi中的機率值為「0」時,表示該跟隨目標11並未出現在該分割影像中;當其中一分割影像Gi中的機率值不為「0」時,表示該跟隨目標11有出現在該分割影像中。因
此,根據各該分割影像中的機率值,可判斷該跟隨目標11在該影像畫面10中的分佈概況,並可據以決定一飛行方向區域。
In step S703, a flight direction area is determined according to the probability value of each segmented image calculated in the previous step. Among them, the probability value of each segmented image calculated in the previous step represents the proportion of the following
請參閱圖9所示,在本發明的一第三較佳實施例中,較佳的,根據各該分割影像中的跟隨目標11機率值決定一飛行方向區域的步驟(S703)中,係根據以下子步驟判斷:判斷該影像畫面10中央的該分割影像的跟隨目標的機率值是否為「0」(S7031);當該影像畫面10中央的該分割影像的跟隨目標的機率值不為「0」,設定該影像畫面10中央的該分割影像為該飛行方向區域(S7032);當該影像畫面10中央的該分割影像的跟隨目標的機率值為「0」,判斷其他分割影像的跟隨目標的機率值是否為「0」(S7033);當其他分割影像的跟隨目標的機率值皆為「0」,結束此方法流程(S7034);當至少一分割影像的跟隨目標的機率值不為「0」,設定具有最大機率值的該分割影像為飛行方向區域(S7035)。
Please refer to FIG. 9, in a third preferred embodiment of the present invention, preferably, in the step (S703) of determining a flight direction area according to the probability value of following the
請參閱圖8所示,當該影像畫面10中央的分割影像G0中出現跟隨目標的機率值不為「0」,表示該跟隨目標11確實位於該影像畫面10中央的分割影像G0中,配合參閱圖4所示,亦表示該飛行器並未偏離該跟隨目標11,因此設定該影像畫面10中央的分割影像G0為飛行方向區域(S704),並執行前述步驟S101~S104的線條跟隨控制方法,據以精確產生位於分割影像G0中的虛擬飛行座標,進而產生飛行指令(S705),使得該飛行器繼續沿著該跟隨目標11飛行。
Please refer to FIG. 8, when the probability value of the following target appearing in the divided image G 0 in the center of the
反之,當該畫面中央的分割影像G0的跟隨目標11的機率值為「0」,該跟隨目標11並未出現在該畫面中央的分割影像G0中,表示該飛行器
已經偏離該跟隨目標11,因此設定具有最大機率值的該分割影像為飛行方向區域(S7035)。以圖10所示之影像畫面10為例,該影像畫面10中的該跟隨目標11並未出現在畫面中央的分割影像G0中,而具有最大機率值的分割影像為正上方之分割影像G1,因此決定該分割影像G1之範圍為該飛行方向區域。
Conversely, when the probability value of the segmented image G 0 in the center of the screen to follow the
如此一來,該飛行方向區域並未在該畫面中央,也就是該飛行方向區域並非該畫面中央的分割影像G0,必須進一步執行一飛行器自身位置校正步驟(S707)。 As a result, the flight direction area is not in the center of the screen, that is, the flight direction area is not the segmented image G 0 in the center of the screen, and a further step of correcting the aircraft's own position (S707) must be performed.
請參閱圖11所示,前述飛行器自身位置校正步驟包含以下子步驟:計算該飛行方向區域中該跟隨目標的一中心點(S7071);根據該跟隨目標的中心點決定一虛擬飛行座標(S7072);根據該虛擬飛行座標產生一飛行指令(S7073)。 As shown in FIG. 11, the aforementioned step of correcting the position of the aircraft includes the following sub-steps: calculating a center point of the following target in the flight direction area (S7071); determining a virtual flight coordinate according to the center point of the following target (S7072) ; Generate a flight instruction according to the virtual flight coordinates (S7073).
較佳的,步驟S7071中計算該飛行方向區域中跟隨目標11的中心點的計算方法,與前述步驟S103中計算該飛行方向中之該跟隨目標11的中心點的方法類似,包含以下步驟:計算該跟隨目標在該分割影像中之重心,並判斷該重心是否在該跟隨目標所在之範圍內;當該重心在該在跟隨目標所在之範圍內,設定該重心為該跟隨目標的中心點;當該重心不在該跟隨目標所在之範圍內,計算該重心的一最近目標寬度中點,設定該最近目標寬度中點為該跟隨目標的中心點。
Preferably, the calculation method of calculating the center point of the following
其中,該跟隨目標11在該分割影像中之重心(l y ,l z )可通過下列公式計算得到:
其中,(y i ,z i )為該分割影像中為該跟隨目標11涵蓋之每一像素座標,M為該分割影像中該跟隨目標11涵蓋之總像素數量。
Among them, ( y i , z i ) is the coordinate of each pixel covered by the following
而當計算所得之重心(l y ,l z )是在該飛行方向區域中的跟隨目標11所在之範圍內,則決定該重心為該中心點;當該重心(l y ,l z )不在該跟隨目標11的範圍內則進一步進行一修正計算,計算離該重心所在位置距離最近的該跟隨目標11的一寬度中點,並以該最近目標寬度中點為該中心點。該修正計算之方法及結果與圖5A及5B所示的再分割影像中跟隨目標11中心點計算方法相同,在此不加以贅述。請參閱圖10所示之影像畫面10,根據上述中心點的計算方法,可得到該飛行方向區域中的重心即為分割影像G1中之白點,而該經修正計算後所得之中心點為分割影像G1中所示黑點。
When the calculated center of gravity ( l y ,l z ) is within the range of the following
在步驟S7072中,決定該中心點為該虛擬飛行座標,並進一步根據該虛擬飛行座標產生一飛行指令,以進一步根據該飛行指令控制該飛行器飛行,使得跟隨目標11位在影像畫面10中央的分割影像中。藉此可完成飛行器自身的位置校正,使得跟隨目標11回到飛行器的攝影模組的視野中央,並在回到飛行器控制方法的第一步驟後,接續執行飛行器線條跟隨控制方法,使得該飛行器繼續沿該跟隨目標11飛行。
In step S7072, it is determined that the center point is the virtual flight coordinates, and a flight instruction is further generated according to the virtual flight coordinates, so as to further control the aircraft to fly according to the flight instruction, so as to follow the segmentation of the
較佳的,該飛行指令(v y ,v z )係根據下令公式計算產生:Δy=(l y -c y )or(l y' -c y ) Preferably, the flight command ( v y , v z ) is calculated according to the following formula: Δ y = ( l y - c y ) or ( l y' - c y )
Δz=(l z -c z )or(l z' -c z ) Δ z = ( l z - c z ) or ( l z' - c z )
v y =Δy/λ,v z =Δz/λ v y =Δ y / λ,v z =Δ z / λ
其中,(c y ,c z )為該飛行器當下的自身位置座標,較佳為該影像畫面中央點座標,λ為一可調變時間常數,該飛行指令(v y ,v z )為一速度指令。 Wherein, (c y, c z) that its own current position coordinates of the aircraft, preferably for the center point of the picture image coordinate, [lambda] is a time constant becomes variable, the flight instruction (v y, v z) is a speed instruction.
綜上所述,本發明提出的種高精準度的線條跟隨方法及飛行控制方法使得根據此方法執行線條跟隨飛行的飛行器能夠使攝影模組的視野中心持續對準該跟隨目標線條,即使受到輕微外力使得跟隨目標離開影像畫面中央,也能即刻通過自身位置校正步驟調整飛行器的位置,藉由明確的飛行指令使得跟隨目標回到影像畫面中央,並且確保飛行器高精準度的沿著該跟隨目標的寬度內飛行。 In summary, the high-precision line following method and flight control method proposed by the present invention enable an aircraft that performs line-following flight according to this method to keep the center of the camera module's field of view aligned with the following target line even if it is slightly affected. The external force makes the following target leave the center of the image screen, and the position of the aircraft can be adjusted immediately through its own position correction step. With clear flight instructions, the following target returns to the center of the image screen and ensures that the aircraft follows the following target with high accuracy. Fly within the width.
進一步來說,由於是通過產生該虛擬飛行座標及據以產生的飛行指令飛行,而非產生轉向角改變飛行器的面對方向及飛行方向,飛行器能夠持續朝向前方垂直面上的該跟隨目標,而不會因改變面對的方向而失去跟隨目標,能夠適用該跟隨目標在下方平面或垂直平面的各種應用環境及飛行任務。 Furthermore, since the virtual flight coordinates are generated and the flight instructions are generated accordingly, instead of generating the steering angle to change the facing direction and flight direction of the aircraft, the aircraft can continue to face the following target on the front vertical plane, and The following target will not be lost due to changing the facing direction, and it can be applied to various application environments and flight missions where the following target is on a plane or a vertical plane below.
以上所述僅是本發明的較佳實施例而已,並非對本發明做任何形式上的限制,雖然本發明已以較佳實施例揭露如上,然而並非用以限定本發明,任何熟悉本專業的技術人員,在不脫離本發明技術方案的範圍內,當可利用上述揭示的技術內容做出些許更動或修飾為等同變化的等效實施例,但凡是未脫離本發明技術方案的內容,依據本發明的技術實質對以上實施例所作的任何簡單修改、等同變化與修飾,均仍屬於本發明技術方案的範圍內。 The above are only preferred embodiments of the present invention, and do not limit the present invention in any form. Although the present invention has been disclosed as above in preferred embodiments, it is not intended to limit the present invention. Anyone familiar with the professional technology Personnel, without departing from the scope of the technical solution of the present invention, when the technical content disclosed above can be used to make slight changes or modification into equivalent embodiments with equivalent changes, but any content that does not deviate from the technical solution of the present invention, according to the present invention Any simple modifications, equivalent changes and modifications made to the above embodiments are still within the scope of the technical solutions of the present invention.
Claims (7)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW108120559A TWI720510B (en) | 2019-06-13 | 2019-06-13 | Aircraft line-following control method and flight control method using the method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW108120559A TWI720510B (en) | 2019-06-13 | 2019-06-13 | Aircraft line-following control method and flight control method using the method |
Publications (2)
Publication Number | Publication Date |
---|---|
TW202046247A TW202046247A (en) | 2020-12-16 |
TWI720510B true TWI720510B (en) | 2021-03-01 |
Family
ID=74668409
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW108120559A TWI720510B (en) | 2019-06-13 | 2019-06-13 | Aircraft line-following control method and flight control method using the method |
Country Status (1)
Country | Link |
---|---|
TW (1) | TWI720510B (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110267221A1 (en) * | 2010-04-30 | 2011-11-03 | Applied Physical Sciences Corp. | Sparse Array RF Imaging for Surveillance Applications |
US20140336848A1 (en) * | 2013-05-10 | 2014-11-13 | Palo Alto Research Center Incorporated | System and method for detecting, tracking and estimating the speed of vehicles from a mobile platform |
CN107087144A (en) * | 2017-05-13 | 2017-08-22 | 西安费斯达自动化工程有限公司 | Panorama and precise image/spherical crown variable excitation chirm integrative detection system |
TW201835784A (en) * | 2016-12-30 | 2018-10-01 | 美商英特爾公司 | The internet of things |
-
2019
- 2019-06-13 TW TW108120559A patent/TWI720510B/en active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110267221A1 (en) * | 2010-04-30 | 2011-11-03 | Applied Physical Sciences Corp. | Sparse Array RF Imaging for Surveillance Applications |
US20140336848A1 (en) * | 2013-05-10 | 2014-11-13 | Palo Alto Research Center Incorporated | System and method for detecting, tracking and estimating the speed of vehicles from a mobile platform |
TW201835784A (en) * | 2016-12-30 | 2018-10-01 | 美商英特爾公司 | The internet of things |
CN107087144A (en) * | 2017-05-13 | 2017-08-22 | 西安费斯达自动化工程有限公司 | Panorama and precise image/spherical crown variable excitation chirm integrative detection system |
Also Published As
Publication number | Publication date |
---|---|
TW202046247A (en) | 2020-12-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10871258B2 (en) | Method and system for controlling gimbal | |
US10928838B2 (en) | Method and device of determining position of target, tracking device and tracking system | |
WO2018120011A1 (en) | Projected image correction method and device, and robot | |
WO2020102927A1 (en) | Photographing method and unmanned aerial vehicle | |
US20190018428A1 (en) | Hover control | |
CN108229587B (en) | Autonomous transmission tower scanning method based on hovering state of aircraft | |
WO2018210078A1 (en) | Distance measurement method for unmanned aerial vehicle, and unmanned aerial vehicle | |
CN107292927A (en) | A kind of symmetric motion platform's position and pose measuring method based on binocular vision | |
CN106683118B (en) | Unmanned aerial vehicle target tracking method based on hierarchical model | |
CN108733066B (en) | Target tracking control method based on pod attitude feedback | |
CN114111799B (en) | Unmanned aerial vehicle aerial-shooting path planning method for high-macromonomer fine modeling | |
EP2724204A1 (en) | Method for acquiring images from arbitrary perspectives with uavs equipped with fixed imagers | |
CN109669474B (en) | Priori knowledge-based multi-rotor unmanned aerial vehicle self-adaptive hovering position optimization algorithm | |
CN109765936A (en) | The positioning and control method of mobile terminal, device and unmanned plane | |
CN112947526B (en) | Unmanned aerial vehicle autonomous landing method and system | |
Mills et al. | Vision based control for fixed wing UAVs inspecting locally linear infrastructure using skid-to-turn maneuvers | |
CN112947569B (en) | Visual servo target tracking control method for quad-rotor unmanned aerial vehicle based on preset performance | |
CN107977985A (en) | Unmanned plane hovering method, apparatus, unmanned plane and storage medium | |
CN106976561A (en) | A kind of unmanned plane photographic method | |
CN108460333A (en) | Ground detection method and device based on depth map | |
CN111510704A (en) | Method for correcting camera dislocation and device using same | |
CN112509054A (en) | Dynamic calibration method for external parameters of camera | |
Greatwood et al. | Perspective correcting visual odometry for agile mavs using a pixel processor array | |
CN110928311A (en) | Indoor mobile robot navigation method based on linear features under panoramic camera | |
TWI720510B (en) | Aircraft line-following control method and flight control method using the method |