CN109636853A - Air refuelling method based on machine vision - Google Patents
Air refuelling method based on machine vision Download PDFInfo
- Publication number
- CN109636853A CN109636853A CN201811413048.XA CN201811413048A CN109636853A CN 109636853 A CN109636853 A CN 109636853A CN 201811413048 A CN201811413048 A CN 201811413048A CN 109636853 A CN109636853 A CN 109636853A
- Authority
- CN
- China
- Prior art keywords
- image
- fuel filling
- taper sleeve
- filling taper
- machine vision
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 24
- 239000000446 fuel Substances 0.000 claims abstract description 61
- 238000003708 edge detection Methods 0.000 claims abstract description 20
- 125000002619 bicyclic group Chemical group 0.000 claims description 11
- 238000010586 diagram Methods 0.000 description 2
- 230000005484 gravity Effects 0.000 description 2
- 238000010146 3D printing Methods 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000003032 molecular docking Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- B64D39/00—Refuelling during flight
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The air refuelling method based on machine vision that this application provides a kind of, comprising: obtain the first image of fuel filling taper sleeve；First image is handled, the second image is obtained；Edge detection and shape recognition are carried out to the second image；According to the result of edge detection and shape recognition as a result, calculating the position coordinates of fuel filling taper sleeve；The motion state of fuel filling taper sleeve is tracked and locked according to position coordinates；According to motion state, air refuelling is carried out to aircraft.
Description
Technical field
This application involves technical field of aerospace, specifically provide a kind of air refuelling method based on machine vision.
Background technique
Hose type in the air independently by oil be aircraft flight drive in several one of subjects being most difficult to, need refueled aircraft by oil
Pipe is accurately docked with the fuel filling taper sleeve of fuel charger is just able to achieve oiling, is all that the eyes of pilot is leaned on to estimate in current refueling process
Meter and driving experience prediction come complete operation docking, due to the air-flow that is disturbed influence and pilot visual angle limitation etc. factors
Restricted influence, all the time air refuelling are all that a success rate is not high and the very high task of operational hazards degree.
Summary of the invention
At least one in order to solve the above-mentioned technical problem, this application provides a kind of air refuelling sides based on machine vision
Method, comprising: obtain the first image of fuel filling taper sleeve, wherein the fuel filling taper sleeve includes twin nuclei；To the first image into
Row processing, obtains the second image；Edge detection and shape recognition are carried out to second image；According to the knot of the edge detection
Fruit and the shape recognition as a result, calculating the position coordinates of the fuel filling taper sleeve；It tracks and locks according to the position coordinates
The motion state of the fuel filling taper sleeve；According to the motion state, air refuelling is carried out to aircraft.
According at least one embodiment of the application, the color mode of the first image is RGB mode；It is described to described
First image is handled, and the second image is obtained, comprising: the first image is converted to HSV color sky by rgb color space
Between；Using best H threshold value, best S threshold value and best V threshold value, the first image is filtered；Filtered image is institute
State the second image.
According at least one embodiment of the application, the best H threshold value be 80, the best S threshold value be 80, it is described most
Good V threshold value is 50.
It is described that the first image is converted into HSV color by rgb color space according at least one embodiment of the application
Color space, using following formula:
V=max (R, G, B)
H=H+360 (H < 0).
It is described that edge detection and shape recognition are carried out to second image according at least one embodiment of the application,
It include: that edge detection and shape recognition are carried out to second image by Canny edge detection algorithm.
According at least one embodiment of the application, the result according to the edge detection and the shape recognition
As a result, calculating the position coordinates of the fuel filling taper sleeve, comprising: by the edge of the bicyclic characteristic area to the fuel filling taper sleeve into
The fitting of row least square ellipse, the ratio between obtained inside and outside transverse；If the ratio between described inside and outside transverse is in setting range
The bicyclic bicyclic characteristic area for the fuel filling taper sleeve that is interior, then detecting；If the ratio between described inside and outside transverse is not described
In setting range, then detect it is bicyclic be not the fuel filling taper sleeve bicyclic characteristic area.
According at least one embodiment of the application, the edge by the bicyclic characteristic area to the fuel filling taper sleeve
Least square ellipse fitting is carried out, is calculated as the following formula:
Wherein,
(x0,y0) indicating elliptical barycentric coodinates, a indicates major axis radius, and b indicates minor axis radius, and θ indicates long axis and x-axis
Angle.
It is described to be tracked according to the position coordinates and lock the fuel filling taper sleeve according at least one embodiment of the application
Motion state, comprising: judge second image with the presence or absence of blocking；It blocks if it does not exist, then track and locks described add
The motion state of oily tapered sleeve.
It is described to be tracked according to the position coordinates and lock the fuel filling taper sleeve according at least one embodiment of the application
Motion state, further includes: block if it exists, then predict the optimal location of the fuel filling taper sleeve.
According at least one embodiment of the application, measured described in the center of the fuel filling taper sleeve and resolving as the following formula
The position coordinates of fuel filling taper sleeve:
Wherein, L is fuel filling taper sleeve scale model size, and f is camera focus, and l is shared by fuel filling taper sleeve image in plane
Pixel number, Z are distance of the zero point of reference frame where camera to fuel filling taper sleeve place plane.
In air refuelling method provided by the embodiments of the present application based on machine vision, cone accurately can be measured and tracked
The movement of set obtains accurate tapered sleeve relative position parameter, and this method resolving speed of service is fast, and strong robustness can fit
It the case where of occurring in air refuellings such as answers movement deformation, the random motion of a degree of tapered sleeve and blocks, there is preferable reality
Engineering application value.
Detailed description of the invention
Fig. 1 is the flow diagram of oiling method provided by the embodiments of the present application；
Fig. 2 is the first image of fuel filling taper sleeve provided by the embodiments of the present application；
Fig. 3 is the second image of fuel filling taper sleeve provided by the embodiments of the present application；
Fig. 4 is image of the second image provided by the embodiments of the present application after edge detection and shape recognition；
Fig. 5 is that fuel filling taper sleeve position of centre of gravity and tapered sleeve position coordinates provided by the embodiments of the present application resolve image.
Specific embodiment
The application is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining related application, rather than the restriction to this application.It also should be noted that in order to
Convenient for description, part relevant to the application is illustrated only in attached drawing.
It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase
Mutually combination.The application is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
Fig. 1 is the flow diagram of oiling method provided by the embodiments of the present application, as shown in Figure 1, this method includes following
Step:
Step 101, the first image of fuel filling taper sleeve is obtained.
Wherein, fuel filling taper sleeve includes twin nuclei.
As an alternative embodiment, in order to carry out air refuelling environmental simulation experiment, it can be using 3D printing
Oiling umbrella cone is simulated in contracting than tapered sleeve model, using can the industry camera of servo motion simulate refueled aircraft, build one and build
The oiling simulated environment of view, calibration for cameras and space coordinates, and the first figure for acquiring fuel filling taper sleeve is connected on computer
Picture, acquired image are as shown in Figure 2.
Step 102, the first image is handled, obtains the second image.
Optionally, the color mode of the first image is RGB mode, then can be carried out by following step to the first image
Processing is to obtain the second image:
First image is converted into HSV color space by rgb color space；
Using best H threshold value, best S threshold value and best V threshold value, the first image is filtered；
Filtered image is the second image.
Wherein, best H threshold value includes but is not limited to 80, and best S threshold value includes but is not limited to 80, best V threshold value include but
It is not limited to 50.
First image, which is converted to HSV color space by rgb color space, to use following formula:
V=max (R, G, B)
H=H+360 (H < 0).
Illustratively, the second image obtained after filtering is as shown in Figure 3.
Step 103, edge detection and shape recognition are carried out to the second image.
Edge detection and shape recognition can be carried out to the second image using Canny edge detection algorithm.
Illustratively, the image after edge detection and shape recognition is as shown in Figure 4.
Step 104, according to the result of edge detection and shape recognition as a result, calculating the position coordinates of fuel filling taper sleeve.
Optionally, the position coordinates for calculating fuel filling taper sleeve can be implemented by the following steps:
Least square ellipse fitting is carried out by the edge of the bicyclic characteristic area to fuel filling taper sleeve, what is obtained is inside and outside ellipse
The ratio between circle long axis；
If the ratio between inside and outside transverse is within the set range, the bicyclic bicyclic characteristic area for fuel filling taper sleeve that detects
Domain；
If the ratio between inside and outside transverse not within the set range, detect it is bicyclic be not fuel filling taper sleeve bicyclic spy
Levy region.
Wherein, the equation expression formula of plane ellipse at any position are as follows:
In formula, (x0,y0) indicate elliptical barycentric coodinates, a indicates major axis radius, and b indicates minor axis radius, θ indicate long axis with
The angle of x-axis enables:
Then elliptic equation can be rewritten are as follows:
x2+Axy+By2+ Cx+Dy+E=0
It, should be by seeking objective function according to the principle of least square
To determine parameter A, B, C, D, E, wherein k=1,2,3 ... ..., n (n > 5).
According to extremum principle, F value to be made is minimum, then:
Elliptical center coordinate can be obtained after obtaining the value of A, B, C, D, E in solution are as follows:
Illustratively, it is as shown in Figure 5 to resolve image for the position of centre of gravity of fuel filling taper sleeve and position coordinates.
Step 105, the motion state of fuel filling taper sleeve is tracked and locked according to position coordinates.
Optionally, it is tracked according to position coordinates and the motion state for locking fuel filling taper sleeve can be by following steps come real
It is existing:
Judge that the second image whether there is to block；It blocks if it does not exist, then tracks and lock the motion state of fuel filling taper sleeve；
It blocks if it exists, then predicts the optimal location of fuel filling taper sleeve.
In one example, according to pinhole imaging system principle, in the situation known to fuel filling taper sleeve scale model size, camera focus
Under, its depth location is resolved according to pixel number shared by fuel filling taper sleeve image in plane, specifically, can be carried out according to the following formula
It calculates:
Wherein, L is fuel filling taper sleeve scale model size, and f is camera focus, and l is shared by fuel filling taper sleeve image in plane
Pixel number, Z are distance of the zero point of reference frame where camera to fuel filling taper sleeve place plane.
Step 106, according to motion state, air refuelling is carried out to aircraft.
Air refuelling method provided in this embodiment based on machine vision accurately can measure and track the fortune of tapered sleeve
It is dynamic, obtain accurate tapered sleeve relative position parameter, and this method resolving speed of service is fast, strong robustness is adapted to centainly
It tapered sleeve movement deformation, the random motion of degree and the case where of occurring in air refuellings such as blocks, there is preferable Practical Project to answer
With value.
So far, it has been combined preferred embodiment shown in the drawings and describes the technical solution of the application, still, this field
Technical staff is it is easily understood that the protection scope of the application is expressly not limited to these specific embodiments.Without departing from this
Under the premise of the principle of application, those skilled in the art can make equivalent change or replacement to the relevant technologies feature, these
Technical solution after change or replacement is fallen within the protection scope of the application.
Claims (10)
1. a kind of air refuelling method based on machine vision characterized by comprising
Obtain the first image of fuel filling taper sleeve, wherein the fuel filling taper sleeve includes twin nuclei；
The first image is handled, the second image is obtained；
Edge detection and shape recognition are carried out to second image；
According to the result of the edge detection and the shape recognition as a result, calculating the position coordinates of the fuel filling taper sleeve；
The motion state of the fuel filling taper sleeve is tracked and locked according to the position coordinates；
According to the motion state, air refuelling is carried out to aircraft.
2. the air refuelling method according to claim 1 based on machine vision, which is characterized in that the first image
Color mode is RGB mode；
It is described that the first image is handled, obtain the second image, comprising:
The first image is converted into HSV color space by rgb color space；
Using best H threshold value, best S threshold value and best V threshold value, the first image is filtered；
Filtered image is second image.
3. the air refuelling method according to claim 2 based on machine vision, which is characterized in that the best H threshold value
It is 80, the best S threshold value is 80, and the best V threshold value is 50.
4. the air refuelling method according to claim 2 based on machine vision, which is characterized in that described by described first
Image is converted to HSV color space by rgb color space, using following formula:
V=max (R, G, B)
H=H+360 (H < 0).
5. the air refuelling method according to claim 1 based on machine vision, which is characterized in that described to described second
Image carries out edge detection and shape recognition, comprising:
Edge detection and shape recognition are carried out to second image by Canny edge detection algorithm.
6. the air refuelling method according to claim 1 based on machine vision, which is characterized in that described according to the side
Edge detection result and the shape recognition as a result, calculating the position coordinates of the fuel filling taper sleeve, comprising:
Least square ellipse fitting is carried out by the edge of the bicyclic characteristic area to the fuel filling taper sleeve, what is obtained is inside and outside ellipse
The ratio between circle long axis；
If the ratio between described inside and outside transverse is within the set range, the bicyclic bicyclic spy for the fuel filling taper sleeve that detects
Levy region；
If the ratio between described inside and outside transverse not in the setting range, detect it is bicyclic be not the fuel filling taper sleeve
Bicyclic characteristic area.
7. the air refuelling method according to claim 6 based on machine vision, which is characterized in that described by described
The edge of the bicyclic characteristic area of fuel filling taper sleeve carries out least square ellipse fitting, is calculated as the following formula:
Wherein,
(x0,y0) indicating elliptical barycentric coodinates, a indicates major axis radius, and b indicates minor axis radius, and θ indicates the folder of long axis and x-axis
Angle.
8. the air refuelling method according to claim 1 based on machine vision, which is characterized in that described according to institute's rheme
It sets coordinate tracking and locks the motion state of the fuel filling taper sleeve, comprising:
Judge that second image whether there is to block；
It blocks if it does not exist, then tracks and lock the motion state of the fuel filling taper sleeve.
9. the air refuelling method according to claim 8 based on machine vision, which is characterized in that described according to institute's rheme
It sets coordinate tracking and locks the motion state of the fuel filling taper sleeve, further includes:
It blocks if it exists, then predicts the optimal location of the fuel filling taper sleeve.
10. the air refuelling method according to claim 1 based on machine vision, which is characterized in that measure institute as the following formula
It states the center of fuel filling taper sleeve and resolves the position coordinates of the fuel filling taper sleeve:
Wherein, L is fuel filling taper sleeve scale model size, and f is camera focus, and l is pixel shared by fuel filling taper sleeve image in plane
Number, Z are distance of the zero point of reference frame where camera to fuel filling taper sleeve place plane.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811413048.XA CN109636853A (en) | 2018-11-23 | 2018-11-23 | Air refuelling method based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811413048.XA CN109636853A (en) | 2018-11-23 | 2018-11-23 | Air refuelling method based on machine vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109636853A true CN109636853A (en) | 2019-04-16 |
Family
ID=66069040
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811413048.XA Pending CN109636853A (en) | 2018-11-23 | 2018-11-23 | Air refuelling method based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109636853A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111123982A (en) * | 2019-12-31 | 2020-05-08 | 中国航空工业集团公司沈阳飞机设计研究所 | Unmanned aerial vehicle air refueling simulation test device |
| CN112862862A (en) * | 2021-02-10 | 2021-05-28 | 中国飞行试验研究院 | Airplane autonomous oil receiving device based on artificial intelligence visual tracking and application method |
| CN114771849A (en) * | 2022-04-15 | 2022-07-22 | 中国航空工业集团公司沈阳飞机设计研究所 | Active control hose air refueling system and aircraft with same |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105825505A (en) * | 2016-03-14 | 2016-08-03 | 北京航空航天大学 | Vision measurement method facing boom air refueling |
| CN106052650A (en) * | 2016-07-21 | 2016-10-26 | 中国人民解放军空军工程大学 | Drogue spatial attitude measuring method and system |
| CN106251337A (en) * | 2016-07-21 | 2016-12-21 | 中国人民解放军空军工程大学 | A kind of drogue space-location method and system |
-
2018
- 2018-11-23 CN CN201811413048.XA patent/CN109636853A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105825505A (en) * | 2016-03-14 | 2016-08-03 | 北京航空航天大学 | Vision measurement method facing boom air refueling |
| CN106052650A (en) * | 2016-07-21 | 2016-10-26 | 中国人民解放军空军工程大学 | Drogue spatial attitude measuring method and system |
| CN106251337A (en) * | 2016-07-21 | 2016-12-21 | 中国人民解放军空军工程大学 | A kind of drogue space-location method and system |
Non-Patent Citations (3)
| Title |
|---|
| 周晓达 等: "基于Kalman预测的空中加油锥套跟踪方法", 《数据采集与处理》 * |
| 支健辉 等: "CamShift 在加油锥套识别跟踪中的应用", 《飞行力学》 * |
| 王旭峰 等: "一种基于 HSV 色彩空间的加油锥套特征提取方法", 《计算机应用与软件》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111123982A (en) * | 2019-12-31 | 2020-05-08 | 中国航空工业集团公司沈阳飞机设计研究所 | Unmanned aerial vehicle air refueling simulation test device |
| CN111123982B (en) * | 2019-12-31 | 2023-04-14 | 中国航空工业集团公司沈阳飞机设计研究所 | Unmanned aerial vehicle air refueling simulation test device |
| CN112862862A (en) * | 2021-02-10 | 2021-05-28 | 中国飞行试验研究院 | Airplane autonomous oil receiving device based on artificial intelligence visual tracking and application method |
| CN112862862B (en) * | 2021-02-10 | 2023-11-17 | 中国飞行试验研究院 | Aircraft autonomous oil receiving device based on artificial intelligence visual tracking and application method |
| CN114771849A (en) * | 2022-04-15 | 2022-07-22 | 中国航空工业集团公司沈阳飞机设计研究所 | Active control hose air refueling system and aircraft with same |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11151741B2 (en) | System and method for obstacle avoidance | |
| RU2609434C2 (en) | Detection of objects arrangement and location | |
| JP5480914B2 (en) | Point cloud data processing device, point cloud data processing method, and point cloud data processing program | |
| CN107063261B (en) | Multi-feature information landmark detection method for precise landing of unmanned aerial vehicle | |
| JP5671281B2 (en) | Position / orientation measuring apparatus, control method and program for position / orientation measuring apparatus | |
| CN109636853A (en) | Air refuelling method based on machine vision | |
| CN103697855B (en) | A kind of hull horizontal attitude measuring method detected based on sea horizon | |
| Dusha et al. | Attitude estimation for a fixed-wing aircraft using horizon detection and optical flow | |
| CN108765489A (en) | A kind of pose computational methods, system, medium and equipment based on combination target | |
| JP2016197287A (en) | Information processing apparatus, information processing method, and program | |
| EP3155369B1 (en) | System and method for measuring a displacement of a mobile platform | |
| EP3032818A1 (en) | Image processing device and markers | |
| JP2015184767A (en) | Information processor, information processing method, position attitude estimation device and robot system | |
| US11132802B2 (en) | Method of detecting moving objects from a temporal sequence of images | |
| CN107592922A (en) | Method for implementing operation to ground | |
| CN109597086A (en) | A kind of motion measuring method of the outer hanging object of contactless helicopter | |
| CN109060290A (en) | The method that wind-tunnel density field is measured based on video and Sub-pixel Technique | |
| Martínez et al. | Towards autonomous air-to-air refuelling for UAVs using visual information | |
| JP5976089B2 (en) | Position / orientation measuring apparatus, position / orientation measuring method, and program | |
| CN105447431A (en) | Docking airplane tracking and positioning method and system based on machine vision | |
| Del Pizzo et al. | Reliable vessel attitude estimation by wide angle camera | |
| JP2010009236A (en) | Plane area estimation device and program | |
| Vera et al. | Determination of the center of radial distortion for a camera lens | |
| CN112862862B (en) | Aircraft autonomous oil receiving device based on artificial intelligence visual tracking and application method | |
| CN104658004B (en) | A kind of air refuelling auxiliary marching method based on video image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190416 |