TW201317544A - Ground target geolocation system and method - Google Patents

Abstract

A ground target geolocation system operates with an aerial vehicle. When a user selects a ground target from aerial image streams transmitted by the aerial vehicle via a user selection device, the user transmits a selected region containing the ground target to a selected-target image tracking module for tracking the selected region. After having obtained each piece of information of the aerial vehicle's location and attitude per time interval, the system eliminates time difference for the aerial vehicle's attitude information through an image and sensor data sequence synchronization module. A ground target coordinate estimation module computes a real-time geolocation for the ground target, according to the moving trajectory in the tracked aerial images for the ground target and the modified aerial vehicle's attitude information.

Description

Ground target positioning system and method

The disclosure relates to a ground target geolocation system and method.

The intelligent monitoring system has gradually emerged. One of the research and developments that have arisen in recent years is how to set up cameras to view ground photography and analyze image detection events by high-altitude vehicles (such as hot air balloons, drones, etc.) to assist the ground monitoring system. Detection of a wide range of areas, to achieve comprehensive monitoring without dead ends. The application of the aerial video streaming of the drone is, for example, limited to image processing that does not locate the aerial image stream, or integrates the drone image with the ground coordinates to achieve more accurate positioning of the ground target point. The accuracy of the sensing instruments that can be carried on the current aircraft is insufficient, and the aircraft is too small to be affected by the wind, so numerical methods are used to enhance the estimation results. Common related technologies are, for example, a geographic location estimation method based on no-load sensor information, or a geographic location estimation method based on image tracking to improve the accuracy of flight attitude parameters, or a geographic location estimation method based on image template comparison. .

The geolocation estimation method based on the information of the airborne sensor uses the altitude and attitude information provided by the Global Positioning System (GPS) coordinates of the aircraft and the barometer, gyroscope, compass, etc., and calculates the relationship based on the trigonometric function. The position coordinates of the ground target. This technology can be used to make the scope of observation wider. The Recursive Least Squares (RLS) algorithm can also be used to process the raw estimates to obtain more accurate results. This technique improves the accuracy of flight attitude parameters.

The geolocation estimation method based on image tracking to improve the accuracy of flight attitude parameters uses a high-precision sensor, and uses the ground image feature point tracking results and GPS coordinate information to operate the equation according to the Inertial Navigation System (INS). Revise the flight attitude parameters. The geographic location estimation method based on the image template comparison uses the vertically photographed aircraft to obtain the aerial image stream, and compares the aerial image stream of the target area with the ground map of the marked coordinates with a large calculation amount. Recently, the coordinate to which the image of the image belongs belongs to the ground coordinate of the target area. For example, the ground aerial picture with a higher flying height is used as the comparison map.

In the literature related to the above technology, for example, one of the techniques of the literature uses two unmanned aerial vehicles, and the higher-altitude unmanned aerial vehicle is used as a reference point for the lower-altitude aircraft, and the lower-altitude aircraft can use this reference point to improve Positioning accuracy for ground targets. One of the techniques of the literature improves the accuracy of aerial positioning by projecting structured light to the ground as a reference for correcting distortion of an aerial camera. One of the techniques of the literature speeds up the speed of positioning by air, calculates the time required for positioning of all ground target points that can be used for positioning, and then sets the final set of ground positioning targets by means of weight distribution. The technique of one of the documents uses the coordinates carried in the aerial photograph to identify a specific area as a road area and corresponds to the road name, and analyzes the image of the road area in the aerial photograph to obtain traffic information.

In the application technology of aerial video streaming, how can it not need to use any ground map, do not need to accurately measure the height of the aircraft from the ground, the degree of influence by the surface texture area is low, and the degree of influence by external communication conditions is low, In a variety of different search and monitoring scenarios, it is possible to estimate the geographical location of the ground target in real time, and can significantly improve the accuracy of the ground coordinates, which is worthy of research and development.

The disclosed embodiments may provide a ground target positioning system and method.

One disclosed embodiment relates to a ground target positioning system that operates in conjunction with an aircraft. The aircraft captures and returns the captured aerial image stream and its position and attitude data. The system comprises a user pointing device, an image temporary storage area, a point selection target image tracking module, an image and sensor data sequence synchronization module, and a ground target coordinate estimation module. This image temporary storage area stores the aerial image stream. Through the user clicking device, the ground user selects a ground target from the aerial image stream, and transmits a selected area containing the ground target to the selected target image tracking module to track the point. Select the area to get the trajectory of the ground target in the aerial video stream. After obtaining the attitude data of one aircraft in each time interval, the image and sensor data sequence synchronization module uses the movement trajectory of the ground target to eliminate the time difference of the aircraft attitude information. The ground target coordinate estimation module uses the movement trajectory of the ground target to select images of a plurality of sets of aircraft attitude values within an error range from the image containing the ground target and estimate a geographical location of the ground target.

One disclosed embodiment relates to a ground target positioning method that operates in conjunction with an aircraft. The method comprises: the aircraft transmitting the aerial image stream and the flight information of the aircraft by wireless transmission; the user selects a ground target from the returned aerial image stream through a user clicking device; tracking the Clicking on the area to obtain the movement trajectory of the ground target in the aerial image stream; after obtaining the flight information of one aircraft in each time interval, the flight information is smoothly interpolated to the frame of the aerial image, And using the moving trajectory of the ground target to synchronize the aerial image with the flight information sequence of the aircraft; and using the moving trajectory of the ground target, selecting a plurality of sets of aircraft attitude values from an error in the image containing the ground target A range of images and an estimate of the geographic location of the ground target.

The above and other advantages of the present invention will be described in detail below with reference to the following drawings, detailed description of the embodiments, and claims.

In an embodiment of the ground target positioning system and method of the present disclosure, real-time geographic location estimation is performed based on sensor information and combined image feature point tracking. This embodiment does not use any ground map, does not need to accurately measure the height of the aircraft from the ground, is less affected by the surface texture area, and is less affected by external communication conditions, and can be applied in many different search and monitoring scenarios. For example, air search and rescue, disaster investigation, agricultural and fishery observation, resource research, soil and water conservation, border patrol, etc. The first figure is a schematic diagram illustrating an application scenario example of the disclosed embodiment.

In the application scenario example of the first figure, a camera is mounted on an aircraft 110 to monitor a range of areas 115, and the captured image stream is streamed with an airborne sensor, such as a GPS or a gyroscope. The instrument, accelerometer, magnetic compass, barometer, etc., the sensed flight information is simultaneously transmitted back to a ground control station 120. The ground guiding station 120 selects a ground target from the returned streaming image, and immediately estimates the coordinate position of the ground target for reference of subsequent actions, and may also send a flight control command to let the aircraft 110 The ground target automatically performs fixed-point hover monitoring to get more detailed instant information.

Accordingly, the second figure is a data transmission process of a ground target positioning system according to an embodiment of the present disclosure. Referring to the data transmission process of the second figure, at the aircraft end, for example, an air vehicle 210, an instant image capturing device such as a camera captures the image and streams the captured image back to the ground guiding station. For example, a surface server 220. A no-load sensor 215 on the air vehicle 210 simultaneously transmits the sensed flight information of the aircraft 210 back to the surface server 220 end. The user of the surface server 220 can perform a ground target selection 225. With the ground target and flight information, the surface server 220 then tracks the target area and integrates with the flight information operation 230, and then estimates the ground target coordinates 240. At the surface server 220 end, the ground target point selection 225 can click, for example, a user pointing device to select a ground target on a real-time aerial image stream formed by the video stream; the captured video stream is, for example, Can be stored in an image buffer.

Although integrating the flight attitude of the aircraft, the height of the aircraft from the ground, and the coordinates of the target point in the aerial image, the earth coordinates corresponding to the aerial image, that is, the relative positions of the ground target points, can be estimated. However, the error of each measurement value will cause the estimation error to be too large, especially the height of the aircraft from the ground end is the most accurate estimation accuracy. Moreover, the mechanical characteristics of the no-load sensor, such as an inertial sensor, etc., also affect the accuracy of the flight attitude parameters of the aircraft. In an embodiment of the ground target positioning system of the present disclosure, target image tracking is employed to correct the time delay effect of the flight attitude parameters. Moreover, the target tracking point moving distance is used to replace the height of the aircraft from the ground as the basis for estimating the ground coordinates. In other words, after eliminating the time difference of the attitude information of the aircraft, the moving distance of the ground target is used as the basis for estimating the ground coordinates of the ground target.

For example, according to an embodiment of the present disclosure, a relatively accurate horizontal direction coordinate such as GPS information may be used instead of the ground height that cannot be accurately measured, wherein the plurality of points at different time points and the aircraft pitch angle are close to each other. The ground target image is used to estimate the relative distance between the ground target and the aircraft, and then the information is recorded according to the position of the aircraft to estimate the actual coordinates of the ground target. In practical applications, due to the sensor specifications and wireless transmissions that are used, it is impossible to obtain corresponding flight information in 30 image frames transmitted per second, so the ground target is estimated in the subsequent stage. The appropriate image cannot be selected for calculation at coordinates.

Usually only one message per time interval (for example, one second) is received at the surface server 220 end. Therefore, according to an embodiment of the present disclosure, after acquiring the aircraft information per second, the state of the aircraft in each frame is estimated by smooth interpolation. In an embodiment of the present disclosure, the prediction of the Karman filter is used, and the estimated state is information of an inertial sensor (such as a gyroscope) such as a pitch and a bank. The flight attitude can be smoothly interpolated to each frame of the target area, and then the posture values in each frame after the smooth interpolation are adjusted, so that the aerial image and the sensor data sequence can be synchronized.

Accordingly, the third figure illustrates the operation of each module and each module of a ground target positioning system according to an embodiment of the present disclosure. Referring to the third diagram, the ground target positioning system 300 operates in conjunction with an aircraft 310. The aircraft 310 can capture and return the captured aerial image stream 312 and return its instantaneous location data 314 and posture data 316. The ground target positioning system 300 can include a user pointing device 321, an image temporary storage area 323, a point target image tracking module 325, an image and sensor data sequence synchronization module 327, and a ground target coordinate. Estimate module 329. The image temporary storage area 323 stores the aerial image stream 312.

Through the user pointing device 321, the ground user can select a ground target from the aerial image stream 312, and transmit a point selection area 321a containing the ground target to the click target image tracking module 325. This click area 321a is tracked. In the embodiment of the present invention, feature point detection and optical flow method can be used to track the selected area, and the homography conversion matrix estimation, the Kalman filter tracking and the rough point removal technology can be used for stable and smooth locking tracking. Ground target. A movement trajectory 325a of the ground target in the aerial image stream is obtained.

On the other hand, the ground target positioning system 300, after obtaining a piece of aircraft information for each short time interval, passes through the image and sensor data sequence synchronization module 327 to eliminate the aircraft attitude information time difference and the corrected aircraft attitude information. 327a is transmitted to the ground target coordinate estimation module 329. The aircraft information includes aircraft position data 314 (eg, GPS data) and attitude data 316 (eg, two attitude information of the aircraft, ie, pitch angle and roll angle. The ground target positioning system 300 obtains each time interval (eg, one second) After a piece of aircraft information, an error reduction filter 333 can also be used first to improve the accuracy of the flight attitude parameter (aircraft attitude information), for example, a low-pass filter or a Karman filter can be used to reduce the inertial sensor. Output value error. The aircraft position data 314 and the increased accuracy flight attitude parameters are transmitted to the image and sensor data sequence synchronization module 327. In the disclosed embodiment, the ground target positioning system 300 uses the error. Whether the filter is reduced or not is only optional, and the scope of the disclosure is not limited thereto.

The operations performed by the image and sensor data sequence synchronization module 327 include aircraft pose data smooth interpolation and sensor data correction. The ground target coordinate estimation module 329 uses the movement target 325a of the ground target to select images of a plurality of sets of aircraft attitude values within an error range from the image containing the ground target and estimate a geographical location of the ground target. 329a. The geographic location 329a is a geographic location on the surface that can be labeled using the Datum + Format/Grid parameters. This symbol is referred to as the earth coordinates. The earth reference is, for example, TWD67, TWD97, WGS84, etc., which are often heard. Coordinate formats such as latitude and longitude, UTM (six-degree banding), TM2 (two-degree banding), power coordinates, and the like.

Accordingly, the fourth figure is an operational flow of a ground target positioning method according to an embodiment of the present disclosure. This ground target positioning method works with an aircraft. Referring to the fourth figure, the aircraft transmits the aerial image stream and the flight information of the aircraft by wireless transmission, as shown in step 410. The user selects a ground target from the returned aerial image stream through a user pointing device and selects a point selection region including the ground target, as shown in step 420. Then, the selected area is tracked to obtain the movement trajectory of the ground target in the aerial image stream, as shown in step 430. After obtaining the flight information of one aircraft in each time interval, the flight information is smoothly interpolated to the frame of the aerial image, and the flight information of the ground target is used to synchronize the aerial image with the flight information sequence of the aircraft. , as shown in step 440. Then, through a ground target coordinate estimation module, using the moving trajectory of the ground target, selecting images of a plurality of sets of aircraft attitude values within an error range from the image containing the ground target and estimating the ground target A geographic location, as shown in step 450.

According to the above, after the ground user selects the ground target in the aerial image, the embodiment of the disclosure may adopt the feature point tracking method, first tracking the frame by frame, and then tracking the ground target by the frame backward. Get the complete movement of the ground target in the aerial movie. With this movement trajectory, the time difference caused by the previous smoothing of the aircraft attitude can be eliminated. In the disclosed embodiment, the aircraft roll angle value is first smoothly interpolated to each of the aerial photo frames, and then the time correction is performed on the movement target of the ground target to eliminate the time difference of the posture information, thereby obtaining more accurate posture information (eg, Estimated value of the roll angle).

The fifth figure is a schematic diagram illustrating the smoothing of flight attitude to each aerial photo frame in accordance with an embodiment of the present disclosure. In the fifth figure, the aircraft returns the aerial photo frame 1 to the aerial photo frame 32. In the actual environment, due to the limited sensor specifications and wireless transmission, the ground server end can only receive one piece of information per second, such as As shown in the actual posture 510 in the figure, the actual posture received by the ground server at time t corresponds to the aerial photo frame 1, and the actual posture received at the next time t+1 corresponds to the aerial photo frame 31. Therefore, the flight attitude is smoothly interpolated to each frame as shown by the smoothed posture 520 in the figure.

Since the measurement value of the inertial sensor (such as the gyroscope) has a time delay, and the measurement data is smoothed and interpolated, the time difference between the attitude information and the frame is inevitably generated, and the posture value interpolated to a certain frame is not It will be the attitude of the aircraft at that point in time. In the embodiment of the present disclosure, the trajectory of the previous tracking ground target and the smoothed roll angle value change can be used for matching, so that the time difference of the attitude information can be eliminated, thereby effectively improving the accuracy of the sensor for estimating the attitude of the aircraft. degree.

In the aerial film, the lateral (horizontal) movement of the ground target point is mainly caused by the roll angle of the aircraft. The sixth figure is a schematic diagram illustrating the matching of the movement trajectory of the ground target with the feature point and the change of the value of the smoothed post-tilt angle according to an embodiment of the present disclosure. Wherein, the x-direction movement (Δx) in the ground target movement trajectory is matched with the roll angle attitude value change (Δbank) after the Kalman filter is smoothed. This movement track contains 8 aerial photo frames, the symbol "→" represents the ground target moving to the right in the x direction, and the symbol "←" represents the ground target moving to the left in the x direction. The symbol "+" represents the increase in the value of the flight roll attitude of the aircraft, that is, the ground target is shifted to the right in the horizontal direction; otherwise, the symbol "-" represents the decrease in the value of the flight roll attitude of the aircraft, that is, the ground target is horizontally left. shift. It can be seen from the ground target point movement Δx and the corrected roll angle attitude value in the sixth figure that the flight image of the ground target is matched with the numerical value of the roll angle after smooth interpolation, and the aerial image and the flight of the aircraft are obtained. The information sequence is synchronized, thus eliminating the time difference of the gesture information.

As previously mentioned, an embodiment of the ground target positioning system and method operates with an aircraft and uses the target tracking point moving distance to replace the aircraft ground height as the basis for estimating the ground coordinates. In the disclosed embodiment, the aircraft is provided with an aerial image capturing device for real-time image capturing, a GPS sensor for instantaneous acquisition of the position data of the aircraft, and an inertial sensor (such as a gyroscope) to acquire the aircraft. Gesture information. The design of the estimated ground coordinates is illustrated in the seventh and eighth figures.

When the aerial image capturing device (such as a camera) is fixed at a certain pitch angle, the image coordinate value of each point in the camera angle of view can be converted into a height h corresponding to the camera by a perspective conversion method. The ratio value. Therefore, in order to convert the actual image coordinates, according to the embodiment of the present disclosure, the camera position is set to the origin of the earth coordinates, and a horizontal checkerboard is captured. Through the perspective transformation method, a transformation matrix is obtained, and the original squint image can be projected onto the ground. After the projection is completed, according to the actual measurement result, the geodetic coordinates corresponding to the coordinates of each image point in the original image can be obtained.

Accordingly, the seventh figure is a schematic diagram illustrating coordinate conversion (e.g., a camera) in accordance with an embodiment of the present disclosure. The aerial image capturing device (such as a camera) is first fixed at a certain pitch angle, and the squint image 710 is converted into a top view image 720 through the checkerboard and the conversion matrix of the pitch angle is recorded. Then, different pitch angles are changed and the conversion matrix of each pitch angle is recorded. As can be seen from the seventh diagram, the x-axis and y-axis coordinates in the squint image 710 can be projected onto the x-axis and y-axis coordinates in the upper view image 720. That is, the x-axis and y-axis coordinates in the aerial image can be projected to the x-axis and y-axis coordinates of the ground.

When the camera maintains a fixed pitch angle and advances for a distance, the target on the ground is found moving below the screen. By utilizing the characteristic that the amount of movement is equal to the forward distance of the camera, the positional information of the aircraft can be used to replace the height h of the aircraft from the ground, and is used as a basis for estimating the ground target of the present disclosure. 8A and 8B are schematic views illustrating ground target relative distance estimation according to an embodiment of the present disclosure. Taking the eighth A diagram and the eighth B diagram as an example, the eighth diagram A is the initial state of the camera on the aircraft, and the eighth diagram B is the state after the camera on the aircraft advances.

The camera (indicated by a diamond) advances from the position of Figure 8A by 0.4h to the position of Figure 8B, and the direction of advancement of the aircraft is the direction in which the camera travels. After the aerial image is projected as a top view image, the ground target (indicated by a triangle) is moved from the initial state of 1.2 h (as indicated by arrow 810) to 0.8 h (as indicated by arrow 820). The movement of the camera by 0.4 h can be obtained from the position record of the aircraft, so that the height h of the aircraft from the ground can be obtained. The following takes the value of the image y-axis as an example to show how to get the height h of the aircraft from the ground.

When the camera height is h, the y-axis coordinate value in the image can be converted to the y-axis coordinate projected_y projected to the ground plane by:

Projected_y=(a*y+b)*h

Where a, b are the parameters of the conversion and have been pre-measured for cameras of different pitch angles.

If the camera moves horizontally a distance, the ground target in the aerial image will move toward the bottom of the image. Assuming that the y-axis coordinate of the ground target before moving is project_y ₁ and the y-axis coordinate of the ground target after moving is project_y ₂ , the actual height h of the camera can be obtained from the amount of movement of the ground target. In this case the movement of the ground target is equal to the movement of the camera, so

Δ(UAV_loc ₁ , UAV_loc ₂ )=Δ(Projected_y ₁ ,Projected_y ₂ )=(ab)*h

The UAV_loc ₁ is the y-axis position of the camera in the initial state, and the UAV_loc ₂ is the y-axis position of the camera after the movement.

After obtaining the camera height h, the relative distance Target_distance between the ground target and the camera in the horizontal direction can be derived as follows:

Target_distance=Projected_y ₁ =a*h

Then add the relative distance to the y-axis position of the camera in the initial state to obtain the actual y-axis coordinate Target_location of the ground target as follows:

Target_location=UAV_loc ₁ +Target_distance

The x-axis coordinates can also be calculated in the same way. No longer repeat.

Taking the position of FIG. 8A forward 0.4h to the position of the eighth B diagram as an example,

The amount of movement of feature points (1.2h-0.8h)

=0.4h=Camera horizontal movement

= The amount of movement of the feature point of the aircraft moving distance (1.2h-0.8h)

=0.4h=Camera horizontal movement

= aircraft moving distance

The moving distance of the aircraft can be obtained from the horizontal position of the aircraft, so that h can be obtained. After obtaining h, the coordinates of the ground target can be calculated.

In other words, by selecting two images with the same roll angle value and including the ground target, the relative distance between the ground target and the aircraft (when the first image of the two images is captured) can be calculated. The relative distance is then added to the coordinates of the aircraft (such as the GPS coordinates) when the first image is captured, and the coordinates of the ground target can be estimated. In an image containing a ground target, it is also possible to select images of a plurality of sets of roll angle values within an error range and estimate the coordinates of the ground target. The coordinate results estimated by the multiple sets of images are then integrated.

According to the above embodiment, the present disclosure is equipped with an aircraft such as a drone to fly horizontally, and records the film at the ground guiding station and simultaneously records the flight information for experiment. The above-mentioned ground target positioning method is verified by the program, the aerial picture and the flight path are simultaneously displayed, and the ground target in the play picture is clicked, and the coordinate estimation result can be quickly obtained. In the image of the ground target, images of a plurality of sets of roll angle values within a predetermined error range are also selected and the ground target coordinates are calculated, and coordinate results calculated by integrating the plurality of sets of images are integrated, and the experimental results show the integrated coordinates. The result is more stable. The disclosure also repeats the aerial film and clicks on the same ground target to verify the reproducibility of the ground target coordinate estimation. The experimental results show that the ground target coordinate estimation is also reproducible. The accuracy of the ground target coordinate estimation is also compared with the results of using GPS and gyroscope information and estimating the ground point coordinates through a trigonometric function. As can be seen from the example table of the ninth figure, the ground target coordinate estimation of the present disclosure As a result of the measurement, the average error is 2.86 meters, which is significantly more accurate than the estimated 41.54 meters of the trigonometric function.

In summary, the ground target positioning technology of the embodiment of the present disclosure is that after the user selects the ground target in the aerial image, the feature point is tracked forward and backward by the selected frame to obtain a complete target track. . Each of the frames included in the target trajectory is firstly smoothed by smooth interpolation, and then corrected by the movement trajectory of the ground target to obtain a more accurate estimation of the roll angle. Start with the target track start frame and look for two frame combinations with similar roll values. If there are multiple sets of frames with approximately the same roll angle value, any combination of two frames can be included in the calculation. A combination of two frames with similar roll values can estimate the coordinate value of a ground target. Finally, all the estimated results are statistically integrated to obtain a more accurate coordinate of the ground target. Embodiments of the present disclosure integrate all of the frame estimates of the coordinates in an averaging manner.

Therefore, the disclosed embodiment is an instant geographic location estimation technology based on sensor information and image feature point tracking, and the feature includes at least no comparison with any ground map, and is not limited to whether the surface has obvious texture. Area, low-precision measuring instruments can provide sufficient information, not limited to the high or low frequency of aircraft attitude update, low degree of influence by external communication conditions, and not limited to the area of the aircraft to be monitored by the aircraft. The size of the aircraft's on-board image capture device can be used to capture images and the like in a diagonally forward direction.

The above is only the embodiment of the disclosure, and the scope of the disclosure is not limited thereto. That is, the equivalent changes and modifications made by the scope of the present invention should remain within the scope of the present invention.

110. . . Aircraft

115. . . Range area

120. . . Ground control station

210. . . Air vehicle

215. . . No load sensor

220. . . Surface server

225. . . Ground target selection

230. . . Track target area and flight information integration operations

240. . . Ground target coordinates

300. . . Ground target positioning system

310. . . Aircraft

312. . . Video stream

314. . . Location data

316. . . Attitude data

321. . . User selection device

321a. . . Click area

323. . . Image temporary storage area

325. . . Click on the target image tracking module

325a. . . Moving track

327. . . Image and sensor data sequence synchronization module

327a. . . Modified aircraft attitude information

329. . . Ground target coordinate estimation module

329a. . . Instant location

333. . . Error reduction filter

410. . . The aircraft transmits the aerial video stream and the flight information of the aircraft by wireless transmission.

420. . . The user selects a ground target from the returned aerial image stream through a user pointing device and selects a selected area containing the ground target.

430. . . Tracking the selected area to obtain the movement track of the ground target in the aerial image stream

440. . . After obtaining the flight information of one aircraft in each time interval, the flight information is smoothly interpolated to the frame of the aerial image, and the flight information of the ground target is used to synchronize the aerial image with the flight information sequence of the aircraft.

450. . . Using the moving trajectory of the ground target, from the image containing the ground target, select images of a plurality of sets of aircraft attitude values within an error range and estimate a geographical location of the ground target

510. . . Actual posture

520. . . Smoothed posture

710. . . Squint image

720. . . Top view image

h. . . Aircraft height from ground

810. . . Initial state of the ground target

820. . . Ground target movement status

The first figure is a schematic diagram illustrating an application scenario example of the disclosed embodiment.

The second figure illustrates a data transmission process of a ground target positioning system according to an embodiment of the present disclosure.

The third figure illustrates the operation between modules and modules of a ground target positioning system according to an embodiment of the present disclosure.

The fourth figure is an operational flow of a ground target positioning method according to an embodiment of the present disclosure.

The fifth figure is a schematic diagram illustrating each of the frames included in the movement trajectory that smoothes the flight attitude to the ground target in accordance with an embodiment of the present disclosure.

The sixth figure is a schematic diagram illustrating the matching of the movement trajectory of the ground target with the feature point and the change of the value of the smoothed post-tilt angle according to an embodiment of the present disclosure.

The seventh figure is a schematic diagram illustrating camera coordinate conversion in accordance with an embodiment of the present disclosure.

8A and 8B are schematic views illustrating ground target relative distance estimation according to an embodiment of the present disclosure.

The ninth figure is a comparison diagram of the ground target coordinate estimation and the conventional trigonometric function estimation ground point coordinates of the present disclosure.

300. . . Ground target positioning system

310. . . Aircraft

312. . . Video stream

314. . . Location data

316. . . Attitude data

321. . . User selection device

321a. . . Click area

323. . . Image temporary storage area

325. . . Click on the target image tracking module

325a. . . Moving track

327. . . Image and sensor data sequence synchronization module

327a. . . Modified aircraft attitude information

329. . . Ground target coordinate estimation module

329a. . . Instant location

333. . . Error reduction filter

Claims (16)

A ground target positioning system is operated with an aircraft that captures and returns an aerial image stream and its own position and attitude data. The system includes: an image temporary storage area for storing the aerial image Streaming; a user clicking device, through the user clicking device, selecting a ground target from the aerial image stream, transmitting a selected area including the ground target; and selecting a target image tracking module Tracking the selected area to obtain a moving trajectory of the ground target in the aerial image stream; an image and sensor data sequence synchronization module, after obtaining the attitude data of an aircraft in each time interval, Using the movement trajectory of the ground target to eliminate the time difference of the aircraft attitude information; and a ground target coordinate estimation module, using the movement trajectory of the ground target, selecting a plurality of sets of aircraft attitude value gaps from the image containing the ground target An image within the error range and an estimate of the geographic location of the ground target. For example, in the ground target positioning system described in claim 1, the system adopts an error reduction filter to improve the accuracy of the aircraft attitude information, and then transmits the position information of the aircraft and the attitude information of the aircraft with improved accuracy to the aircraft. Image and sensor data sequence synchronization module. The ground target positioning system of claim 1, wherein the image and sensor data sequence synchronization module performs aircraft pose data smooth interpolation and sensor data correction. For example, the ground target positioning system described in claim 1 is characterized in that the target image tracking module uses feature point detection and optical flow method to track the selected area, and cooperates with the homography conversion matrix to estimate and card. Man filter tracking and rough point removal techniques for stable and smooth locking to track the ground target. The ground target positioning system of claim 1, wherein the attitude information of the aircraft includes the pitch angle and the inverted attitude information of the aircraft. The ground target positioning system according to claim 1, wherein the aircraft is equipped with an aerial image capturing device for real-time image capturing, a global positioning system sensor for instantaneously acquiring the position information of the aircraft, and a An inertial sensor to obtain the attitude data of the aircraft. The ground target positioning system according to claim 1, wherein the geographical position is a geographical location on the earth surface, and is marked by two parameters: a geodetic reference and a coordinate format. The ground target positioning system of claim 6, wherein the aerial image capturing device captures an image in a diagonally forward direction. A method for positioning a ground target, which is operated by an aircraft, the method comprising: transmitting, by the aircraft, the aerial image stream and the flight information of the aircraft by wireless transmission; and transmitting the information from the user through a user clicking device Selecting a ground target in the aerial image stream and selecting a selected area containing the ground target; tracking the selected area to obtain a moving track of the ground target in the aerial image stream; After the flight information of the aircraft in the time interval, the flight information is smoothly interpolated to the frame of the aerial image, and the aerial image is synchronized with the flight information sequence of the aircraft by using the movement trajectory of the ground target; The moving trajectory of the ground target selects images of a plurality of sets of aircraft attitude values within an error range from the image containing the ground target and estimates a geographic location of the ground target. The ground target positioning method according to claim 9, wherein the flight information of the aircraft includes posture information and position data of the aircraft. The method for positioning a ground target according to claim 10, wherein the synchronization of the aerial image with the flight information sequence of the aircraft comprises: matching the moving trajectory of the previously tracked ground target with the smoothed roll value change. In order to eliminate the time difference of the attitude of the aircraft. The ground target positioning method according to claim 9, wherein the estimating a geographical location of the ground target comprises: selecting two images having the same roll angle value and including the ground target to calculate two The relative distance between the ground target and the aircraft when the first image of the image is captured; and the geographic location coordinates of the aircraft when the relative image is captured, to estimate the ground target The geographic location. The ground target positioning method according to claim 10, wherein the aircraft is equipped with an aerial image capturing device for real-time image capturing, a global positioning system sensor for instantaneously acquiring the position information of the aircraft, and a An inertial sensor to obtain the attitude data of the aircraft. The ground target positioning method according to claim 13 , wherein the estimating a geographical location of the ground target comprises: fixing the aerial image capturing device at a certain pitch angle, and transmitting a perspective conversion method Converting the image coordinate value of each point in the angle of view of the aerial image capturing device to a ratio corresponding to the height of the space image capturing device; and obtaining the space by the amount of movement of the ground object The distance from the image capture device to the ground. The ground target positioning method according to claim 11, wherein after eliminating the time difference of the attitude information of the aircraft, the method uses the moving distance of the ground target as a basis for estimating the ground coordinates of the ground target. The ground target positioning method according to claim 9, wherein the method uses a geodetic coordinate to indicate that the geographical location is a geographical location on the earth surface, and the earth coordinates are indicated by two parameters: a ground reference and a coordinate format.