KR101193445B1 - Automatic scene tracking system for air photograph - Google Patents

Abstract

PURPOSE: An automatic background tracking apparatus for air photograph is provided to stably take a photograph under the movement of an aircraft or disturbance conditions due to the movement of the aircraft when photographing a specific position on the ground in the aircraft. CONSTITUTION: An automatic background tracking apparatus for air photograph comprises a camera(10), an automatic background tracking part(20), a manual operating part(40), an automatic and manual integration part(50), a camera operation control part(60), and a camera operating part(70). The camera takes a photograph targets on the ground. The automatic background tracking part outputs the parallel moving value of a current image receiving from the camera and a past image before a first frame. The manual operating part manually operates the view of the camera. The automatic and manual integration part outputs azimuth and altitude signals for operating the camera. The camera operation control part generates camera operating signals according to the azimuth and altitude signals outputted from the automatic and manual integration part. The camera operating part is mechanically connected to the camera to move the view of the camera according to the operating signals outputted from the camera operation control part. [Reference numerals] (10) Camera; (100) Image stabilizing part; (20) Automatic background tracking part; (30) Image output part; (40) Manual operating part; (50) Automatic and manual integration part; (50b) Second integrator; (50d) First integrator; (60) Camera operation control part; (70) Camera operating part; (80) Motion detecting part; (90) Motion compensating part; (AA) Transverse component; (BB) Longitudinal component; (CC) Altitude angle; (DD) Longitudinal component; (EE) Azimuth angle; (FF) Altitude angle; (GG) Azimuth angle; (HH) Transverse component; (II) Operator

Description

Automatic background tracking device for aerial photography {AUTOMATIC SCENE TRACKING SYSTEM FOR AIR PHOTOGRAPH}

The present invention uses the ground image obtained from the camera mounted on the aircraft for aerial photography that can stably track the background (SCENE), such as buildings and terrain on the ground even if the aircraft moves, and also manually adjust the target background that the operator wants to track. Relates to an automatic background tracking device.

When shooting a specific point on the ground by using a camera in the aircraft, you can get a stable image by tracking the scene such as stationary buildings or terrain on the ground.

Like this, unlike the conventional image tracking that tracks a specific target, the SCENE tracker divides the input camera image into appropriate areas to extract feature points for each area, and then horizontally moves the feature points in the next input image. After calculating the displacement, the camera's gaze is driven by the displacement to track the entire background.

On the other hand, when automatically tracking the ground target background using the image obtained from the camera, the operator may track the unwanted place due to the error of the tracking signal processing. In addition, during the tracking, the operator may need to track another place by changing the line of sight of the camera.

When the above two situations occur, the operator must move the gaze of the camera through manual operation. The manual operation method includes a method of blocking automatic tracking using a switch and performing a manual operation. However, this method may be difficult to precisely operate manually because the line of sight of the camera moves along with the linear movement or rotational movement during the manual operation time.

However, if you change the target point at the same time during the automatic tracking, you can operate the system conveniently.

SUMMARY OF THE INVENTION An object of the present invention is to provide an automatic background tracking apparatus for aerial photography, which is capable of automatic and manual simultaneous adjustment in order to photograph a background such as a stationary building, a bridge or a terrain on the ground using a camera mounted on an aircraft.

Another object of the present invention is to provide an automatic background tracking apparatus for aerial photography, which can easily perform background tracking offset correction and camera gaze change when performing background tracking automatically and manually.

In order to achieve the above object, an aerial background tracking apparatus for aerial photography according to the present invention includes a camera shooting a ground target; An automatic background tracking unit for outputting a parallel shift value between the two images using the current image received from the camera and a past image one frame earlier; Manual operation unit for the operator to manually manipulate the gaze of the camera; An automatic / manual integration unit for outputting an azimuth and elevation by integrating the horizontal and vertical components of the automatic background tracking unit and the manual operation unit when the gaze of the camera is changed through the manual operation unit while the automatic background tracking unit is performed; A camera driving controller configured to generate a camera driving signal (a gaze movement signal) according to the azimuth and elevation signals output from the automatic / manual integration unit; And a camera driver mechanically connected to the camera to move the gaze of the camera according to a camera driving signal output from the camera driving controller.

The parallel shift value of the image is obtained by using the position of the maximum value of cross-correlation between two consecutive frame images.

The automatic background tracking unit may include an image storage unit for storing a current image photographed by a camera and a past image one frame before the current view point; And a parallel movement measuring unit configured to calculate a parallel movement value between current and past images stored in the image storage unit and output the divided image into horizontal and vertical components.

The automatic / manual integration unit divides the output of the automatic background tracking unit and the output of the manual operation unit into horizontal and vertical components.

The automatic / manual integration unit includes: a first adder for adding horizontal component outputs of the automatic background tracking unit and the manual operation unit; A first integrator configured to output an azimuth angle by integrating the output of the first adder; A second adder for adding the vertical component outputs of the automatic background tracking unit and the manual operation unit; And a second integrator for integrating the output of the second adder and outputting an elevation angle.

The first and second integrators convert a parallel shift value between adjacent frame images output from the automatic background tracking unit into a parallel shift value between the first frame image and the current frame image.

A motion detector connected to the camera to detect an angular velocity movement and disturbance introduced by the movement of the aircraft; And a motion compensator for outputting an azimuth and elevation signal to the camera driving controller for compensating the gaze of the camera to stably track the ground target by using the sensed information of the motion detector.

According to the present invention, when shooting a specific point of the ground in the aircraft, it is possible to stably shoot even under the maneuver of the aircraft or disturbance accordingly, and the operator can easily correct the offset by manual operation while watching the tracking image even during the shooting of the ground target point. This has the effect of directing the camera's attention to another target point.

1 is a block diagram of an automatic background tracking device for aerial photography according to an embodiment of the present invention.
2 is a detailed configuration diagram of an automatic background tracking unit.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

According to the present invention, when taking a fixed background such as a building or terrain on the ground by using a camera mounted on an aircraft, the background tracking is performed so that the target point can be stably photographed, and the operator has an error due to an error in image signal processing during tracking. Solves the problem that occurs when the user wants to track where he or she doesn't want or to change the camera's line of sight during background tracking.

1 is a block diagram of an automatic background tracking device for aerial photography according to the present invention.

As shown in FIG. 1, the automatic background tracking apparatus for aerial photography is an automatic background tracking for generating a signal for automatically tracking a background image using a camera 10 photographing a ground target and an image received from the camera 10. The unit 20, the screen output unit 30 for displaying the camera image for the operator to view, and the operator can operate to move the gaze of the camera 10 while viewing the image of the screen output unit 30 A manual operation unit 40, an automatic / manual integration unit 50 for integrating the output of the automatic background tracking unit 20 and the output of the manual operation unit 40 using an adder and an integrator, and an automatic / manual integration unit The camera driving control unit 60 generates a camera driving signal (a gaze movement signal) by using the output value of 50, and the camera driving signal that is mechanically connected to the camera 10 and output from the camera driving control unit 60. Follow the eyes of the camera 10 It includes a camera driver 70 for moving.

In addition, the aerial background tracking device for aerial photography is mechanically connected to the camera 10, the motion detection unit 80 for detecting the angular velocity movement and disturbance flowing into the camera 10 in accordance with the movement of the aircraft, and the motion detection unit The motion compensation unit 90 outputs a compensation value (azimuth angle and elevation angle) for compensating the gaze of the camera 10 to be stabilized by using the sensed information of 80 and the screen output unit 30 by relieving the shaking of the camera image. It further comprises an image stabilization unit 100 for outputting.

2 is a detailed block diagram of the automatic background tracking unit 20.

As shown in FIG. 2, the automatic background tracking unit 20 includes an image storage unit 21 for storing a current image photographed by the camera 10 and a past image one frame before the current view point, and the image storage unit. And a parallel movement measuring unit 22 for dividing and outputting a parallel movement value between the two images in the horizontal and vertical directions by using the current and past images stored in 21.

Referring to the operation of the automatic background tracking device for aerial photography according to an embodiment of the present invention configured as described above are as follows.

When the moving aircraft tracks a target (target) on the ground, the image captured by the camera 10 is displayed on the screen output unit 30. At this time, the automatic background tracking unit 20 calculates a parallel movement value between the two images using the current image output from the camera 10 and a past image input and stored one frame before the current viewpoint. The calculated parallel movement value is divided into horizontal and vertical components and input to the automatic / manual integration unit 50.

The operator may check the image captured by the camera 10 through the screen output unit 80 to adjust the line of sight (direction) of the camera 10 when tracking the current unwanted situation or wants to track another place. . That is, the operator may change the line of sight (direction) of the camera 10 in a direction desired by the manipulator 40. In this case, the output of the manual operation unit 90 is divided into horizontal and vertical components and is input to the automatic / manual integration unit 100. When tracking ground targets with cameras on a moving aircraft, automatic tracking and manual tracking situations occur.

The automatic / manual integration unit 50 is a part that integrates the output (tracking amount) of the automatic background tracking unit 20 and the manual operation unit 40 when an automatic tracking and manual tracking situation occurs, and the automatic background tracking unit 20 ) And the output of the manual operation unit 40 is divided into horizontal and vertical directions to integrate.

That is, the first adder 50a of the automatic / manual integration unit 50 adds the horizontal direction among the outputs of the automatic background tracking unit 20 and the horizontal direction among the outputs of the manual operation unit 40, and the initial value is zero. The first integrator 50b integrates the output of the first adder 50a and outputs an azimuth component. In addition, the second adder 50c of the automatic / manual integration unit 50 adds the vertical direction of the output of the automatic background tracking unit 20 and the vertical direction of the output of the manual operation unit 40, and the initial value is zero. The second integrator 50d integrates the output of the second adder 50c and outputs a high angle component.

At this time, the role of the first and second integrators 50b and 50d is to convert the parallel movement value between two adjacent frame images output from the automatic background tracking unit 20 into a parallel movement value between the first frame image and the current frame image. It plays a role.

When this is explained using a formula, the vertical component (wherein the unit is a pixel) of the position where a fixed point of the ground forms on the image pickup device of the camera 10 in the nth frame is referred to as G [n], and in the nth frame Assume that the output in the vertical direction output from the automatic background tracking unit 20 is A [n]. When A [n] is expressed by a formula relating to G [n], the following equation (1) is obtained.

   If the output of the manual operation unit 40 is M [n] in the nth frame, the output Add [n] of the second adder 50c in the nth frame is expressed by Equation 2 below.

Then, when the output Add [n] of the second adder 50c is accumulated in the second integrator 50d, the output I [n] of the second integrator 50d in the nth frame is expressed by the following equation (3). .

However, in Equation 3, A [1] = 0.

In Equation 3, G [1] -G [n] is a vertical component of the coordinates of the fixed point of the ground formed on the image pickup device of the camera 10 in the first frame from the start of the automatic background tracking, and the current frame. One fixed point of the ground means a difference in the vertical component among the coordinates formed on the image pickup device of the camera. In other words, it is the vertical difference between the target value (corresponding to G [1]) and the current value (corresponding to G [n]). The value is transmitted to the camera driving controller 60 so that the current value reaches the target value. Do it.

In order to obtain G [1] -G [n] in Equation 3, the current image and the first frame image are used without using the current image and the image before 1 frame from the present as in the automatic background tracking unit 20. You can think of how to do it. However, the latter method has a disadvantage in that the background tracking may fail when the viewpoint change due to the movement of the aircraft is large. For example, if a camera mounted on an aircraft tracks a building on the ground, the front of the building will be on the camera in the first frame, but the back of the building may be on the camera in the current frame as the aircraft moves. In this case, the image tracking may fail because the correlation between the image in front of the building and the image in the back of the building is low. However, when the current image and the image before 1 frame from the present are used, the correlation between the two images is very high, thus enabling accurate image tracking.

는 수동 조작에 의한 오프셋으로서, 상기 G[1]-G[n]에 더해져 목표값이

로 변경되도록 한다. 예를 들어 운용자가 수동 조작을 통해 10프레임에 해당하는 시간 동안 1이라는 명령을 내리면, 목표값은 G[1]+10이 된다. 이에 따라 목표값에 10화소의 오프셋이 발생하여 자동 배경 추적부(20)의 오차를 보상하거나 초기 목표와 다른 위치를 추적할 수 있게 된다. Also in Equation 3

Is an offset by manual operation. The target value is added to G [1] -G [n].

To be changed. For example, if the operator gives a command of 1 for 10 frames by manual operation, the target value is G [1] +10. Accordingly, an offset of 10 pixels is generated in the target value, thereby compensating for the error of the automatic background tracking unit 20 or tracking a position different from the initial target.

= 카메라의 세로 시계/카메라의 세로 화소수). 이때, 상기 Kv는 세로 방향의 단위 화소당 카메라의 시계를 의미한다. In order to convert the output of the pixel unit of the second adder 50c into the output of the angle unit, Kv is multiplied by Equation 3 (

= Vertical clock of the camera / vertical pixels of the camera). In this case, Kv means the camera's field of view per unit pixel in the vertical direction.

Therefore, the elevation output D [n] of the automatic / manual integration unit 50 in the nth frame may be represented by Equation 4 as follows.

In Equation 4, D [n] is the sum of the elevation angle of the camera's gaze and the accumulated operator's manual operation value in order to track the image of the first frame from the start of automatic background tracking in the current camera gaze. Indicates.

(

=카메라 영상의 가로 시계/카메라 영상의 가로 화소수)가 곱해진다. 이때, 상기

는 가로 방향의 단위 화소당 카메라 영상의 시계를 의미한다.In the horizontal direction, the same operations as in Equations 1, 2, 3, and 4 are performed through the first adder 50a and the first integrator 50b in the horizontal direction, and the output of the pixel unit of the first adder 50a is angled. In order to convert to unit output,

(

= The horizontal clock of the camera image / the number of horizontal pixels of the camera image) are multiplied. At this time,

Denotes a clock of a camera image per unit pixel in a horizontal direction.

When the automatic / manual integration unit 50 outputs the azimuth and elevation, the motion detector 80 detects and outputs the distance between the camera and the target, the attitude of the camera, the linear speed and the angular velocity of the camera. The motion compensator 90 outputs an azimuth and elevation angle for compensating the gaze of the camera 10 to be stabilized (tracking the target background) by using the sensed value. The value is transmitted to the camera driving control unit 60 together with the output of the automatic / manual integration unit 50 to transmit a driving command to the camera driving unit 70. Therefore, the camera driver 70 outputs a mechanical torque to move the eyes of the camera 10 by the output of the camera drive controller 60.

As described above, the present invention allows automatic image tracking to operate during manual operation of an operator in an environment in which movement of a camera's gaze mounted to an aircraft occurs due to the movement of the aircraft. This method compensates the movement of the camera's gaze due to the movement of the aircraft during the manual operation of the operator without expensive navigation equipment, and when the camera's gaze is directed to a specific position using the navigation equipment, The camera driving command calculation error due to the distance error between the aircraft and the target can be reduced.

The above-described automatic background tracking apparatus for aerial photography is not limited to the configuration and method of the embodiments described above, but the embodiments may be selectively or partially all of the embodiments so that various modifications may be made. It may be configured in combination.

10: camera 20: automatic background tracking unit
30: screen output unit 40: manual operation unit
50: automatic / manual integrated unit 60: camera drive control unit
70: camera driver 80: motion detection unit
90: motion compensation unit 100: video stabilization unit

Claims (13)

A camera for shooting ground targets;
An automatic background tracking unit for outputting a parallel shift value between the two images using the current image received from the camera and a past image one frame earlier;
A manual operation unit for manually operating a gaze of a camera;
If the operator moves the camera's eye through the manual control unit while the auto background tracking unit is performing, the azimuth and elevation signals for driving the camera are output by integrating the horizontal and vertical components of the auto background tracker and the manual control unit output. Automatic / manual integration unit;
A camera driving controller configured to generate a camera driving signal according to the azimuth and elevation signals output from the automatic / manual integration unit; And
And a camera driver mechanically connected to a camera to move the gaze of the camera according to a camera driving signal output from the camera driving controller. The method of claim 1, wherein the parallel shift value of the image is
An automatic background tracking device for aerial photography, wherein the image is obtained using the position of the maximum value of cross-correlation between two consecutive frame images. The method of claim 1, wherein the automatic background tracking unit
An image storage unit for storing a current image captured by the camera and a past image one frame before the current view; And
An automatic background tracking apparatus for aerial photography, comprising: a parallel movement measuring unit configured to calculate and output parallel movement values between current and past images stored in an image storage unit and divide the horizontal and vertical components. The method of claim 1, wherein the automatic / manual integration unit
An automatic background tracking device for aerial photography, characterized by integrating the output of the automatic background tracking unit and the output of the manual control unit into horizontal and vertical components. The method of claim 1, wherein the automatic / manual integration unit
A first adder for adding the horizontal component outputs of the automatic background tracking unit and the manual operation unit;
A first integrator configured to output an azimuth angle by integrating the output of the first adder;
A second adder for adding the vertical component outputs of the automatic background tracking unit and the manual operation unit; And
And a second integrator for integrating the output of the second adder and outputting an elevation angle. The method of claim 5, wherein the first and second integrator
An automatic background tracking device for aerial photography, characterized in that the parallel movement value between adjacent frame images output from the automatic background tracking unit is converted into a parallel movement value between the first frame image and the current frame image. According to claim 1, Mechanically connected to the camera, the movement detecting unit for detecting the angular velocity movement and disturbance flowing into the camera according to the movement of the aircraft; And
Automatic background tracking for aerial photography, further comprising a motion compensation unit for outputting an azimuth and elevation signal to the camera drive control unit for compensating the gaze of the camera to track the ground target stably using the detection information of the motion detection unit. Device. A camera for shooting ground targets;
An automatic background tracking unit for automatically tracking an image received from a camera and outputting a parallel shift value between a current image and a previous image one frame ago;
A manual operation unit for manually moving the gaze of the camera;
If the operator moves the camera's eye through the manual control unit while auto background tracking is performed through the auto background tracking unit, the azimuth and elevation signals are output to integrate the horizontal and vertical components of the auto background tracker and the manual control unit output. Automatic / manual integration unit;
And a camera driving controller configured to generate a camera driving signal for moving the gaze of the camera according to the azimuth and elevation signals output from the automatic / manual integration unit, wherein the automatic / manual integration unit
A first adder for adding the horizontal component outputs of the automatic background tracking unit and the manual operation unit;
A first integrator configured to output an azimuth angle by integrating the output of the first adder;
A second adder for adding the vertical component outputs of the automatic background tracking unit and the manual operation unit; And
And a second integrator for integrating the output of the second adder and outputting an elevation angle. The method of claim 8, wherein the parallel shift value of the image is
An automatic background tracking device for aerial photography, wherein the image is obtained using the position of the maximum value of cross-correlation between two consecutive frame images. The method of claim 8, wherein the automatic background tracking unit
An image storage unit for storing a current image captured by the camera and a past image one frame before the current view;
An automatic background tracking device for aerial photography, comprising: a parallel movement measuring unit configured to calculate a parallel movement value between current and past images stored in an image storage unit and divide the output into horizontal and vertical directions. The method of claim 8, wherein the automatic / manual integration unit
An automatic background tracking device for aerial photography, characterized by integrating the output of the automatic background tracking unit and the output of the manual control unit in the horizontal and vertical directions. The method of claim 8, wherein the first and second integrator
An automatic background tracking device for aerial photography, characterized in that the parallel movement value between adjacent frame images output from the automatic background tracking unit is converted into a parallel movement value between the first frame image and the current frame image. According to claim 8, Mechanically connected to the camera, the movement detecting unit for detecting the linear velocity movement and disturbance flowing into the camera in accordance with the movement of the aircraft;
A motion compensator for outputting azimuth and elevation signals to the camera driving controller for compensating the gaze of the camera to track the ground target stably using the sensing information of the motion detector; and
And a camera driver mechanically connected to the camera to move the gaze of the camera according to the camera driving signal output from the camera driving controller.

Images