KR19980031721A - Moving subject automatic tracking device and method - Google Patents

Abstract

The present invention relates to an apparatus and method for automatically tracking a moving subject to enlarge a subject search area by using a zoom function and an auto focusing function when tracking a subject by moving an imaging direction of a video camera. The present invention includes a zoom / focus controller that performs zooming and automatically focuses according to the state of the limit position sensor installed in the camera support. The CPU determines whether a subject moves by using a motion vector, which is a difference between two consecutive image frames. When the subject moves, the movement vector and the number of motor driving pulses are calculated using the movement vector, and the fan and tilt motor are rotated accordingly to track the subject. At this time, the zooming is performed or the video camera is moved or initialized so as to enlarge the subject search area by checking the state of the limit position sensors. Therefore, even if the movement distance of the subject is large, there is an effect that can be easily tracked without missing the subject.

Description

Moving subject automatic tracking device and method

1 is a block diagram schematically showing the configuration of a moving subject automatic tracking device proposed in the present invention.

Figure 2 is an exemplary view for explaining in more detail the structure of the video camera and the camera support of Figure 1;

3 is a flowchart illustrating a method for automatically tracking a moving subject of the present invention.

4 is a graph for explaining a process of obtaining a moving angle and a displacement of a video camera from a moving vector of a target moving on an image plane [pi].

5 is a state diagram illustrating a position occupied by a subject on a screen when zooming by one step in the wide direction;

6 is a screen state diagram showing an effective area set in this embodiment.

※ Explanation of code for main part of drawing

1: video camera 2: camera support device

6: Zoom / Focus Controller 7: Central Processing Unit

10: Fan motion carrier 13, 14, 17, 18: limit position sensor

12: hole 15: tilt motion carrier

[Objective of the invention]

SUMMARY OF THE INVENTION An object of the present invention is to provide an apparatus and method for automatically tracking a moving subject, by moving an imaging direction of a video camera according to a movement of a subject detected using a moving vector so that the subject is always positioned in front of the camera.

Another object of the present invention is to provide an apparatus and a method for automatically tracking a moving subject, which makes it possible to easily track a subject having a large movement by enlarging the subject search region by using a zoom function and an auto focusing function of a camera.

Still another object of the present invention is to provide an apparatus and method for automatically tracking a moving subject, so that even when the subject moves in a wide range, the subject being tracked can not be missed.

[Technical Field to which the Invention belongs and Conventional Technology in the Field]

The present invention relates to an apparatus for tracking a subject so that the subject is always located in front of the camera. In particular, an apparatus and method for automatically tracking a moving subject to easily track a large-moving subject by enlarging a search area by using a zoom function It is about.

In general, a video telephone or a conference system detects a motion of a subject and transmits video data along with audio data to the other party. In the case of a video telephone, a video camera is installed in the main body to realize this function. However, in the related art, since the camera is fixed to the main body of the phone, when the subject moves, the user has to adjust the position of the main body of the video phone so that the subject is positioned in front of the camera. Or, the user himself sees the movement of the subject through the monitor and moves the video camera fixed to the videophone up and down or left and right by physical force so that the user's appearance is displayed on the monitor.

However, such a self-mode method was very inconvenient to use because the user must adjust the position of the camera to transmit the current movement to the other party. In addition, since the user of the video phone must talk while conscious of the camera, there was a problem that causes inconvenience to the user.

Conventionally, in order to solve this problem, a video telephone which detects a moving direction of a subject and adapts the movement thereof to move the imaging direction of the video camera up or down or left and right. The video phone is equipped with two motors in the video camera, allowing the camera to move up and down and left and right. Then, the motion direction and size of the subject are detected from image data currently input using a commercially available block matching algorithm (Blick Matching Algolithm). When the user moves during a call using a video phone, the video camera moves the image capturing direction up or down or left and right according to the detected direction of movement of the subject so that the subject is always positioned at the center of the screen.

[Technical Challenges to Invent]

The automatic tracking method as described above has the advantage of eliminating the inconvenience that the user has to adjust the position of the camera or call while conscious of the camera to compensate for the problem of the self-mode method. However, since the movement of the subject is tracked by moving the imaging direction of the video camera up and down or left and right, there is a fundamental limitation that the space for tracking the subject is very limited. In other words, if the user moves beyond the limit of camera movement, the subject being tracked is lost. This problem causes a failure in encoding immediately due to the structure of a video telephone that performs encoding using a motion vector.

Composition and Action of the Invention

The moving object automatic tracking device of the present invention for achieving the above objects is equipped with a video motor and a fan motor so that the video camera rotates about one axis and rotates about another axis orthogonal to the axis. A camera support device equipped with a tilt motor is provided. The pan / tilt controller connected to the pan motor and the tilt motor calculates the movement vector of the subject, calculates the rotation angles in the azimuth and elevation directions, and generates the motor driving pulse signal as far as the video camera should be moved. It will control the driving. The zoom / focus controller performs zooming by adjusting the zoom lens according to the output state of the limit position sensor installed in the camera support, and automatically focuses using the maximum luminance component of the image input to the video camera. The central processing unit connected to the pan / tilt controller and the zoom / focus controller initializes the moving object auto tracking device and controls the operation order of the respective function blocks.

In addition, the method for automatically tracking a moving subject of the present invention includes a first process of initializing a moving subject automatic tracking apparatus, and a second process of detecting the first image frame and the next image frame and determining whether the subject is moving by a difference vector. The third step of tracking the subject by calculating the moving distance and the number of motor driving pulses by using the movement vector and rotating the pan motor and the tilt motor according to the process, and checking the output state of the limit position sensors. A fourth process of performing zooming or moving or initializing the video camera to enlarge the search area is performed.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to FIGS. 1 to 6 as follows.

1 is a block diagram schematically illustrating a configuration of an apparatus for automatically tracking a moving subject proposed in the present invention. Reference numeral 1 denotes a video camera which captures an image of a subject and converts the image into an electrical signal, and includes a lens unit including a zoom lens and an automatic focus (AF) lens. The video camera 1 captures an image input through a lens on a CCD (Charge Coupled Device) element and converts it into an electrical signal. A video camera mount (2) mounted with a video camera (1) includes a pan motor (3) and a tilt motor (Tilt Motor) to enable rotational motion of the azimuth and elevation angles of the video camera 1; 4) is provided.

The pan / tilt controller 5 connected to the fan motor 3 and the tilt motor 4 controls the driving of the fan motor 3 and the tilt motor 4 in response to the movement of the subject. To do this, first calculate the movement vector of the subject, calculate the rotation angle of the azimuth and elevation angles of the video camera 1 from the movement vector, and then generate a motor driving pulse signal as far as the video camera 1 should be moved to generate the pan. Output to the motor 3 and the tilt motor (4). Then, the fan motor 3 is driven to rotate the video camera 1 in the left or right direction, and at the same time the tilt motor 4 is driven to move the video camera 1 in the upward or downward direction.

A zoom / focus controller 6 having input / output ends connected to the camera support device 2 and the video camera 1 respectively outputs an end-position sensor installed in the camera support device 2. According to the state, the zoom lens of the video camera 1 is adjusted to perform zooming, zoom-in or zoom-out. Thereafter, the focus is automatically adjusted using the maximum luminance component of the image input to the video camera 1. Here, the limit position sensor is mounted to designate the rotation limit position when the video camera 1 rotates in the left-right (horizontal) or up-down (vertical) direction.

The central processing unit 7 connected to the pan / tilt controller 5 and the zoom / focus controller 6 respectively initializes the moving object automatic tracking device of the present invention and controls the operation order of the respective function blocks.

FIG. 2 is an exemplary diagram for describing in detail the structures of the video camera 1 and the camera support device 2 of FIG. 1. The video camera 1 with the lens unit 8 can rotate about one axis and at the same time rotate about another axis orthogonal to that axis. In the camera support device 2, the portion supporting the rotational movement of the video camera 1 is located at a mount base 9. Here, A means the angle of rotation in the horizontal direction by the fan motor, B means the angle of rotation in the vertical direction by the tilt motor.

In order to support the rotational movement in the azimuth direction (left and right directions), a Pan Motion Carrier 10 is installed on the mount base 9. This fan motion carrier 10 is directly connected to the fan motor 3 via a fan shaft 11. A hole 12 in the fan motion carrier 10 is used to capture the limit angle of rotation with two pairs of photosensors 13 and 14. The photo sensor 13 and 14 including the light emitting part a and the light receiving part b have a pan left-end sensor 13 for detecting a left turn limit of the video camera 1 and a PRE for detecting a right turn limit. (Pan Right-End) sensor 14, which is installed at the rotational limit position. When the holes 12 and the photosensors 13 and 14 of the fan motion carrier 10 are not in a straight line, the light of the light emitting unit a cannot be received by the light receiving unit b. However, when the fan motion carrier 10 is rotated so that the holes 12 and the photosensors 13 and 14 are arranged in a straight line, the light of the light emitting part a may be received by the light receiving part b. You can detect that you have reached.

A tilt motion carrier 15 is installed on the pan motion carrier 10 to support the rotational motion in the elevation angle (up and down direction). This tilt motion carrier 15 is directly connected to the tilt motor 4 via a tilt shaft 16. Although not shown in the drawing, the tilt motion carrier 15 has a hole as well as the fan motion carrier 10, and is used to capture the limit rotation angle together with the photosensors 17 and 18. That is, the tilt upper-end (TUE) sensor 17 detects the upper rotation limit of the video camera 1, and the tilt lower-end (TLE) sensor 18 detects the lower rotation limit. The method of detecting the rotational limit position in the vertical direction by the TUE sensor 17 and the TLE sensor 18 includes a method of sensing the rotational limit position in the left and right directions by the PLE sensor 13 and the PRE sensor 14 described above. same.

3 is an operation flowchart showing a method for automatically tracking a moving subject of the present invention executed by the CPU 7. First, the central processing unit 7 sends an initialization command to the pan / tilt controller 5 and the zoom / focus controller 6 to initialize the moving object automatic tracking device. The pan motion carrier 10 on the camera support device 2 and the tilt motion carrier 15 are directed to the center of the lens unit 8 of the video camera 1 by an initialization command sent to the pan / tilt controller 5. The azimuth and elevation angles are adjusted to 0 [deg.] (S1). In response to the initialization command sent to the zoom / focus controller 6, the video camera 1 initializes the zoom lens and the AF lens to have the minimum focal length (S2).

When the initialization process is completed, the first image frame is detected through the video camera 1 to capture the subject (S3), and the next image frame is detected again (S4) to determine whether there is movement of the subject. In this case, the presence or absence of the motion is determined by the motion vector, and the motion vector is represented by the difference between the first video frame and the next video frame, thereby determining the difference between the two frames before and after the motion (S5). If the motion vector obtained above has a magnitude greater than or equal to the set value, it is assumed that there is motion, and the distance to which the video camera 1 should move is calculated using the motion vector in the pan / tilt controller 5. Here, the moving distance of the video camera 1 is proportional to the moving vector. If the fan motor 3 and the tilt motor 4 are driven by calculating the number of driving pulses according to the moving distance, the pan motion carrier 10 and the tilt motion carrier 15 rotate in accordance with the direction of the movement vector, thereby Correct displacement caused by movement. By this operation, the video camera 1 tracks the subject while moving in the left and right or up and down directions (S6).

4 is a graph for explaining a process of obtaining a moving angle and a displacement of the video camera 1 from the moving vector of the target moving on the image plane π. Moving vector 4, the angle at which the video camera 1 moves left and right or up and down with reference to FIG. 4. Find the following way. First, the movement of the object on the frame may be represented by a distance movement in the x direction or the y direction in the image plane π as shown in FIG. 4. In order to correct the displacement of the point t and the optical axis of the video camera 1 when the position of the object is t on this plane π, the optical axis of the camera 1 is adjusted to φ and θ components. Must be moved. Position on the image plane (π) of the object The equation for converting into φ and θ components is shown in Equation 1 below.

[Equation 1]

here, Is the focal length of the video camera 1, Is the initial optical axis of the video camera 1 Represents a component. Equation 1 can be used to calculate the displacement, and the motion vector between two consecutive frames Initial position of the back video camera (1) Displacement from Is obtained by equation (2).

[Equation 2]

here, Since the AF lens of the video camera 1 focuses on the subject, the moving vector When Equation 2 is obtained, the moving angle of the video camera 1 can be easily obtained using Equation 2.

The pan / tilt controller 5 displaces from the motion vector detected in the previous step S5 using Equation 2 And calculate the number of pulses to be applied to the motor to actually rotate the video camera 1. The fan motor 3 and the tilt motor 4 rotate only by the number of pulses applied to correct the displacement caused by the movement of the subject (S6). When the video camera 1 tracks the movement of the subject, the zoom / focus controller 6 continuously checks the output state of the limit position sensors (PLE sensor, PRE sensor, TUE sensor, TLE sensor) (S7).

When the video camera 1 reaches a limit value in one of the horizontal and vertical directions, the limit position sensors detect this. For example, when the video camera 1 is rotated to the left to reach the limit position, the hole 12 of the fan motion carrier 10 and the PLE sensor 13 are in line with each other. The output value becomes 1 (S8). If the subject continues to move in the same direction, the video camera 1 can no longer track the subject, so the zoom / focus controller 6 adjusts the zoom lens of the video camera 1 in the wide direction to adjust the image of the subject. Reduce one step (S9).

5 is a state diagram illustrating a position occupied by a subject on a screen when zooming by one step in the wide direction. In the first area I, the subject has the left side of the person off the screen. When zooming in a wide step to capture a wider area (area II), the subject is closer to the center of the screen than area I is. In this way, the position occupied by the subject is checked, and the zooming is performed one step in the wide direction until the subject approaches the center of the screen.

In order to perform this function efficiently, it is necessary to define an effective region on the screen. The effective area indicates an area where the subject should actually exist. The moving object automatic tracking device of the present invention is intended for use in a video telephone or a conference system where the change in the movement of the subject is smooth and the moving distance is relatively short. The subject is most often the upper body of a person. Therefore, it is preferable to set the effective area to an area except for left and right ends and a part of the upper end. In the present embodiment, as shown in Fig. 6, portions except for two blocks on the screen A and one block on the upper side are set as the effective area B, respectively. Here, one block is composed of 8 x 8 pixels.

Since it is difficult to directly determine whether the subject has approached the center of the screen, the CPU 7 determines whether to continue or stop zooming based on whether the subject is included in the effective area. If it is determined that the subject can no longer be zoomed even though it is not included in the effective area (S10), it is determined that it is out of the boundary area (S11) and the video camera 1 is returned to the initial state (S1). It is determined whether the subject is included in the effective area (S12), and if not included, zooming continues in the wide direction (S9). If the subject is included in the effective area before reaching the last zooming step, the video camera 1 is moved one step to the right (in the opposite direction to the previous moving direction) (S13), and then the TUE sensor 17 and the TLE sensor 18 Check the condition of).

If the output value of the PLE sensor 13 is not 1 in step S8, it is determined whether the output value of the PRE sensor 14 is 1 (S14). When the video camera 1 reaches the right limit position and the output value of the PRE sensor 14 becomes 1, the zooming in the wide direction and the investigation of whether the effective area is included are repeated as described above (S15 to S17). If the subject is included in the effective area before continuing to the last zooming step until the subject is included in the effective area, the video camera 1 is moved one step to the left (S18), and then the TUE sensor 17 and the TLE are The state of the sensor 18 is checked. When the last zooming step is reached while the subject is not included in the effective area (S16), it is determined to be out of the boundary area (S19), and the video camera 1 is returned to the initial state (S1).

When the video camera 1 reaches the upper limit position and the output value of the TUE sensor 17 becomes 1 (S20), the zooming in the wide direction and the investigation whether the effective area is included as described above are repeated (S21 to S23). . If zooming continues and the subject is included in the effective area, the video camera 1 is moved one step downward (S24), and the process returns to step S4 to detect the image frame again. However, when the final zooming step is reached while the subject is not included in the effective area (S22), it is determined to be out of the boundary area (S25) and the video camera 1 is returned to the initial state (S1).

If the output value of the TUE sensor 17 is not 1 in step S20, it is determined whether the output value of the TLE sensor 18 is 1 (S26). When the video camera 1 reaches the lower limit position and the output value of the TLE sensor 18 becomes 1 (S26), as described above, the zooming in the wide direction and whether the effective area is included are repeated (S27 to S29). . If zooming continues and the subject is included in the effective area, the video camera 1 is moved upward by one step (S30), and the process returns to step S4 to detect the image frame again. However, when the final zooming step is reached while the subject is not included in the effective area (S28), it is determined to be out of the boundary area (S31) and the video camera 1 is returned to the initial state (S1).

If the output value of the TLE sensor 18 is not 1 in step S26, the next process is determined by the ratio (area ratio) between the area occupied by the subject and the effective area (S32). If the area ratio is smaller than a predetermined value, the zoom lens of the video camera 1 is adjusted in the tele direction to enlarge the image of the subject by one step (S33). As described above, when zooming continues in the tele direction and the area ratio becomes larger than the set value, the process returns to step S4 to detect the image frame again. This is to prevent the subject from being zoomed out beyond that in the center of the screen.

[Effects of the Invention]

As described above, the present invention tracks the movement of the subject by moving the imaging direction of the video camera in a horizontal or vertical direction, thereby improving user convenience. In particular, since the screen area is enlarged using the zoom function and the auto focus control function of the camera, the object search area can be much more extensive than before. Therefore, even if the moving distance of the subject is large, there is an effect that can be easily tracked without missing the subject.

Claims (10)

A moving subject automatic tracking device having a video camera for converting an image of a subject input through a zoom lens and an AF lens into an electrical signal, A camera support device equipped with a pan motor and a tilt motor so that the video camera is mounted and the video camera rotates about one axis and rotates about another axis perpendicular to the axis; A pan / tilt controller that calculates a rotation angle of the azimuth and elevation angles by calculating a movement vector of the subject and generates a motor driving pulse signal as far as the video camera should be moved to control driving of the pan motor and the tilt motor; A zoom / focus controller for performing zooming by adjusting the zoom lens according to the output state of the limit position sensor installed in the camera support device and automatically focusing using the maximum luminance component of the image input to the video camera; And a central processing unit which initializes a moving subject auto tracking device and is connected to a pan / tilt controller and a zoom / focus controller to control an operation sequence of each function block. The apparatus of claim 1, wherein the camera support device comprises: a fan motion carrier connected to the fan motor through a fan shaft and rotating together in an azimuth direction (left and right directions) on the mount base as the fan motor rotates; A tilt motion carrier connected to the tilt motor through the tilt shaft and rotating together in an elevation direction (up and down direction) on the pan motion carrier as the tilt motor rotates; A movement provided with limit position sensors which are installed at the limit positions of rotation of the pan motion carrier and the tilt motion carrier and detect the rotation limit position when the fan motion and tilt motion carriers are in line with the holes in the pan motion and tilt motion carriers, respectively. Auto subject tracking. The sensor of claim 2, wherein the limit position sensor comprises: a PLE sensor detecting a left turn limit of the video camera; A PRE sensor for detecting a right turn limit; A TUE sensor for sensing an upward rotation limit; A moving subject automatic tracking device, characterized in that the TLE sensor for detecting the downward rotation limit. In a method for tracking a movement of a subject by rotating a video camera having a zoom lens and an AF lens in a horizontal and vertical direction, A first step of initializing a moving subject automatic tracking device; A second step of detecting the first image frame and the next image frame and determining whether the subject is moving by the difference of the motion vectors; Calculating a moving distance and a number of motor driving pulses using a moving vector when the subject moves and tracking the subject by rotating the pan motor and the tilt motor accordingly; And a fourth process of performing zooming or moving or resetting the video camera to enlarge the subject search area by checking the output state of the limit position sensors. 5. The method of claim 4, wherein the first step comprises the steps of adjusting the azimuth and elevation angles of the pan motion carrier and the tilt motion carrier to 0 [deg.] So that the lens of the video camera faces the center; And resetting the zoom lens and the AF lens to have a minimum focal length. 5. The method of claim 4, wherein the third process further comprises the steps of: calculating a movement angle and displacement of the video camera from the motion vector by determining that there is motion when the motion vector is larger than a set value; Driving the fan motor and the tilt motor by calculating the number of pulses to be applied to the motor according to the calculated displacement; And moving the video camera in the direction of the movement vector by the motor driving to track the subject and correct the displacement caused by the movement of the subject. 7. The method of claim 6, wherein the position of the moving object on the image plane? Is the angle at which the video camera moves horizontally or vertically The automatic tracking method of moving subjects, characterized in that obtained by the following equation. here, Is the focal length of the video camera, Is the horizontal and vertical moving angle component of the initial optical axis of the video camera. 8. The displacement of claim 7, wherein the displacement from the initial position of the video camera. The automatic tracking method of moving subjects, characterized in that obtained by the following equation. 5. The method of claim 4, wherein the fourth process comprises: checking the output state of the limit position sensors to determine whether the video camera has reached the limit position in one of left and right directions; Expanding the subject search area by zooming one step in the wide direction when the limit position is reached; Determining whether the subject is included in the effective area and continuing to zoom in the wide direction until the subject is included; Moving the video camera one step in a direction opposite to the previous moving direction when the subject is included in the effective area; Initializing the video camera by determining that the object is out of the boundary area when the final zooming step is reached while the subject is not included in the effective area; Comparing the ratio of the area occupied by the subject to the effective area with a set value when the video camera has not reached the limit position in any direction; When the area ratio is smaller than a set value, zooming step by step in a tele direction until the area ratio becomes large. 10. The method of claim 9, wherein assuming 8 × 8 pixels as one block, the effective area is the remaining areas except for two blocks on the screen and one block on the top, respectively.