JP2001160916A - Automatic tracking device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an automatic tracking device that can automatically track an object with an easy-to-see video image with minimized operations of a camera, in response to a motion of the object by changing the shot of the camera with respect to only a preset shot, thereby tracking the traced object. SOLUTION: A memory 64 of a universal head controller 12 registers in advance shots to be reproduced in the case of automatic tracking. When automatic tracking is started, a CPU 60 of the universal head controller 12 selects a shot, where an object in contained within a photographing field angle from the shots registered in the memory 64 on the basis of a center of gravity coordinate, in response to the position of the object received from an image processing unit 14 and transmits a control signal to a universal head 10 to reproduce the shot.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic tracking device, and more particularly to an automatic tracking device for automatically changing a photographing condition of a camera and automatically tracking a desired subject.

[0002]

2. Description of the Related Art An automatic tracking device automatically changes a shooting direction of a television camera mounted on a camera platform by driving and panning and tilting the camera, thereby moving a moving object (moving object).
Is a device that automatically tracks As an automatic tracking method, a method described in Japanese Patent Publication No. 6-14698 is generally known. In this method, an image of a moving subject is extracted by calculating a difference between images of two frames at a predetermined time interval from a video signal obtained from a camera, and a moving amount and a moving direction of the subject are determined from a center of gravity of the image of the subject. To detect. And
The camera direction is automatically moved by a camera platform based on the detected moving amount and moving direction of the subject so that the moving subject is always at the center of the screen.

[0003]

By the way, in the conventional automatic tracking method, the camera is panned and tilted so that the subject to be tracked is always near the center of the angle of view, so that the subject cannot be tracked. Even if the camera moves slightly left and right within the range of the angle of view of the camera, the camera always pans and tilts, causing a problem that the screen looks blurred and the image of the camera becomes difficult to see.

The present invention has been made in view of such circumstances, and has an automatic tracking apparatus which can automatically track a subject with an easy-to-view image by minimizing the operation of a camera with respect to the movement of the subject to be tracked. The purpose is to provide.

[0005]

According to the present invention, there is provided an automatic tracking system for automatically tracking a subject to be tracked by changing at least one of a shooting direction and a shooting angle of view of a camera. In the apparatus, shot setting means for setting one or more desired shots in advance by setting shooting conditions of the camera, tracking position detecting means for detecting the position of the subject to be tracked, and tracking position detection A photographing condition changing unit configured to change the photographing condition of the camera to any one of the shots set by the shot setting unit so that the subject falls within the photographing angle of view of the camera based on the position detected by the unit; And, it is characterized by having.

According to the present invention, the photographing condition of the camera can be changed only for a shot set in advance, and the photographing condition of the camera is changed to a shot in which the subject falls within the photographing angle of view in accordance with the position of the subject to be tracked. Since the change is made, even if the subject slightly moves within the shooting angle of view of a certain shot, the shooting conditions of the camera are not changed, and the subject can be automatically tracked with an easy-to-view image. Also, since the zoom position can be set as a shot together with the pan / tilt position of the camera,
The subject can be photographed at a suitable photographing angle of view in accordance with the pan / tilt position of the camera during automatic tracking.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of an automatic tracking device according to the present invention will be described below in detail with reference to the accompanying drawings.

FIG. 1 is an overall configuration diagram of a remote control pan head system equipped with an automatic tracking function to which the present invention is applied. The remote control head system shown in FIG. 1 includes a remote control head 10 (hereinafter, simply referred to as a head 10), a head controller 12, and an image processing device 14. Head 10
The housing 15 includes a television camera (hereinafter, simply referred to as a camera) and a camera platform body 16 that pans and tilts the housing 15. A transparent protective glass 17 is provided on the front surface of the housing 15,
The camera stored in the housing 15 captures an image outside the housing 15 through the protective glass 17.

The housing 15 is supported by a tilt shaft (not shown) extending from the camera platform body 16, and the tilt shaft is driven to rotate by a tilt motor built in the camera platform body 16. The camera platform body 16 is supported by a pan shaft 18 fixed on a mounting table (not shown), and the pan shaft 18 is rotationally driven by a pan motor built in the camera platform body 16. Therefore, when the pan motor and the tilt motor built in the camera platform body 16 are driven, the camera housed in the housing 15 performs a pan / tilt operation, and the shooting direction of the camera is changed.

The camera platform controller 12 is connected to the camera platform 10 via a cable (it is also possible to connect via a communication line such as a dedicated line or a public circuit). A control signal is transmitted to the camera platform 10 based on the operation of the various operating members provided, and various operations of the camera platform 10 and the camera mounted on the camera platform 10 are controlled.
An operation stick (joystick) 20 and a focus knob 2 are provided on the pan head controller 12 as shown in the plan view of FIG.
2. A zoom knob 23 is provided. When the joystick 20 is operated back and forth and left and right, the housing 15 (and the camera) of the camera platform 10 can be panned and tilted, and when the focus knob 22 and the zoom knob 23 are rotated, Focus adjustment and zoom adjustment of the camera (lens device mounted on the camera) of the housing 15 can be performed.

The pan head controller 12 shown in FIG. 1 has a preset function, and a memo switch 24, a page switch 28, a shot switch 30 and the like are provided in association with the preset function. The preset function is a function for automatically reproducing a shot by one-touch operation of the shot switch 30 when a desired shot determined by a pan / tilt position, a zoom position, and a focus position of the camera is registered in advance.

When registering a shot, a memo switch 2
Press 4 to enter the registration mode, and press the joystick 2
By operating 0, the focus knob 22, and the zoom knob 23, the photographing conditions of the camera are set to a desired shot. Then, one of the plurality of shot switches 30 provided from shots NO1 to NO10 is pressed. As a result, the pan / tilt position, the zoom position, and the focus position at that time are stored in the internal memory as shot data of the shot number of the shot switch of the turned on.

The three page switches 28A to 28C
By selecting one of the pages A, B, and C, three shots can be registered in the same shot switch, and a total of 30 shots can be registered. In addition, the registration of the shot may not store all of the pan / tilt position, the zoom position, and the focus position, but may store, for example, only the pan / tilt position and the zoom position.
Only the pan / tilt position or the zoom position may be stored.

On the other hand, when the shot registered in the shot switch 30 is reproduced, the shot switch in which the shot to be reproduced is registered is pressed with the memo switch 24 turned off. As a result, the shot data stored in the memory of the pan head controller 12 is read out, and a control signal for instructing reproduction to the pan / tilt position, the zoom position, and the focus position registered as the shot data is issued. It is transmitted to the table 10. Based on this control signal, the housing 15 of the camera platform 10 performs a pan / tilt operation, the shooting direction of the camera moves to the instructed pan / tilt position, and the focus position and the zoom position in which the zoom and focus of the camera are instructed. Go to still,
By adjusting the speed knob 32, the operation speed at the time of the above-described shot reproduction can be changed.

The pan head controller 12 has an automatic tracking function, and this function is activated by turning on an automatic tracking switch 26 shown in FIG. The automatic tracking function is a function of tracking a moving subject among the subjects within the shooting angle of view with the camera. In the present invention, the subject to be tracked among shots registered by the preset function is set within the shooting angle of view. The subject is automatically tracked by sequentially switching to shots that can be stored in the camera. Therefore,
This is different from the conventional method of automatically tracking the subject by arbitrarily moving the shooting direction of the camera according to the position of the subject. Also, in the case of the present invention, the subject can be tracked while changing not only the pan / tilt position but also the zoom position and the focus position according to the position of the subject. The position of the subject to be tracked by the camera is derived by the image processing device 14 and given by the image processing device 14. Details of this will be described below.

FIG. 3 is a block diagram showing the configuration of the remote control pan head system.
FIG. 3 is a diagram showing the configuration of FIG. As shown in the figure, the camera 40 housed in the housing 15 of the camera platform 10 includes a camera body 42 and a lens device 44 mounted on the camera body 42. The camera body 42 includes an image sensor and a required processing circuit, and outputs an image (moving image) captured by the image sensor to the outside as a video signal. The control of the photographing operation such as the start and end of the photographing is performed based on a control signal given from the camera platform controller 12 via the camera platform 10.

The lens device 44 includes an optical member such as a focus lens and a zoom lens that can be driven by a motor. The focus and zoom of the camera 40 are adjusted by moving the focus lens and the zoom lens. Control regarding the operation of the lens device 44 such as focus and zoom is performed based on a control signal given from the camera platform controller 12 as in the camera body 42.

The camera platform 10 drives the housing 15 by the control signal given from the camera platform controller 12 as described above, and causes the camera 40 to perform a pan / tilt operation.
For the control of the camera body 42 or the lens device 44, a control signal given from the camera platform controller 12 is transferred to the camera body 42 or the lens device 44.

The image processing device 14 includes the pan head controller 1
2 is a device which functions effectively when the automatic tracking switch 26 is turned on and the automatic tracking mode is set, and mainly includes a Y / C separation circuit 46, an image memory 50, an image processor 52, and a CPU 54 as shown in FIG. And so on. The Y / C separation circuit 46 converts the video signal output from the camera body 42 of the camera 40 into a luminance signal (Y signal).
And a chroma signal, and only the Y signal is extracted. The Y signal extracted here is converted by the A / D converter 48 into a digital signal (hereinafter referred to as image data).
The image memory 50 stores image data for two frames output from the Y / C separation circuit 46.

The image processor 52 stores image data for two frames in the image memory 50 based on a synchronization signal given in synchronization with the video signal from the Y / C separation circuit 46, and stores the image data in the image memory 50. Remembered 2
A process for detecting the position of the subject to be tracked by the camera 40 (the position of the center of gravity shown below) from the image data for the frame is executed.

The CPU 54 acquires the position of the subject to be tracked by the camera 40 based on the processing result of the image processor 52, and gives the position to the camera platform controller 12. In addition, other necessary processing is executed. When the position of the subject to be tracked is thus provided from the CPU 54 of the image processing device 14 in the automatic tracking mode in the automatic tracking mode, the camera platform controller 12 controls the automatic tracking as described later.

The processing procedure of the image processing apparatus 14 will be described with reference to the flowchart of FIG. 4 and FIG. The image processor 52 first stores one frame of image data (this image is an image) obtained from the camera body 42 of the camera 40 via the Y / C separation circuit 46 and the A / D converter 48. (Step S1)
0). Subsequently, after a lapse of a predetermined time, image data for one frame (this image is referred to as an image) is loaded into the image memory 50 in the same manner as the image (step S12). FIG. 5 (A)
Is a diagram showing an example of an image and an image taken into the image memory 50, and the subject indicated by Δ is stationary;
The object indicated by is moving.

Next, the image processor 52 obtains a difference image between the images (step S14). That is,
The difference between the same pixels in the image and the image data of the image is obtained, and the absolute value is obtained. Thereby, only the image data of the moving subject is extracted. FIG. 5B is a diagram showing a difference image between the image shown in FIG. 5A and the image, and it can be seen that only the moved 被 写 体 subject is extracted. That is, the subject of the circle becomes the subject to be tracked.

Subsequently, the image processor 52 binarizes the image data of each pixel of the difference image obtained in step S14 (step S16). By this processing, ideally, the pixel data of the moving subject becomes 1 and 0 otherwise. Next, a process of contracting the binarized difference image is performed to remove fine noise (step S18). Then, based on the data of the difference image, a label is attached to each subject regarded as one (step S20). FIG.
(C) shows a state in which the subject is labeled, and the subjects included in the difference image are labeled "1" and "2", respectively.

Subsequently, the image processor 52 calculates an area for each of the labeled objects (step S2).
2). The range indicated by oblique lines in FIG. 5D is the area of each of the subjects “1” and “2”. The CPU 54 obtains the area of each subject from the image processor 52, compares each area with a predetermined threshold value, and selects only a subject larger than the threshold value (step S24). Thus, a subject smaller than the threshold value or a subject with little movement is excluded from the targets of the following processing. If there is no subject larger than the threshold value, step S
Returning to step 12, the above processing is repeated. It is assumed that the areas of the subject “1” and the subject “2” in FIG. 5D are larger than a predetermined threshold value and have been selected by the CPU 54.

Next, the image processor 52 includes a CPU
1 including all subjects of the label selected by 54
Next moments are obtained (step S26), and the positions of the centers of gravity (coordinates of the centers of gravity) of the subjects are obtained (step S28).
FIG. 5E is a diagram showing the result of calculating the barycentric coordinates of the subject “1” and the subject “2”, and the + position shown in FIG. 5E indicates the barycentric position. The CPU 54 transmits the barycentric coordinates to the camera platform controller 12 as the position of the subject to be tracked by the camera 40. Based on the barycentric coordinates, the pan head controller 12 controls the pan and pan of the camera 40 as described below.
Switching of shooting conditions such as a tilt position, a zoom position, and a focus position. When this operation is completed, step S1 is performed.
The processing from 0 is repeatedly executed.

Next, the configuration and processing of the camera platform controller 12 will be described. FIG. 6 is a block diagram showing the configuration of the camera platform controller 12. CPU 60 shown in FIG.
Are control signals for instructing the pan head 10 to perform a pan / tilt operation, control signals for driving the zoom and focus of the lens device 44, and the like, based on the operation contents of the operation unit 62 including various operation members shown in FIG. Transmits various control signals. Also,
When the memo switch 24 shown in FIG. 2 is turned on and one of the shot switches 30 is turned on, and a shot registration is instructed, the pan / tilt position, the zoom position, and the focus position (position Signal), and these positions are stored as shot data at the address of the memory 64 corresponding to the turned on shot switch. When one of the shot switches 30 is turned on while the memo switch 24 is off, an instruction to reproduce a shot is issued, the shot data corresponding to the turned on shot switch is read from the memory 64, and the shot is reproduced. Is transmitted to the camera platform 10.

The automatic tracking switch 26 shown in FIG.
Is turned on, and the automatic tracking mode is turned on, the automatic tracking process is executed based on the barycentric coordinates provided from the image processing device 14.

Hereinafter, the content of the automatic tracking process will be described with reference to the flowchart of FIG. 7 and the diagrams of FIGS. First, before starting automatic tracking, a desired shot is registered in the memory 64 in advance by the above-described preset function (step S50). For example, as shown in FIG. 8, a table 80 is placed at the center, and panels 82,
In the shooting studio where the camera 84 is arranged, the caster M is operated by the automatic tracking mode of the remote control pan head system.
Is to be photographed, for example, in the center of the table 8
A shot for photographing the position of 0 (shot 1) and shots for photographing both sides thereof (shot 2, shot 3) are stored in the memory 64 in advance by a preset function.

Next, for each shot registered in the memory 64, the CPU 60 obtains the position of the frame of each shot based on the pan / tilt position and the zoom position of the shot, and records it in the memory 64 together with the shot data. (Step S52). The position of the frame can be recorded, for example, at the pan / tilt angle at that position, but the range within the frame, not the position of the frame, may be recorded at the pan / tilt angle.

Next, a desired shot is reproduced by the preset function, and a frame at the start of photographing is set (step S54). For example, a shot 1 indicated by a solid frame in FIG. 8 is reproduced, and a caster M behind the table 80 is photographed. Note that the frame at the start of shooting is not based on a shot, but may be set to a desired frame by operating the joystick 20, the focus knob 22, and the zoom knob 23 of the pan head controller 12.
Then, the automatic tracking switch 26 is turned on to start automatic tracking (step S56).

When the automatic tracking is started, the CPU 60 acquires the barycentric coordinates from the image processing device 14 (step S58). Then, based on the barycentric coordinates and the position of the frame frame of each shot recorded in the memory 64, the barycentric coordinates are set within a range within the frame frame of any shot registered in the memory 64 other than the current shot. It is determined whether or not it has moved (step S60). At this time, NO
If so, the processing from step S58 is repeatedly executed. On the other hand, if YES, the control signal for instructing the reproduction of the shot is transmitted to the camera platform 10, and the shot is changed so that the moving subject falls within the shooting angle of view (step S62). For example, as shown in FIG. 9, if the caster M moves rightward after starting automatic tracking,
The barycentric coordinates acquired by the CPU 60 from the image processing device 14 move to the right along with the casters M. Then, assuming that the barycentric coordinates move to the position indicated by + in the figure and enter the frame of the shot 2, a control signal for reproducing the shot 2 is transmitted to the camera platform 10, and the frame indicated by the solid line in FIG. Is switched to shooting of shot 2.

As described above, the pan / tilt position, the zoom position, and the focus position of the camera 40 are changed only for shots registered in the memory 64 in advance. Even if the camera 40 moves a little, the pan / tilt position, zoom position and focus position of the camera 40 are not changed, and the image captured by the camera 40 becomes easy to see. Also, the pan of the camera 40
Since the desired zoom position can be registered as a shot together with the tilt position, the subject can be photographed at a suitable photographing angle of view according to the pan / tilt position at the time of automatic tracking.

In the automatic tracking process, the condition for switching from one shot to another shot, that is,
The condition for switching to the shot when the subject (center of gravity coordinates) falls within the shooting angle of view (frame frame) of another shot is an example, and the subject is shot according to the movement of the subject to be tracked. The shots may be switched under any conditions as long as the shots within the angle of view can be reproduced. For example, among shots registered in the memory 64, a shot in which the subject to be tracked (coordinate of gravity) falls within the shooting angle of view (frame frame) and the center position of the shooting angle of view is closest to the subject is selected. If the selected shot is different from the current shot, the shot may be switched to the selected shot. Also,
In addition to these conditions, if there is a plurality of shots, the condition that the shot is switched to the shot with the smallest shooting angle of view is added. For example, only the zoom position (shooting angle of view) with the same pan / tilt position Even when different shots are registered in the plurality of memories 64, it is possible to change only the zoom in accordance with the position of the subject to be tracked and switch to a suitable shot in which the subject is within the shooting angle of view. In addition, a determination area is set within the frame of each shot along the frame, and when it is detected in the determination area of the current shot that the barycentric coordinates have moved to the outside of the frame, the subject is shot. The shot may be switched to another shot within the corner. Further, when it is detected that the subject (center of gravity coordinates) has moved toward the outside of the frame frame of the current shot in an area where the shooting angle of view of the current shot and the shooting angle of view of another registered shot overlap. Alternatively, the shot may be switched to another shot whose shooting angle of view overlaps.

In the above embodiment, the case where the pan / tilt position, the zoom position and the focus position are registered as shots has been described. However, the case where only the pan / tilt position or the zoom position is registered as shots may be used. The present invention can be applied.

Also, in the above-described embodiment, a case has been described in which automatic tracking is performed by a television camera. However, the present invention is not limited to this, and a camera (for example, a digital still camera or the like) that captures a still image is used to detect a subject. It can also be applied to automatic tracking.

[0037]

As described above, according to the automatic tracking apparatus of the present invention, the shooting conditions of the camera can be changed only for shots set in advance, and the subject can be shot according to the position of the subject to be tracked. The camera's shooting conditions are changed to shots that fall within the angle of view, so even if the subject moves slightly within the shooting angle of view of a certain shot, the camera's shooting conditions do not change, and the subject is displayed in an easy-to-view image. Automatic tracking can be performed. In addition, since the zoom position as well as the pan / tilt position of the camera can be set as a shot, the subject can be photographed at a suitable photographing angle of view according to the pan / tilt position of the camera during automatic tracking.

[Brief description of the drawings]

FIG. 1 is an overall configuration diagram of a remote control pan head system equipped with an automatic tracking function to which the present invention is applied.

FIG. 2 is a plan view showing an operation member of the camera platform controller.

FIG. 3 is a block diagram illustrating a configuration of a remote control pan head system, and is a diagram mainly illustrating a configuration of an image processing apparatus.

FIG. 4 is a flowchart illustrating a processing procedure of the image processing apparatus.

FIG. 5 is an explanatory diagram used for describing a processing procedure of the image processing apparatus.

FIG. 6 is a block diagram showing a configuration of a pan head controller.

FIG. 7 is a flowchart illustrating the content of processing of automatic tracking.

FIG. 8 is an explanatory diagram used for describing the content of the automatic tracking process.

FIG. 9 is an explanatory diagram used for describing the processing content of the automatic tracking.

FIG. 10 is an explanatory diagram used for explaining the content of processing of automatic tracking.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 10 ... Head, 12 ... Head controller, 14 ... Image processing apparatus, 40 ... Camera, 42 ... Camera main body, 44 ... Lens device, 46 ... Y / C separation circuit, 48 ... A / D converter, 50 ... Image memory , 52... Image processor, 5
4, 60 CPU, 62 operating unit, 64 memory

Claims (2)

[Claims] 1. An automatic tracking apparatus for automatically tracking a subject to be tracked by changing at least one of a shooting direction and a shooting angle of view of a camera. Shot setting means for setting one or a plurality of shots; tracking position detecting means for detecting the position of the subject to be tracked; and the subject being located on the camera based on the position detected by the tracking position detecting means. An automatic tracking device, comprising: a photographing condition changing unit that changes a photographing condition of the camera to one of the shots, the photographing condition changing unit being set by the shot setting unit so as to fall within a photographing view angle. 2. The photographing condition changing unit, when the position detected by the tracking position detecting unit falls within a photographing angle of view of a shot under a photographing condition different from the photographing condition of the camera,
2. The automatic tracking device according to claim 1, wherein a photographing condition of the camera is changed for the shot.