JP2002247440A - Automatic tracking device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an automatic tracking device that generates and registers trace data storing a series of operations such as panning, tilt, zoom and focus of a television camera, controls the panning position on the basis of gravity center coordinates of a mobile object to be traced, and controls the tilt, zooming and focal position on the basis of the trace data so as to automatically trace the moving object while providing an easy to see video image. SOLUTION: An image processing unit 14 detects coordinates of the gravity center of the moving object from an image photographed by the television camera. A universal head controller 12 controls a panning position by a universal head 10 so that the coordinates of the gravity center in the panning direction comes in the middle of an image and controls the position by the tilt, zoom and focus so that the position comes to a position associated with the panning position according to the trace data generated in advance.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic tracking device, and more particularly to an automatic tracking device for automatically tracking a moving object by panning or tilting a camera.

[0002]

2. Description of the Related Art An automatic tracking device mounts a television camera on a camera platform and pans / tilts the camera by driving the camera platform with a motor to automatically pan and tilt a moving object (moving object). This is a device that enables tracking. As an automatic tracking method, a method described in JP-A-59-208983 is generally known. According to this method, images of two frames (two screens) at predetermined time intervals are captured from a video signal output from a camera, and an image of a moving object is extracted by comparing the images. Then, the moving amount and moving direction of the moving object are detected from the position of the center of gravity of the extracted image of the moving object, and based on this, the pan and tilt positions of the camera platform (camera) are automatically controlled, and the center of gravity of the moving object is detected. Is always at the center of the screen (the center of the shooting range). The function of the automatic tracking device (automatic tracking function) can be incorporated as one control function (automatic tracking function) of the camera platform (head system).

On the other hand, if a series of operations of pan, tilt, focus, and zoom are registered as trace data in advance as a control function of the camera platform system, the operation can be reproduced by one-touch operation of a switch. A trace function (Japanese Patent No. 2868087) is known. Such a trace function is useful when the range of movement of the moving object is predetermined, for example, when it is determined that the caster of the news program moves from one standing position to another standing position.

[0004]

The automatic tracking function and the tracing function as described above can be said to be functions for automatically controlling positions such as panning and tilting. However, each function has a different application, and each has advantages and disadvantages. There is.

In the automatic tracking function, for example, when a moving object is detected, the position of panning, tilting, etc. is automatically controlled so that the moving object is at the center of the screen. There is an advantage that there is no necessity to perform the operation and the operation range is not limited to the operation range defined by the trace data. Furthermore, if the automatic tracking mode is selected, photographing without an operator is possible, and the present invention can be applied to uses such as monitoring. On the other hand, since panning, tilting, and the like operate such that the moving object is always at the center of the screen, the image on the screen may move up and down unnecessarily, making it difficult to see the image.

On the other hand, in the trace function, a series of operations such as panning and tilting are registered in advance, so that a moving object can be photographed in an easy-to-view image by a smooth operation, and a series of zooming and focusing can be performed. Can be set, and more precise camera work can be set and executed, and a television camera without an autofocus function can be used. On the other hand, if the moving range of the moving object is not determined at all, this function cannot be used for the purpose of tracking the moving object, and even if the moving object has a limited moving range, Is not determined, for example, this function cannot be used when it is not known what position, timing, direction, and speed the moving object moves within the limited moving range. Furthermore, the trace function is not suitable for use in monitoring by unmanned operation.

[0007] The present invention has been made in view of such circumstances, and an automatic tracking device that can automatically track a moving object with a more easily viewable image by making use of the respective advantages of the conventional automatic tracking function and tracing function. The purpose is to provide.

[0008]

According to a first aspect of the present invention, there is provided a camera which can be moved in at least one of pan, tilt, and height directions by a camera platform. Supporting the camera, detecting the position of the moving object to be tracked from the image taken by the camera,
In an automatic tracking device that moves the camera in the moving direction based on the detected position so that the moving object does not fall outside the shooting range of the camera, at least one of the moving directions in which the camera can move A position in one of the two moving directions is defined as a main control position, and at least one of a moving direction different from the moving direction related to the main control position and a zoom and focus of the camera among the movable directions of the camera. Trace data creating means for creating trace data associating the sub-control position with the main control position with the position of one of the two as the sub-control position; and While controlling the main control position so as not to be outside the shooting range of the camera, the main control position with respect to the main control position Is characterized by and a control means for controlling the sub-control position so that the associated position by racing data.

The invention described in claim 2 is the first invention.
In the invention described in (1), the main control position is a position for pan, and the sub control position is a position for tilt, zoom, and focus.

[0010] Further, the invention described in claim 3 is based on claim 2.
In the invention described in the above, the trace data is a pan,
It is characterized in that the data is a series of tilt, zoom and focus operations recorded at each position at certain time intervals.

The invention described in claim 4 is the first invention.
In the invention described in (1), there is provided a detecting means for identifying a desired moving object to be tracked by a hue and detecting the barycentric coordinates of the identified moving object to be tracked.

According to the present invention, the position in the movement direction of at least one of the pan, tilt and height is set as the main control position, and the other movement direction and at least one of the zoom and focus of the camera are selected. One of the positions is set as a sub control position, and the main control position is controlled based on the position of the moving object to be tracked (coordinates of the center of gravity), while the sub control position is associated with the main control position by trace data. Since the control is performed so as to be at the specified position, fine camera control according to the position of the moving object can be performed according to the content of the trace data. The position in the camera movement direction (for example, the pan direction) with respect to the main movement direction of the moving object is set as the main control position, and the positions in the other camera movement directions (tilt direction, height direction) are set as the sub control positions. Thus, the sub-control position with respect to the main control position can be determined in advance by creating the trace data, and it is possible to make it easy to see the video being automatically tracked. Also, by setting the focus to the sub-control position, the focus can be automatically adjusted according to the position of the moving object even when the camera does not have an auto-focus function, and by setting the zoom to the sub-control position, the position of the moving object can be adjusted. Can be automatically adjusted according to.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of an automatic tracking device according to the present invention will be described below in detail with reference to the accompanying drawings.

FIG. 1 is an overall configuration diagram of a remote control pan head system equipped with an automatic tracking function to which the present invention is applied. The remote control head system shown in FIG. 1 includes a remote control head 10 (hereinafter, simply referred to as a head 10), a head controller 12, and an image processing device 14. Head 10
It comprises a housing 15 for housing a television camera (hereinafter simply referred to as a camera), and a camera platform body 16 for rotating the entire housing 15 to pan and tilt the camera in the housing 15. A transparent protective glass 17 is provided on the front surface of the housing 15, and a camera housed in the housing 15 captures an image outside the housing 15 through the protective glass 17.

The housing 15 is supported by a tilt shaft (not shown) extending from the camera platform main body 16, and the tilt shaft is driven to rotate by a tilt motor built in the camera platform main body 16. Therefore, when the tilt motor is driven, the housing 15 rotates via the tilt shaft, and the camera in the housing 15 tilts accordingly. The pan head main body 16 is supported by a pan shaft 18 fixed on a mounting table (not shown), and the pan head main body 16 is driven to rotate by a built-in pan motor around the pan shaft 18. Therefore, when the pan motor is driven, the pan head body 16
At the same time, the housing 15 rotates, and the camera in the housing 15 pans.

The camera platform controller 12 is connected to the camera platform 10 via a cable (it is also possible to connect the camera via a communication line such as a dedicated line or a public line). A control signal is transmitted to the camera platform 10 based on the operation of the various operating members provided, and various operations of the camera platform 10 and the camera mounted on the camera platform 10 are controlled.
The head controller 12 has an automatic tracking function. When the camera is in the automatic tracking mode, the head controller 12 transmits a control signal to the head 10 based on a signal given from the image processing device 14, and An object (moving object) moving by the pan / tilt operation of the table 10 is tracked by the camera.

FIG. 2 is a block diagram showing the configuration of the remote control pan head system. As shown in FIG.
The camera 40 housed in the housing 15 is composed of a camera body 42 and a lens device 44 mounted on the camera body 42. The camera body 42 has an image sensor and necessary processing circuits mounted thereon, and an image (moving image) formed on the image sensor via the optical system of the lens device 44 is converted into a video signal (NTSC system in this embodiment). As a video signal). The shooting operation such as the start and end of shooting in the camera body 42 is controlled based on a control signal given from the camera platform controller 12 via the camera platform 10. In addition, the lens device 44 is equipped with an optical member such as a focus lens and a zoom lens that can be driven by a motor.
The focus and zoom of the camera 40 are adjusted by moving these focus lenses and zoom lenses. The operation of the lens device 44 such as focus and zoom,
Like the camera body 42, the camera is controlled based on a control signal given from the camera platform controller 12.

The pan head and the tilt motor are mounted on the pan head 10 as described above.
When these pan motors and tilt motors are driven by the control signal given by the camera 40, the camera 40 pans or tilts, and the shooting range of the camera 40 moves.

As shown in FIG. 1, the image processing device 14
/ C separation circuit 46, image memory 50, image processing processor 52, various signal processing circuits such as CPU 54. These signal processing circuits operate effectively when the pan head controller 12 is in the automatic tracking mode. I do. A video signal output from the camera body 42 is input to the image processing device 14, and the video signal is separated into a luminance signal (Y signal) and a color difference signal by a Y / C separation circuit 46 in the automatic tracking mode. . The separated luminance signal is converted into a digital signal (hereinafter referred to as image data) by an A / D converter 48 and input to an image memory 50. On the other hand, a synchronization signal of a video signal is given from the Y / C separation circuit 46 to the image processor 52, and a command for writing data from the image processor 52 to the image memory 50 at a required timing based on the synchronization signal. Is given. Thereby, the image memory 50
Stores image data of a plurality of frames at predetermined time intervals as described later. It should be noted that an image for one frame is composed of an image for two fields since the NTSC video signal employs an interlaced system. In the present specification, when one frame of image data is called, It means image data constituting one screen in a series of images taken continuously, and in this embodiment, one frame of image data refers to one field of image data.

The image data for a plurality of frames stored in the image memory 50 as described above is read out and image-processed by the image processor 52 as will be described in detail below. Based cpu
The arithmetic processing is performed by 54, and the barycentric coordinates for automatically tracking the moving object with the camera 40 are calculated. In addition, CPU5
4, the focal length (angle of view) of the lens device 44 and the moving speed (pan speed and tilt speed) of the camera platform 10 at each time are provided from the camera platform controller 12 to calculate a comparison frame to be described later.

The head controller 12 has a CPU 60, a memory 62 and the like as shown in FIG.
Various processes are executed in U60, and required data is stored in the memory 62. When the pan head controller 12 is set to the automatic tracking mode by a predetermined switch, the barycenter coordinates of the moving object obtained by the image processing device 14 are given to the CPU 60 of the pan head controller 12 as described above. Then, a control signal is transmitted to the camera platform 10 based on the coordinates of the center of gravity, and the camera 40 tracks the moving object by performing pan / tilt operations and the like on the camera platform 10. On the other hand, when the automatic tracking mode is not set, a control signal is transmitted to the camera platform 10 based on the operation of various operation members provided in the camera platform controller 12, and the camera is mounted on the camera platform 10 and the camera platform 10. Various operations of the camera 40 are controlled.

In the remote control head system configured as described above, first, when the automatic tracking mode is set, the image processing processor 52 and the CPU
The detection processing of the barycenter coordinate of the moving object performed by 54 will be described with reference to the flowchart of FIG. In the following description, the process of detecting the center of gravity of the moving object in the pan direction (horizontal direction) will be mainly described, and the tilt direction is the same as that in the pan direction. However, as described later, in the automatic tracking according to the present embodiment,
If the coordinates of the center of gravity in the pan direction are detected, it is not always necessary to detect the coordinates of the center of gravity in the tilt direction.

First, the image processor 52 sends the Y / C separation circuit 46 from the camera body 42 of the camera 40 to the A / D
One frame of image data (this image is defined as an image) obtained via the converter 48 is loaded into the image memory 50 (step S10).

Subsequently, the image processor 52 sets the Y /
Time for 4 fields (1 / 59.94 seconds per field) by the synchronization signal given from the C separation circuit 46
Is determined (step S12), and when the time for four fields has elapsed, one frame of image data (this image is defined as an image) is fetched into the image memory 50 in the same manner as the image (step S14). ). It should be noted that the time from the capture of the image in step S12 to the capture of the image need not be the time for four fields.

When the image data for two frames is stored in the image memory 50 as described above, the CPU 54 determines the current focal length (angle of view) of the camera 40 (lens device 44) and the moving speed of the camera platform 10 ( The pan speed and the tilt speed are acquired from the pan head controller 12, and based on the acquired pan and tilt, a comparison range (comparison frame) between images is set, and the comparison frame is designated to the image processor 52 (step S16).

Here, the comparison frame will be described. As described later, the image processor 52 compares the image with the image, and generates a difference image by obtaining a difference between the values (hereinafter, pixel values) of respective pixels of the image data. The reason for generating the difference image is to extract an image of a moving object to be tracked from the two images.
In order to extract an image of a moving object from the difference image, it is necessary to obtain a difference image from images within the same shooting range (a range where the shooting ranges overlap) of the image shooting range and the image shooting range.

For example, when the camera platform 10 is stopped immediately after the start of the automatic tracking process, the image and the image capturing range are equal. FIG. 4A shows the relationship between the subject and the shooting range in this case, and the shooting range L of the image and the shooting range L ′ of the image completely match. In the drawing, the subject indicated by a circle is a moving object, and the subject indicated by a triangle is a stationary object. The moving object is indicated by A in the figure when the image is captured.
It is assumed that the camera has moved to the position indicated by B in the figure at the time of capturing an image.

At this time, the images and the images stored in the image memory 50 form screens as shown in FIGS. 4B and 4C, respectively. The image (△) of the stationary object exists on the same coordinates, and only the image (○) of the moving object exists on different coordinates.

Accordingly, if the pixel value difference between the pixels at the corresponding positions (same coordinates) in the entire image range (within the entire photographing range) of the image and the image, the pixel value of the stationary object in the difference image is 0. As shown in FIG. 5, a difference image in which only the moving object is extracted is obtained. The coordinates of each point on the screen of each image are in accordance with the arrangement of pixels in a matrix forming one screen, and as shown in FIG. 6, the column numbers of the pixels of each point in the horizontal direction (from the leftmost pixel on the screen) 1 to X assigned in order (X corresponds to the number of pixels indicating horizontal resolution))
The vertical direction is represented by the row number of the pixel at each point (1 to Y (Y corresponds to the number of pixels indicating the vertical resolution) assigned in order from the pixel at the top of the screen).

On the other hand, when the camera platform 10 is moving, the photographing range differs between the image and the image. FIG. 7A shows the relationship between the subject and the shooting range in this case, and the shooting range L of the image is different from the shooting range L ′ of the image. Note that the same conditions as in FIG. 4A are applied to the subjects (moving objects and stationary objects) in the figure. At this time, the images stored in the image memory 50 are
Screens such as those shown in FIGS. 7B and 7C are respectively formed, and on each of these images and images, the image (△) of the stationary object does not exist on the same coordinates. Therefore, if a difference image is obtained in the entire screen range in the same manner as described above, a difference image as if the stationary object is a moving object is obtained, and only the moving object is appropriately extracted. Can not do.

Therefore, of the entire screen area of the image and the entire screen area of the image, the area where the photographing area overlaps, that is, FIG.
In (A), an image in a range where an image capturing range L and an image capturing range L ′ overlap each other is extracted from the image and the image, and a difference between pixel values at a corresponding position between the images in the extracted range is determined. By doing so, a difference image is obtained. As a result, the pixel value of the stationary object in the difference image becomes 0, and a difference image in which only the moving object is extracted can be obtained as shown in FIG.

In order to obtain a difference image in which a moving object is extracted from an image and an image, it is necessary to set a comparison range for obtaining the difference image on each screen of the image and the image. The comparison range is represented by a frame as a comparison frame. Then, the comparison frame is set as a frame of a shooting range where the shooting range L of the image and the shooting range L 'of the image overlap. When the camera platform 10 is stopped as shown in FIG. 4, the image shooting range L and the image shooting range L '
4B and 4C, the comparison frame P is set as a frame of the entire screen range on each screen of the image and the image as shown in FIGS. On the other hand, when the camera platform 10 is moving as shown in FIG. 7, the photographing range L of the image is different from the photographing range L ′ of the image.
As shown in (B) and (C), on each screen of the image and the image, a comparison frame P is set as a frame of a range where the image shooting range L and the image shooting range L 'overlap.

The specific processing for setting the comparison frame will be described. When an image and an image are loaded into the image memory 50 as described above, the CPU 54 sets the current focal length of the lens device 44 and the current movement of the camera platform 10. The speed (pan speed) is acquired from the pan head controller 12. Then, the CPU 54 obtains a shooting angle of view (horizontal angle of view indicating an angle range in the horizontal direction) from the focal length of the lens device 44, and calculates the image shooting range with respect to the image shooting range based on the shooting angle and moving speed. Is calculated by the number of pixels in the horizontal direction. Shift amount shift
(Unit: pixel) is V (unit: angle / second) for the moving speed (pan speed) of the camera platform 10, R (unit: pixel) for image resolution (horizontal resolution), and shooting angle of view (horizontal angle of view). 2
θ (unit: angle) and T (unit: image capture time difference between images)
Second), it is expressed as shift = (R / 2θ) × V × T. In calculating the vertical shift amount shift, the moving speed V is the tilt speed, the resolution R is the vertical resolution, and the shooting angle of view A is the vertical angle of view indicating the vertical angle range. If the image is captured after a lapse of four fields of time after the capture of the image, the photographing time difference T is equal to the time of one field (1/5).
(9.94 seconds).

When the shift amount shift is calculated by the above equation,
The CPU 54 sets a comparison frame on the screen of the image and the image based on the shift amount shift. That is, as shown in FIG. 9, when the photographing range L 'of the image moves rightward with respect to the photographing range L of the image (the camera platform 10 moves rightward), the comparison frame of the image Is set as a comparison frame P from the left end of the screen in the entire screen range excluding the number of pixels of the shift amount shift (the range indicated by oblique lines in the figure). Shift amount shif from the right end of the screen in the entire screen range
Excluded area for the number of pixels of t (range shaded in the figure)
Is set as the comparison frame P. As shown in FIG. 10, when the photographing range L ′ of the image moves leftward relative to the photographing range L of the image (the camera platform 10 moves leftward),
As for the comparison frame of the image, the outline of the range (shaded area in the figure) excluded from the right end of the screen by the number of pixels of the shift amount out of the entire screen range of the image is set as the comparison frame P. As for the comparison frame of the image, the outline of the range (the range shown by the diagonal lines in the figure) excluded from the left end of the screen by the number of pixels of the shift amount out of the entire screen range of the image is set as the comparison frame P. The same applies to the case where the camera platform 10 moves in the vertical direction.

Returning to the flowchart of FIG. 3, a comparison frame is set by the CPU 54 as described above, and when the comparison frame is instructed to the image processor 52, the image processor 52 , An image in the comparison frame is extracted from each of them, and a difference image between the images is obtained from the extracted image (step S1).
8). That is, the difference between the pixel values of the pixels at the corresponding positions (pixels at the same position in the comparison frame) is determined from the image data in the comparison frame between the images, and the absolute value is calculated (|-|). As a result, only the image of the moving object is extracted as shown in FIGS. 5 and 8, and the moving object is recognized as the subject to be tracked.

Subsequently, the image processor 52 binarizes the image data of each pixel of the difference image obtained in step S18 (step S20). By this processing, the pixel value of the image of the moving object becomes 1 ideally, and 0 otherwise. Then, a process of contracting the binarized difference image is performed to remove fine noise (step S22). In the following, the difference image that has been binarized and contracted is simply referred to as a difference image.

Next, the image processor 52 calculates the total number of pixels having a pixel value of 1 based on the difference image, and calculates the area of the entire image of the moving object (hereinafter simply referred to as the area of the moving object) (step). S24). In the examples of FIGS. 5 and 8, the areas of two circles indicating images before and after the movement of the moving object are obtained. Then, the obtained area is given to the CPU 54.

When the area of the moving object is obtained from the image processor 52, the CPU 54 determines whether or not the area is larger than a predetermined threshold (step S2).
6). Here, when the determination is NO, there is no moving object enough to be tracked, when the moving object that has been tracked so far stops, when the movement of the moving object is small, or There are cases where only a small object that is not a tracking target is moving. At this time, the process returns to the step S14, and the image is stored in the image memory 5
It repeats from the process which takes in to 0.

On the other hand, YES in step S26
Is determined, that is, when a moving object is detected from the difference image, the image processor 52 next calculates the first moment of the entire image of the moving object from the difference image (Step S).
28) Divide the first moment by the area of the moving object to obtain the center of gravity of the moving object (step S30). Here, the center of gravity of the moving object is represented by coordinates on the screen of the image, and the coordinates of the center of gravity (the coordinates of the center of gravity in the horizontal direction) are transmitted by the CPU 54 to the CPU 60 of the pan head controller 12 (step S32). When the process of step S32 is completed, the process returns to step S10, and the process from step S10 is repeatedly executed.

Next, the processing of the automatic tracking function in the camera platform controller 12 will be described. Head controller 1
When the automatic tracking mode is set by a predetermined switch, the second CPU 60 obtains the barycentric coordinates of the moving object to be tracked from the image processing device 14 as described above, and the moving object falls within the shooting range of the camera 40. Based on the obtained center-of-gravity coordinates and the trace data, the pan and tilt positions of the camera platform 10 and the lens device 44 of the camera 40
To control the zoom and focus positions.

Here, the trace data is data used in a trace function different from the automatic tracking function mounted on the camera platform controller 12, and includes pan, tilt, zoom,
This is data that records a series of operations for focus. The trace function is a function that, when such trace data is created and registered in advance, a series of operations recorded as trace data by a trace reproduction instruction operation can be automatically reproduced. When creating and registering trace data, for example, the pan head controller 12 is set to a trace registration mode, and the operator actually performs a series of operations of pan, tilt, zoom, and focus to be registered by manual operation. As a result, the pan, tilt, zoom, and focus position data during that period are captured at regular intervals and stored in the memory 62 as trace data.

The trace data used in the automatic tracking function may be shared with the trace data used in the trace function, or may be used only in the automatic tracking function. That is, both the trace data used in the automatic tracking function and the trace data used in the trace function are pan,
Tilt, zoom, and focus can be shared because they are operated in association with each other. If the pan head controller 12 is equipped with a tracing function separately from the automatic tracking function, an automatic tracing function can be used. Trace data can be created, registered, and used without distinction between a tracking function and a trace function. If multiple trace data can be created and registered, the trace data to be used in the automatic tracking function or the trace function can be selected from the multiple trace data created and registered regardless of these functions. It is also possible. On the other hand, even when the camera platform controller 12 is not equipped with a trace function (even if it is equipped with a trace function), it is possible to create and register exclusive trace data in the automatic tracking function, The trace data can be used only for the automatic tracking function.

Specifically, the trace data is used in the automatic tracking function as follows. For example, a moving object to be tracked (for example, a caster in a news program) mainly moves left and right, and hardly moves up and down, back and forth,
Or, even if the image is moved up and down and back and forth, the position in the up and down and back and forth directions may be almost determined by the position in the left and right direction. In such a case, if the pan position of the camera platform 10 is moved based on the barycentric coordinates of the moving object in the pan direction (horizontal direction of the screen), tilt,
Regarding the zoom and focus, the moving object can be tracked within the shooting range of the camera 40 even when the moving object is moved to a specific position that is predetermined with respect to the pan position. on the other hand,
The trace data is obtained by associating each position in a series of operations for pan, tilt, zoom, and focus. Therefore, if the pan, tilt, zoom, and focus positions when tracking a moving object are preliminarily linked by creating and registering trace data, the panning direction of the moving object can be set when performing automatic tracking. The pan position is moved so that the moving object is located at the center of the screen in the pan direction according to the barycenter coordinates of, and tilt,
The zoom and focus are moved to a position associated with the pan position by the trace data. As described above, by using the trace data in the automatic tracking function, the operation during the automatic tracking can be determined in advance so that the video during the automatic tracking becomes easy to see when creating the trace data. For example, it is possible to prevent the screen during automatic tracking from moving unnecessarily up and down in conjunction with a slight up and down movement of the moving object. Further, even when the camera 40 does not have an autofocus function, the focus adjustment can be automatically performed, and the zoom adjustment can also be automatically performed.

Next, the processing contents of the automatic tracking function in the camera platform controller 12 will be described in detail. First, a process of creating and registering the trace data used in the automatic tracking function will be described. When creating and registering trace data, the operator sets the pan head controller 12 to the trace registration mode. Then, a series of pan, tilt, zoom, and focus operations to be registered are executed by manual operation of the camera platform controller 12. For example,
In producing a news program or the like, it is assumed that the pan head 10 and the camera 40 are installed in the studio as shown in FIG. 11, and the caster M is photographed by the automatic tracking function. Then, it is assumed that the range in which the caster M moves during the actual shooting is limited to the position in the figure, the position, and the range in which the caster M moves between the positions (hatched portion in the figure). At this time, for example, assuming that the caster M has moved in the order of position, position, and position within that range, panning, tilting,
Execute focus and zoom operations manually.

During the manual operation in the trace registration mode, the CPU 60 of the pan head controller 12
From 0, the pan, tilt, zoom, and focus position data are sequentially obtained, and trace data as shown in FIG. 12 is created and registered in the memory 62. In the figure, the trace data includes data numbers (TNO) 1, 2, 3,... And position data of pan, tilt, zoom, and focus of each data number. In trace registration mode,
The position data of pan, tilt, zoom, and focus is based on the timing of the horizontal synchronization signal of the video signal (the time of one cycle is 1).
H)) and data numbers 1, 2, 3,...
Note that, in FIG. 11, the position, the position, and the trace data at the time of capturing each position are, for example, pan, tilt, zoom, and focus position data ((PANDA) of data numbers TNO = 1, 1000, and 2000 in FIG.
TA1, TILTDATA1, ZOOMDATA1, FOCUSDATA1), (PANDATA2, TIL
TDATA2, ZOOMDATA2, FOCUSDATA2), (PANDATA1, TILTDATA
1, ZOOMDATA1, FOCUSDATA1)). Further, the pan, tilt, zoom, and focus position data are converted and recorded in, for example, a 16-bit numerical value range of 0x0000 to 0xffff.

Next, the processing procedure of the automatic tracking function in the camera platform controller 12 will be described with reference to the flowchart of FIG. First, the operator creates trace data as described above and registers it in the memory 62 (step S50). Next, the camera platform controller 12 is set to the automatic tracking mode. Thereby, the pan head controller 12
CPU 60 starts the automatic tracking process (step S52). When the automatic tracking process starts, the CPU 60
Acquires the coordinates of the center of gravity of the moving object (the coordinates of the center of gravity in the pan direction) transmitted from the image processing apparatus 14 as described above (step S54). Next, it is determined whether or not the barycentric coordinates are outside the range of the dead zone set on the screen (step S56).

Here, the range of the dead zone is, for example, as shown in FIG.
Is set at a predetermined width at the center of the screen as shown by the hatched area of the shooting range L of the camera 40 shown in FIG. The barycentric coordinates are a
If the point is within the range of the dead zone as in the case of the point, NO is determined in the above step S56, and if the barycentric coordinate is outside the range of the dead zone as in the point b in the same figure, the determination is YES in the above step S56. You. The determination process in step S56 is performed only when the pan / tilt operation of the camera platform 10 is stopped. However, the operation may be performed even when the pan / tilt operation is not stopped.
Step S when the pan / tilt head 10 is performing the pan / tilt operation.
The flow shifts to the processing in step S58 without performing the determination processing in 56.

As described above, if NO is determined in step S56, the pan, tilt, zoom, and focus positions are not changed, that is, the process returns to step S54 while keeping these positions stationary. On the other hand, in step S56, YE
If determined to be S, then the CPU 60 obtains the focal length of the lens device 44 and calculates the pan movement amount based on the focal length and other known values as follows (step S58). ).

As shown in FIG. 15, on the screen of the photographing range L of the camera 40, the barycenter coordinate is a with respect to the center O of the screen.
It is assumed that a point is detected. At this time, the amount of movement of the camera platform 10 in the pan direction to match the point a with the center of the screen is called a pan movement amount, and the pan movement amount is obtained. Therefore, FIG.
5. As shown in FIG. 16, when the angle of view (horizontal angle of view) of the camera 40 in the horizontal direction is 2θ (unit: angle), θ is
From the focal length f of the lens device 44 and the image size H of the image sensor of the camera 40, it can be obtained by θ = tan ⁻¹ (H / 2 / f). Assuming that the resolution (horizontal resolution) of the image is R (unit: pixel) and the distance of the barycentric coordinates from the center of the screen on the screen is d (unit: pixel), the pan movement amount θm (unit: angle) is θm = (Θ × d) / (R / 2). The CPU 60 processes the pan position with a 16-bit value of 0x0000 to 0xffff for an angle range of 360 °, and the pan movement amount θm converted into the processed value is: θm = θm (unit: angle) × 0xffff / 360 °.

After calculating the pan movement amount θm as described above, subsequently, the pan tracking position TRK is calculated (step S60). The pan tracking position TRK is a position obtained by adding the pan movement amount θm to the current pan position. Next, C
The PU 60 searches the trace data registered in the memory 62 for one having position data on the pan closest to the pan tracking position TRK, and sets its data number to T
RKNO is set (step S62). Further, as can be seen from the processing described later, the CPU 60 controls the actual pan, tilt, zoom, and focus positions according to the pan, tilt, zoom, and focus position data of any data number in the trace data. The data number of the trace data specifying the current pan position is set to TNO (step S64).

Next, the CPU 60 determines that TRKNO> TN
It is determined whether it is O (step S66). If the determination is YES, 1 is added to TNO, and the value is set as TNO (step S68). And the data number is TNO
A control signal for instructing the movement of the trace data to pan, tilt, zoom, and focus positions is transmitted to the camera platform 10 and the lens device 44 (step S70). Next, it is determined whether or not TNO has become equal to TRKNO (step S72). If the determination is NO, the processing from step S68 is repeatedly executed, and the pan, tilt, zoom, and focus data numbers are set to TRKNO. To the position of the trace data. If YES is determined in the step S72, the step S54 is performed.
It is repeatedly executed from the processing of.

If NO is determined in step S66, 1 is subtracted from TNO and the value is set as TNO (step S74). Then, a control signal for instructing movement of the trace data having the data number TNO to the pan, tilt, zoom, and focus positions is transmitted to the camera platform 10 and the lens device 44 (step S76). Then, T
It is determined whether or not NO has become equal to TRKNO (step S78). If the determination is NO, the processing from step S74 is repeatedly performed, and the data number for pan, tilt, zoom, and focus is TRKNO. Move closer to the position of the trace data. Then, Step S78
If it is determined as YES, the process is repeatedly executed from the process of step S54.

With the above automatic tracking processing, the pan position moves based on the barycentric coordinates of the moving object to be tracked, and the tilt, zoom, and focus positions correspond to the positions associated with the main control position by the trace data. Move to become.

The above steps S68 and S7
In No. 4, the data number of the trace data is increased or decreased by one, and the pan, tilt, zoom, and focus are sequentially moved to the position of the increased or decreased data number, but according to the moving speed of the moving object. That is,
The magnitude of the value to be added to or subtracted from the data number in steps S68 and S74 may be varied according to the magnitude of the pan movement amount θm calculated in step S58 in the flowchart of FIG.

The trace data used in the automatic tracking function as described above can be created by a method different from that described above. In the method described above, the operator manually creates the trace data by actually moving the pan, tilt, zoom, and focus.However, a specific shot called a preset is registered and played. The trace data can also be created using the function to perform the trace. For example, in FIG.
The position of pan, tilt, zoom, and focus at the time of shooting the position, position, and position by automatic tracking is registered as a shot. That is, in FIG.
Only the data of 0 and 2000 are shot registered. Note that shot registration can be performed manually by actually operating pan, tilt,
After setting the zoom and focus to the position where the shot registration is desired, the user can turn on a predetermined registration execution button. Then, intermediate data from the position to the position and from the position to the position are created by the following method.

As one method, from the pan, tilt, zoom, and focus positions registered for shooting the position to the pan, tilt, zoom, and focus positions registered for the shot to shoot the position, a preset function is used. The operation is actually performed, and then the operation is similarly performed from the pan, tilt, zoom, and focus positions registered for shots for photographing the position to the pan, tilt, zoom, and focus positions registered for shots for photographing the position. During this time, intermediate data of pan, tilt, zoom, and focus are taken in at the timing of the horizontal synchronization signal, and trace data is created and stored in the memory 62 based on the intermediate data.

As another method, intermediate data can be created by calculation. For example, if bread is 2 (° / s)
When operated with, position data change PA per 1H
NDATA1H is: PANDATA1H = 2 (° / s) × 0xffff / 360 (°) × 1 /
59.94. Therefore, the data number T from position to position
The position data for the pan of NO is calculated by the formula: PANDATA1 + PANDATA1H × TNO. Note that the total data number DATA is as follows: DATA = | PANDATA1-PANDATA2 | / PANDATA1H The position data for the pan from position to position is calculated in a similar manner. By the same operation, tilt,
Zoom and focus intermediate data can also be calculated. Then, the intermediate data calculated as described above is stored in the memory 62 as trace data. The above intermediate data is calculated in advance and stored in the memory 62.
Instead of storing the information in the tracking operation, it is also possible to obtain the value while calculating during the tracking operation.

A position directly controlled based on the barycentric coordinates of the moving object to be tracked is called a main control position, and is controlled to be a position associated with the main control position based on the trace data. Assuming that the position is a sub-control position, in the above embodiment, the position for pan is the main control position, and the position for tilt, zoom, and focus is the sub-control position. However, the main control position and the sub control position are not limited to this. The main control position can be a position in at least one of the moving directions in which the camera platform 10 (camera 40) can move. For example, in the camera platform 10 of the above embodiment,
Since the camera 40 can be moved in the directions of pan and tilt, the position can be set as the main control position for tilt instead of pan, and the position for both but not one of pan and tilt can be set as the main control position. It can also be a control position. Further, when there is a function of moving the camera 40 also in the height direction (height) as a function of the camera platform 10,
A position in at least one of the moving directions including the pan and tilt and the height can be set as the main control position. On the other hand, the sub-control position can be a position in at least one of a moving direction other than the moving direction of the camera 40 as the main control position, and zoom and focus of the camera 40. For example, if the main control position is a position for tilt,
The positions for pan, zoom, and focus can be the sub-control positions. If the main control position is the position for pan and tilt, the sub-control position can be the position for zoom and focus. In addition, pan, tilt,
Height, zoom, it is not necessary to include the position for all of the focus as a main control position or a sub-control position in the control target, for example, the main control position as a position for pan,
The sub control position may be only the position for tilt, or may be only the position for zoom or focus. The user may arbitrarily select what position among pan, tilt, zoom, focus, and the like is to be the main control position and the sub control position.

The format of the trace data is not limited to the above-described embodiment, but may be any data as long as the data associates the main control position with the sub-control position. It is not limited to.

Next, when there are a plurality of moving objects on the screen, an image processing apparatus 1 which identifies a desired moving object by hue and can track only the moving object.
4 will be described. FIG. 17 is a block diagram showing the configuration of the remote control pan head system in this case. 2 are denoted by the same reference numerals as those in FIG. 2, and the configuration and processing of the camera platform 10, the camera platform controller 12, and the camera 40 are the same as those in FIG. The description is omitted because it is completely the same.

The video signal input from the camera body 42 of the camera 40 to the image processing device 14 is converted into a Y / C separation circuit 46.
The luminance signal (Y signal) and the color difference signal (RY signal,
(BY signal). The separated luminance signal and color difference signal are converted into digital signals by the A / D converters 48 and 49, respectively (hereinafter, referred to as luminance signal image data and color difference signal image data, and when both are included, simply referred to as image data). The data is converted and input to the image memory 50. On the other hand, a synchronization signal of a video signal is given from the Y / C separation circuit 46 to the image processor 52,
Based on this synchronization signal, a data write command is given from the image processor 52 to the image memory 50 at a required timing. Thus, the image memory 50 stores image data for a plurality of frames at predetermined time intervals as described later.

The image data of the color difference signal stored in the image memory 50 is read out for each pixel in the range designated by the image processor 52, and the HUE conversion circuit 5
After being converted into data indicating the hue (HUE value) by 1, the HUE value is provided to the image processor 52. The HUE value (H) is obtained by the following equation.

H = tan ^-1 ((BY) / (RY)) (where (RY) and (BY) are respectively RY
The values of the image data of the signal and the BY signal are shown. The image processor 52 and the CPU 54 determine the camera 4 based on the image data of the luminance signal stored in the image memory 50 and the HUE value obtained from the HUE conversion circuit 51.
The coordinates of the center of gravity of a specific moving object to be tracked at 0 are detected as described later, and the detected coordinates of the center of gravity are transmitted to the pan head controller 12.

The processing procedure of automatic tracking (coordinate detection of the center of gravity) in the image processing apparatus 14 configured as described above will be described with reference to the flowchart of FIG. 18 and FIG. First, before starting the automatic tracking, the image processor 52
Creates a histogram of the HUE value (hue) of the moving object to be tracked (step S8). That is, a specific subject to be tracked is photographed by the camera 40 before the automatic tracking is started, and one frame worth of the subject is projected from the camera body 42 via the Y / C separation circuit 46 and the A / D converters 48 and 49. The image data is taken into the image memory 50 of the image processing device 14. The operator displays the image data stored in the image memory 50 on a monitor, and specifies the image range of the subject to be tracked with a mouse or the like. Image processor 52
Obtains the HUE value of each pixel from the HUE conversion circuit 51 based on the image data of the color difference signal in the image memory 50 for the specified range, and obtains the HUE of the specified range.
Create a histogram of values. Then, the histogram data is stored in a memory in the CPU 54.

Steps S10 to S22 are the same as those in the above embodiment. However,
Regarding the capture of the image in step S10,
Only the image data of the luminance signal is fetched, and step S14
The image data of both the luminance signal and the color difference signal is input to the image memory 50.
When obtaining the difference image between the images in step S18, the image data of the luminance signal is used.

FIG. 19A is a diagram showing an example of an image and an image taken into the image memory 50, where △ indicates a stationary object, and □ and ○ indicate a moving object. Step S above
For example, when a histogram of the HUE value of the moving object indicated by □ is created in 8, the moving object indicated by □ is the moving object to be tracked. FIG. 19B shows steps S18 to S2.
FIG. 7 is a diagram showing a difference image between an image obtained by the process 2 and only moving objects indicated by □ and ○.
Note that FIG. 19 shows a case where the image and the shooting range of the image coincide with each other, and the comparison frame is set to include the entire shooting range of the image and the image in step S16.

When the processing in step S22 ends, the CP
U54 attaches a labeling NO to each image of the moving object regarded as one based on the data of the difference image (step S23). Note that even if the moving objects are the same, different labeling NOs are assigned to the difference images unless they are integrated. FIG. 19 (C) shows the labeling NO on the image of the moving object.
The labeling NOs 1 to 4 are attached to the respective moving objects included in the difference image.

Subsequently, the image processor 52 obtains an area for each image of the moving object of each labeling NO (step S24). The CPU 54 obtains the area of the moving object of each labeling NO from the image processing processor 52, compares each area with a predetermined threshold value, and selects only the image of the moving object larger than the threshold value (step). S
26). This allows moving objects smaller than the threshold,
Alternatively, a moving object having little motion is excluded from the following processing. If there is no moving object larger than the threshold, the process returns to step S14, and the above-described processing is repeated.

Next, the image processor 52 has a CPU
The first moment is obtained for each moving object of the labeling NO selected by 54 (step S28), and the barycentric coordinates of each moving object on the screen are obtained (step S3).
0). FIG. 19D is a diagram showing the result of obtaining the barycentric coordinates of the subject of each of the labeling NOs 1 to 4, and the + position shown in the drawing indicates the barycentric position.

Next, the image processor 52 reads out the image data of the color difference signal of the image from the image memory 50 with respect to the pixel range of the moving object of each labeling No.
The UE conversion circuit 51 obtains the HUE value. Then, a histogram of the HUE value for the moving object of each labeling NO is created (step S34). continue,
A moving object closest to the hue of the moving object to be tracked created in step S10 is selected (step S36).
For example, in the histogram of the moving object to be tracked created in step S8, a moving object having the largest number of pixels of the HUE value having the maximum frequency and the HUE value in the vicinity thereof is selected. In the example shown in FIG. 19, when the histogram of the HUE value is created in step S8 with the moving object indicated by □ as the moving object to be tracked, the difference images labeled NO1 and NO2 are the moving objects to be tracked. Since the histogram of the HUE value is created based on the image data of the color difference signal, as shown in FIG. 19E, the moving object with the labeling NO1 is the moving object selected in the process of step S36.

The CPU 54 transmits the coordinates of the center of gravity of the moving object having the labeling NO selected as described above to the camera platform controller 12 (step S32).
The processing from step 10 is repeated. The processing in the pan head controller 12 when acquiring the barycentric coordinates of the moving object from the CPU 54 is exactly the same as described above.

As described above, by specifying the desired moving object to be tracked by its hue, it is possible to detect the barycentric coordinates of the desired moving object even when there are a plurality of moving objects on the screen. It is possible to appropriately and automatically track only the desired moving object.

The histogram of the HUE value of the moving object to be tracked is created by the operator by designating the moving object on the monitor. However, the present invention is not limited to this. An object may be detected, and a histogram of the HUE values of the moving object may be automatically created as the moving object to be tracked.

In the above embodiment, a case has been described in which automatic tracking is performed by a television camera. However, the present invention is not limited to this, and a moving object can be captured using a camera (for example, a digital still camera) that captures a still image. Can also be applied to automatic tracking of

[0075]

As described above, according to the automatic tracking apparatus of the present invention, the position in the moving direction of at least one of the pan, tilt, and height is set as the main control position, and the other moving directions are controlled. The position of at least one of zoom and focus of the camera is set as a sub-control position, and the main control position is controlled based on the position (coordinate of the center of gravity) of the moving object to be tracked. Is controlled so as to be a position associated with the main control position by the trace data, so that precise camera control can be performed according to the position of the moving object depending on the contents of the trace data created. The position in the camera movement direction (for example, the pan direction) with respect to the main movement direction of the moving object is set as the main control position, and the positions in the other camera movement directions (tilt direction, height direction) are set as the sub control positions. By
The sub-control position with respect to the main control position can be determined in advance by creating the trace data, and it is possible to make it easy to see the video being automatically tracked. Also, by setting the focus to the sub-control position, the focus can be automatically adjusted according to the position of the moving object even when the camera does not have an auto-focus function, and by setting the zoom to the sub-control position, the position of the moving object can be adjusted. Can be automatically adjusted according to.

[Brief description of the drawings]

FIG. 1 is an overall configuration diagram of a remote control pan head system equipped with an automatic tracking function to which the present invention is applied.

FIG. 2 is a block diagram showing a configuration of a remote control pan head system.

FIG. 3 is a flowchart illustrating a procedure of a barycenter coordinate detection of a moving object performed by an image processor and a CPU of the image processing apparatus in the automatic tracking mode.

FIGS. 4A to 4C are explanatory diagrams showing a relationship between a subject and a shooting range when a camera platform is stopped.

FIG. 5 is an explanatory diagram illustrating a difference image in the case of FIG. 4;

FIG. 6 is an explanatory diagram used for describing coordinates on a screen.

FIGS. 7A to 7C are explanatory diagrams showing a relationship between a subject and a shooting range when a camera platform is moving.

FIG. 8 is an explanatory diagram exemplifying a difference image in the case of FIG. 7;

FIG. 9 is an explanatory diagram illustrating a state of a comparison frame (comparison range) when the shooting range is shifted rightward.

FIG. 10 is an explanatory diagram showing a state of a comparison frame (comparison range) when the photographing range is shifted to the left.

FIG. 11 is an explanatory diagram used for describing creation and registration of trace data.

FIG. 12 is a diagram showing a configuration of trace data.

FIG. 13 is a flowchart illustrating a processing procedure of an automatic tracking function in the camera platform controller.

FIG. 14 is an explanatory diagram used for explaining a dead zone.

FIG. 15 is an explanatory diagram used for explaining the pan movement amount calculation;

FIG. 16 is an explanatory diagram used for explaining a pan movement amount calculation;

FIG. 17 is a block diagram showing a configuration of another embodiment of the remote control pan head system.

FIG. 18 is a flowchart illustrating a processing procedure of the image processing apparatus that identifies a moving object to be tracked based on hue and detects barycentric coordinates.

FIG. 19 is an explanatory diagram used for describing the processing in FIG. 18;

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 10 ... head, 12 ... head controller, 14 ... image processing apparatus, 40 ... camera, 42 ... camera main body, 44 ... lens apparatus, 46 ... Y / C separation circuit, 48, 49 ... A / D converter, 50 ... Image memory, 52: Image processor, 54: CPU, 60: CPU, 62: Memory

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 7/18 H04N 7/18 GF Term (Reference) 2F065 AA17 AA58 FF04 FF26 FF28 LL06 MM24 MM25 PP04 QQ04 QQ24 UU05 5C022 AB21 AB62 AB63 AB66 AC27 AC69 5C054 CF03 CF05 CF06 EA01 FC09 FC11 FC12 FC13 FC16 FF02 GA04 HA04 HA31 5L096 AA02 BA02 CA02 DA02 FA15 FA57 FA60 FA67 FA69 FA76 GA08 GA41 HA03 HA04 LA04 LA05 LA12

Claims (4)

[Claims] A camera is supported by a camera platform so as to be movable in at least one of pan, tilt, and height directions, and the position of a moving object to be tracked from an image captured by the camera is determined. Detecting, in an automatic tracking device that moves the camera in the moving direction so that the moving object is not outside the shooting range of the camera based on the detected position, of the movable directions of the camera, A position in at least one of the moving directions is a main control position, and among the movable directions of the camera, a moving direction different from the moving direction related to the main control position, and a zoom and focus of the camera, Trace data for associating the sub-control position with the main control position is created using the position of at least one of the sub-control positions as a sub-control position. And a trace data generating unit configured to control the main control position so that the position of the moving object does not fall outside the photographing range of the camera in the moving direction related to the main control position, and Control means for controlling the sub-control position so as to be a position associated with the trace data. 2. The automatic tracking device according to claim 1, wherein the main control position is a position for panning, and the sub-control position is a position for tilt, zoom and focus. 3. The trace data according to claim 2, wherein each of the trace data is a record of a position at each point in time at a predetermined time interval with respect to a series of operations for pan, tilt, zoom, and focus. Automatic tracking device. 4. The automatic tracking apparatus according to claim 1, further comprising a detecting means for identifying a desired moving object to be tracked by hue and detecting the position of the identified moving object to be tracked.