JP2002112250A - Automatic tracking device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an automatic tracking device that can automatically track a mobile object with a low-cost circuit by deciding the setting of a comparison range of images on the basis of a moving speed of a universal head in the case of comparing the images by two frames captured from a camera during movement of the universal head so as to extract an image of the mobile object. SOLUTION: The automatic tracking device captures an image (1) of a photographing range L during movement of the universal head and then captures an image (2) of a photographing range L'. Then a range where the photographing ranges of both the images are in duplicate on a picture of the images (1), (2) is decided as a comparison range (comparison frame P). The comparison range P is obtained by calculating how much the photographing range L' of the image (2) is shifted with respect to the photographing range L of the image (1) on the basis of a moving speed (panning speed) of the universal head. Then comparing the images (1), (2) within the comparison frame P extracts the image of the mobile object to set the suitable moving speed of the universal head on the basis of a position of the image of the mobile object of the picture.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic tracking device, and more particularly, to an automatic tracking device that automatically moves a shooting direction of a camera and automatically tracks a moving object.

[0002]

2. Description of the Related Art An automatic tracking device is a device for automatically moving a shooting direction of a camera by panning / tilting a pan head mounted with a television camera by driving a motor and automatically tracking a moving object (moving object). . As an automatic tracking method, a method described in JP-A-59-208983 is generally known. According to this method, images of two frames (two screens) at predetermined time intervals are captured from a video signal output from a camera, and an image of a moving object is extracted by comparing the images. Then, the moving amount and moving direction of the moving object are detected from the position of the center of gravity of the extracted image of the moving object, and based on this, the shooting direction of the camera is set so that the center of gravity of the moving object is always at the center of the screen. Automatically controlled by pan / tilt. In this specification, the operation of the camera platform for moving the shooting direction of the camera in the pan direction or the tilt direction is referred to as the movement of the camera platform in the pan or tilt direction, or simply as the movement of the camera platform.

Incidentally, the above-mentioned Japanese Patent Application Laid-Open No. Sho 59-20898 is disclosed.
In the automatic tracking method described in Japanese Patent Application Laid-Open No. 3 (1999) -1995, since images of a moving object are extracted by comparing images of two frames captured from a camera, while capturing the images to be compared,
The head had to be stopped. In other words, if the shooting range of the compared image differs due to the movement of the camera platform,
Since the image of the stationary object also moves on the screen and the image of the moving object cannot be identified, the camera platform cannot be moved while capturing the image to be compared. For this reason,
The above-described automatic tracking method has a drawback that a fast-moving moving object cannot be tracked, and the camera platform repeatedly moves and stops, making the screen difficult to see.

[0004] In order to solve such a drawback, the applicant of the present invention disclosed in Japanese Patent Application No. 10-345664 so that an image of a moving object can be extracted from an image in a different photographing range. We have proposed an automatic tracking method that can recognize images of moving objects.
According to this, regardless of whether the head is stopped or moving
At the same time as capturing an image from the camera, the pan / tilt position of the camera platform at that time is acquired. After a predetermined time 2
When the image of the frame is captured, the photographing range of the camera at the time of capturing each image is obtained based on the pan / tilt position at the time of capturing each image, and the overlapping range of the capturing range of each image is compared with the comparison range. To be determined. Since the image of the stationary object does not move within this comparison range, it is possible to extract only the image of the moving object by comparing the images within the comparison range.

[0005]

However, as in the automatic tracking method proposed in Japanese Patent Application No. 10-345664, the photographing range of each image is determined based on the pan / tilt position in order to determine the comparison range. If such calculation is performed, high-precision and high-resolution position detection is required, and high time accuracy for accurately detecting the pan / tilt position at the time of capturing each image is also required. That is, when the comparison range is obtained as described above, the amount by which the shooting range of one image is shifted with respect to the shooting range of the other image (shift amount).
Must be obtained in units of pixels constituting the image, and 1
It can be seen that the displacement of the pan / tilt position corresponding to the shift of the pixel is quite small in a sense. Therefore, in order to accurately detect such a minute displacement by position detection, it is indispensable to use an expensive sensor and an A / D converter with a high resolution, resulting in a high cost. When a stepping motor is used for pan / tilt driving, a position detection sensor is not essential, and may not be provided. In this case, there is a disadvantage that the pan / tilt position data is not output, and it is impossible to obtain a comparison range therefrom.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and when comparing two frames of images captured from a camera while moving a camera platform to extract an image of a moving object, the images are compared. It is an object of the present invention to provide an automatic tracking device that can determine a comparison range to be determined based on a position of a camera platform at the time of capturing an image, without using a cost.

[0007]

According to the present invention, in order to achieve the above object, the invention according to claim 1 is characterized in that the camera is moved while moving the shooting direction by moving a camera platform on which the camera is mounted. Sequentially capture the image captured in, while extracting the image of the moving object by comparing the captured image, based on the position of the image of the moving object on the screen and the moving object outside the shooting range of the camera In an automatic tracking device for controlling the moving speed of the camera platform so as not to be disturbed, an image storage means for storing at least two screens of images taken from the camera, and an image for two screens stored by the image storage means. Moving speed acquiring means for acquiring the moving speed of the camera platform when capturing from the camera; and based on the moving speed of the camera platform acquired by the moving speed acquiring device. A comparison range determining unit that determines, as a comparison range, a range in which an image capturing range when each image of the two screens is captured by the camera is determined by the comparison range determining unit from each image of the two screens. Image processing means for extracting an image of the moving object extracted by extracting the image within the comparison range, and comparing the extracted images, and an image of the moving object image extracted by the image processing means on a screen. Moving speed control means for controlling the moving speed of the camera platform based on the position so that the moving object does not fall outside the shooting range of the camera.

[0008] The invention described in claim 2 is the first invention.
In the invention described in (1), when the angle of view of the camera is variable, the 2
When the image of the screen is captured from the camera, the camera further includes a photographing angle of view acquiring unit that acquires a photographing angle of view of the camera, and the comparison range determining unit includes a photographing angle of view acquired by the photographing angle of view acquiring unit. The comparison range is determined based on a moving speed of the camera platform.

According to the present invention, the comparison range for extracting the image of the moving object from the image captured by the camera is determined based on the moving speed of the camera platform. The comparison range can be determined by using inexpensive parts as compared with the case where the comparison range is determined based on the comparison range. That is, the time interval for capturing each image for two screens can be accurately determined, and since the time interval is short, for example, the resolution and accuracy of speed detection for detecting that the shooting range has been shifted by one pixel. May be lower than in the case of position detection, which allows the use of inexpensive sensors and A / D converters. In addition, if an image for two screens is captured while the moving speed of the camera platform is constant, it is not necessary to exactly match the time of capturing the image with the time of detecting the moving speed, and high time accuracy is not required. . Furthermore, when a stepping motor is used for pan / tilt driving, the comparison range can be obtained from the moving speed without providing a position detection sensor.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of an automatic tracking device according to the present invention will be described below in detail with reference to the accompanying drawings.

FIG. 1 is an overall configuration diagram of a remote control pan head system equipped with an automatic tracking function to which the present invention is applied. The remote control head system shown in FIG. 1 includes a remote control head 10 (hereinafter, simply referred to as a head 10), a head controller 12, and an image processing device 14. Head 10
It comprises a housing 15 for housing a television camera (hereinafter simply referred to as a camera), and a camera platform body 16 for rotating the entire housing 15 to pan and tilt the camera in the housing 15. A transparent protective glass 17 is provided on the front surface of the housing 15, and a camera housed in the housing 15 captures an image outside the housing 15 through the protective glass 17.

The housing 15 is supported on a tilt shaft (not shown) extending from the camera platform main body 16, and the tilt shaft is driven to rotate by a tilt motor built in the camera platform main body 16. Therefore, when the tilt motor is driven, the housing 15 rotates through the tilt shaft, and the camera in the housing 15 tilts. The pan head body 16 is supported by a pan shaft 18 fixed on a mounting table (not shown).
6 is rotated by a built-in pan motor. Therefore, when the pan motor is driven, the housing 15 rotates together with the camera platform body 16, and the camera in the housing 15 pans.

The camera platform controller 12 is connected to the camera platform 10 via a cable (it is also possible to connect via a communication line such as a dedicated line or a public line). A control signal is transmitted to the camera platform 10 based on the operation of the various operating members provided, and various operations of the camera platform 10 and the camera mounted on the camera platform 10 are controlled.
When the automatic tracking mode is set, the pan head controller 12 transmits a control signal to the pan head 10 based on a signal given from the image processing device 14 and moves the pan head 10 (pan / tilt operation). ) And the moving object (moving object) is tracked by the camera.

FIG. 2 is a block diagram showing the configuration of the remote control pan head system. As shown in FIG.
The camera 40 housed in the housing 15 is composed of a camera body 42 and a lens device 44 mounted on the camera body 42. The camera body 42 has an image sensor and necessary processing circuits mounted thereon. An image (moving image) formed on the image sensor via the optical system of the lens device 44 is converted into a video signal (in this embodiment, an NTSC system). As a video signal). The control of the photographing operation such as the start and end of photographing in the camera body 42 is performed based on a control signal given from the camera platform controller 12 via the camera platform 10.
The lens device 44 is equipped with optical members such as a focus lens and a zoom lens that can be driven by a motor. The focus and zoom of the camera 40 are adjusted by moving the focus lens and the zoom lens. Control regarding the operation of the lens device 44 such as focus and zoom is performed based on a control signal given from the camera platform controller 12 as in the camera body 42.

The pan head and the tilt motor are mounted on the pan head 10 as described above.
When these pan motors and tilt motors are driven by the control signal given by the camera 40, the camera 40 pans or tilts, and the shooting range of the camera 40 moves. The operation of the camera platform 10 when panning or tilting the camera 40 is referred to as the camera platform 10 moving in this specification.

As shown in FIG. 1, the image processing device 14
/ C separation circuit 46, image memory 50, image processing processor 52, various signal processing circuits such as CPU 54. These signal processing circuits operate effectively when the pan head controller 12 is in the automatic tracking mode. I do. The video signal output from the camera body 42 is input to the image processing device 14, and the video signal is separated into a luminance signal (Y signal) and a color difference signal by the camera body Y / C separation circuit 46 in the automatic tracking mode. Is done. The separated luminance signal is converted into a digital signal (hereinafter referred to as image data) by an A / D converter 48 and input to an image memory 50. On the other hand, a synchronizing signal of a video signal is given from the Y / C separation circuit 46 to the image processor 52, and a command for writing data from the image processor 52 to the image memory 50 at a required timing based on the synchronizing signal. Is given. Thus, the image memory 50 stores image data for a plurality of frames at predetermined time intervals as described later. It should be noted that an image for one frame is composed of an image for two fields since the NTSC video signal employs an interlaced system. In the present specification, when one frame of image data is called, One of a series of images taken continuously
In the present embodiment, one frame of image data refers to image data of one field.

The image data for a plurality of frames stored in the image memory 50 as described above is read and image-processed by the image processor 52 as described in detail below, and the result of the image processing is Based cpu
The arithmetic processing is performed by 54 to calculate the moving speed (pan speed and tilt speed) of the camera platform 10 in the pan direction and the tilt direction for automatically tracking the moving object with the camera 40. The CPU 54 calculates the moving speed (for calculating a later-described comparison frame), the focal length (angle of view) of the lens device 44 and the moving speed (pan speed and tilt speed) of the camera platform 10 at each time. ) Is provided from the pan head controller 12.

When the camera controller 12 is set to the automatic tracking mode, the camera controller 12
The moving speed calculated by the CPU 54 of the image processing apparatus 14 is given as a command signal, and a control signal is transmitted from the pan head controller 12 to the pan head 10 so as to have the moving speed. On the other hand, when the automatic tracking mode is not set, a control signal is transmitted from the pan head controller 12 to the pan head 10 based on the operation of the various operation members provided in the pan head controller 12 as described above, Various operations of the camera platform 10 and the camera 40 mounted on the camera platform 10 are controlled.

The image processing processor 52 and the CPU of the image processing apparatus 14 when the automatic tracking mode is set in the remote control platform system configured as described above.
The processing contents of 54 will be described based on the flowchart of FIG. In the flowchart of FIG. 3 and the following description, processing relating to movement of the camera platform 10 in the pan direction is mainly described. However, processing similar to that in the pan direction is also performed for movement in the tilt direction.

First, the image processor 52 sends the Y / C separation circuit 46 from the camera body 42 of the camera 40 to the A / D
One frame of image data (this image is defined as an image) obtained via the converter 48 is loaded into the image memory 50 (step S10).

Subsequently, the image processor 52 determines whether or not Y /
Time for 4 fields (1 / 59.94 seconds per field) by the synchronization signal given from the C separation circuit 46
Is determined (step S12), and when the time for four fields has elapsed, one frame of image data (this image is defined as an image) is fetched into the image memory 50 in the same manner as the image (step S14). ).

When two frames of image data are stored in the image memory 50 as described above, the CPU 54 determines the current focal length (angle of view) of the camera 40 (lens device 44) and the moving speed of the camera platform 10. The comparison range (comparison frame) is obtained from the camera platform controller 12 based on the acquired image, and the comparison frame is designated to the image processor 52 (step S16). The moving speed data used for setting the comparison frame is not acquired from the pan head controller 12, but may be moving speed data instructed from the CPU 54 to the pan head controller 12 as described later. .

Here, the comparison frame will be described. As described later, the image processor 52 compares the image with the image, and generates a difference image by obtaining a difference between the values (hereinafter, pixel values) of respective pixels of the image data. The reason for generating the difference image is to extract an image of a moving object to be tracked from the two images.
In order to extract an image of a moving object from the difference image, it is necessary to obtain a difference image from images within the same shooting range (a range where the shooting ranges overlap) of the image shooting range and the image shooting range.

For example, when the camera platform 10 is stopped, for example, immediately after the start of the automatic tracking process, the image and the photographing range of the image are equal. FIG. 4A shows the relationship between the subject and the shooting range in this case, and the shooting range L of the image and the shooting range L ′ of the image completely match. In the drawing, the object indicated by a circle is a moving object, and the object indicated by a triangle is a stationary object. Further, it is assumed that the moving object is at the position indicated by A in the figure at the time of photographing the image, and has moved to the position indicated by B in the figure at the time of photographing the image.

At this time, the images and images stored in the image memory 50 form screens as shown in FIGS. 4B and 4C, respectively. On each of these images and images, The image (△) of the stationary object exists on the same coordinates, and only the image (○) of the moving object exists on different coordinates.

Accordingly, if the pixel value difference between the pixels at the corresponding positions (with the same coordinates) in the image and the entire screen range (within the entire photographing range) of the image is obtained, the pixel value of the stationary object is 0 in the difference image. As shown in FIG. 5, a difference image in which only the moving object is extracted is obtained. The coordinates of each point on the screen of each image are in accordance with the arrangement of pixels in a matrix forming one screen, and as shown in FIG. 6, the column numbers of the pixels of each point in the horizontal direction (from the leftmost pixel on the screen) 1 to X assigned in order (X corresponds to the number of pixels indicating horizontal resolution))
The vertical direction is represented by the row number of the pixel at each point (1 to Y (Y corresponds to the number of pixels indicating the vertical resolution) assigned in order from the pixel at the top of the screen).

On the other hand, when the camera platform 10 is moving, the photographing range differs between the image and the image. FIG. 7A shows the relationship between the subject and the shooting range in this case, and the shooting range L of the image is different from the shooting range L ′ of the image. Note that the same conditions as in FIG. 4A are applied to the subjects (moving objects and stationary objects) in the figure. At this time, the images stored in the image memory 50 are
Screens such as those shown in FIGS. 7B and 7C are respectively formed, and on each of these images and images, the image (△) of the stationary object does not exist on the same coordinates. Therefore, if a difference image is obtained in the entire screen range in the same manner as described above, a difference image as if the stationary object is a moving object is obtained, and only the moving object is appropriately extracted. Can not do.

Therefore, of the entire screen range of the image and the entire screen range of the image, a range where the photographing range overlaps, that is, FIG.
In (A), an image in a range where an image capturing range L and an image capturing range L ′ overlap each other is extracted from the image and the image, and a difference between pixel values at a corresponding position between the images in the extracted range is determined. By doing so, a difference image is obtained. As a result, the pixel value of the stationary object in the difference image becomes 0, and a difference image in which only the moving object is extracted can be obtained as shown in FIG.

In order to obtain a difference image in which a moving object is extracted from an image and an image, it is necessary to set a comparison range for obtaining the difference image on each screen of the image and the image. The comparison range is represented by a frame as a comparison frame. Then, the comparison frame is set as a frame of a shooting range where the shooting range L of the image and the shooting range L 'of the image overlap. When the camera platform 10 is stopped as shown in FIG. 4, the image shooting range L and the image shooting range L '
4B and 4C, the comparison frame P is set as a frame of the entire screen range on each screen of the image and the image as shown in FIGS. On the other hand, when the camera platform 10 is moving as shown in FIG. 7, the photographing range L of the image is different from the photographing range L ′ of the image.
As shown in (B) and (C), on each screen of the image and the image, a comparison frame P is set as a frame of a range where the image shooting range L and the image shooting range L 'overlap.

The specific processing for setting the comparison frame will be described. When an image and an image are fetched into the image memory 50 as described above, the CPU 54 sets the current focal length of the lens device 44 and the current movement of the platform 10. The speed (pan speed) is acquired from the pan head controller 12. Then, the CPU 54 obtains a shooting angle of view (horizontal angle of view indicating an angle range in the horizontal direction) from the focal length of the lens device 44, and calculates the image shooting range with respect to the image shooting range based on the shooting angle and moving speed. Is calculated by the number of pixels in the horizontal direction. Shift amount shift
(Unit: pixel) is V (unit: angle / second) for the moving speed (pan speed) of the camera platform 10, R (unit: pixel) for image resolution (horizontal resolution), and shooting angle of view (horizontal angle of view). A
(Unit: angle), the difference between the image capturing time of the image and the image (image capturing time interval between images) is T (unit:
Sec), shift = (R / A) × V × T (1) In calculating the vertical shift amount shift, the moving speed V is the tilt speed, the resolution R is the vertical resolution, and the shooting angle of view A is the vertical angle of view indicating the vertical angle range. If the image is captured after a lapse of four fields of time after the capture of the image, the photographing time difference T is equal to the time of one field (1/5).
(9.94 seconds).

When the shift amount shift is calculated by the above equation (1), the CPU 54 calculates the shift amount based on the shift amount shift.
Set the comparison frame on the image and image screen. That is, as shown in FIG. 9, the photographing range L 'of the image moves rightward with respect to the photographing range L of the image (the pan head 10 moves rightward).
In this case, for the comparison frame of the image, the outline of the range excluding the number of pixels of the shift amount shift from the left end of the screen (the range indicated by the diagonal lines in the figure) of the entire screen range of the image is set as the comparison frame P On the other hand, with respect to the comparison frame of the image, the outline of the range (shaded area in the figure) excluding the number of pixels of the shift amount shift from the right end of the screen in the entire screen range of the image is set as the comparison frame P. Set. As shown in FIG. 10, when the photographing range L 'of the image moves to the left with respect to the photographing range L of the image (the camera platform 10 moves to the left), the comparison frame of the image The outline of a range (a range indicated by oblique lines in the drawing) excluding the number of pixels of the shift amount shift from the right end of the screen in the screen range is set as the comparison frame P. Shift amount shif from the left edge of the screen within the screen range
Excluded area for the number of pixels of t (range shaded in the figure)
Is set as the comparison frame P. The same applies to the case where the camera platform 10 moves in the vertical direction.

Returning to the flowchart of FIG. 3, the comparison frame is set by the CPU 54 as described above, and when the comparison frame is instructed to the image processor 52, the image processor 52 , An image in the comparison frame is extracted from each of them, and a difference image between the images is obtained from the extracted image (step S1)
8). That is, the difference between the pixel values of the pixels at the corresponding positions (pixels at the same position in the comparison frame) is determined from the image data in the comparison frame between the images, and the absolute value is calculated (|-|). As a result, only the image of the moving object is extracted as shown in FIGS. 5 and 8, and the moving object is recognized as the subject to be tracked.

Subsequently, the image processor 52 binarizes the image data of each pixel of the difference image obtained in step S18 (step S20). By this processing, the pixel value of the image of the moving object becomes 1 ideally, and 0 otherwise. Then, a process of contracting the binarized difference image is performed to remove fine noise (step S22). In the following, the difference image that has been binarized and contracted is simply referred to as a difference image.

Next, the image processor 52 calculates the total number of pixels having a pixel value of 1 based on the difference image, and calculates the area of the entire image of the moving object (hereinafter, simply referred to as the area of the moving object) (step). S24). In the examples of FIGS. 5 and 8, the areas of two circles indicating images before and after the movement of the moving object are obtained. Then, the obtained area is given to the CPU 54.

When the area of the moving object is obtained from the image processor 52, the CPU 54 determines whether the area is larger than a predetermined threshold (step S2).
6). Here, when the determination is NO, there is no moving object enough to be tracked, when the moving object that has been tracked so far stops, when the movement of the moving object is small, or There are cases where only a small object that is not a tracking target is moving. At this time, a process for stopping the camera platform 10 is performed in steps S28 to S32. Working time coefficient moveco described later
unt is a moving object that has been moved and followed by moving the camera platform 10 by repeating the loop from step S14 to step S26 and step S28 to step S32, and is determined to be NO in step S26. This indicates whether or not the pan head 10 can be stopped by setting the moving object (hereinafter, referred to as a stopped moving object) at the center of the screen. First, the CPU 54 determines that the operation time coefficient movecount It is determined whether it is 0 (step S28). If the determination is YES, a command signal for stopping the movement of the camera platform 10 in the pan direction, that is, a command signal for setting the pan speed to 0, is transmitted to the camera platform controller 12 (step S30). Stop moving in the pan direction. At the time of determination in step S26, the camera
When the movement of the camera platform is stopped, the stopped state of the camera platform 10 is maintained. Then, the process returns to step S14. On the other hand, if NO is determined in step S28, the value of movecount is decreased by 1 (step S32),
The number of repetitions of steps S14 to S26 and steps S28 to S32 before stopping the camera platform 10 is reduced by one. When the moving object stopped by the processing of steps S28 to S32 becomes the center of the screen, the movement of the pan head 10 in the pan direction stops.

On the other hand, YES in step S26
Is determined, that is, when a moving object is detected from the difference image, the image processor 52 next calculates the first moment of the entire image of the moving object from the difference image (Step S).
34) Divide the first moment by the area of the moving object to obtain the center of gravity of the moving object (step S36). Here, the center of gravity of the moving object is represented by coordinates on the screen of the image, and the coordinates of the center of gravity are given to the CPU 54.

When given the coordinates of the center of gravity of the moving object from the image processor 52, the CPU 54 determines whether or not the camera platform 10 is in a stopped state (step S38). if,
If the determination is YES, the CPU 54 determines which position on the screen (on the screen of the image) the center of gravity of the moving object is, and determines whether the head 10 should be stopped or whether the head 10 should be stopped. In which direction left or right should be moved is determined by the processing of steps S40 to S46. First, C
The PU 54 includes a plurality of regions I, II, III, and symmetrically with respect to the center of the screen in the horizontal direction as shown in FIG.
Divide into IV. In the control related to the movement of the camera platform 10 in the tilt direction, the screen is similarly divided into four areas in the vertical direction. Here, the center of the screen (area I
B, the coordinate value of the boundary between the region I and the region III is A, and the coordinate value of the boundary between the region II and the region IV is C. Then, the range of the center of gravity of the moving object in the regions I, II, III, and IV is determined by the coordinate values in the horizontal direction of the moving object and the coordinate values A, B of the boundaries between the regions I, II, III, and IV. , C. If the center of gravity of the moving object is in region I
Alternatively, when it is determined that the camera is in the area II, the camera platform 10 is maintained in a stopped state, and when it is determined that the camera is in the area III, the camera platform 10 is moved rightward to determine that the camera is in the area IV. If it is determined, the camera platform 10 is moved to the left. As described above, when it is not necessary to move the camera platform 10 as in the case where the moving object is moving in the regions I and II, the camera platform 10 is maintained in a stopped state, and the camera platform 10 is prevented from moving wastefully. Is done.

Returning to FIG. 3, a specific processing procedure will be described. First, the CPU 54 determines whether or not the center of gravity of the moving object is larger than A, that is, the center of gravity of the moving object is in the region II in FIG.
It is determined whether it is within the range of I (step S40).
If the determination is YES, the moving direction of the camera platform 10 is determined to be the right direction (step S42). On the other hand, if the determination is NO, it is determined whether or not the barycentric coordinates of the moving object are smaller than C, that is, whether or not the barycenter of the moving object is within the range of the region IV in FIG. 11 (step S44). If the determination is YES, the moving direction of the camera platform 10 is determined to be the left direction (step S46). On the other hand, if NO is determined,
Since the center of gravity of the moving object is in the region I or the region II in FIG. 11, the process returns to the first process (step S10) while the stopped state of the camera platform 10 is maintained.

If the moving direction of the camera platform 10 is determined to be right or left in step S42 or step S46, the CPU 54 moves the camera platform 10 in the moving direction at a predetermined speed to the initial pan. The speed is initially set, and a command signal for commanding movement of the platform 10 at the pan speed is transmitted to the platform controller 12 (step S).
48). Thus, the stopped pan head 10 starts moving in the pan direction at a predetermined speed.

Subsequently, the CPU 54 calculates the above-mentioned operation time coefficient movecount (step S50). When the moving object stops, the operation time coefficient movecount is determined based on the current moving speed of the camera platform 10 in steps S14 to S14.
This shows how many times the loop of step S26 and steps S28 to S32 is repeated to stop the camera platform 10 so that the stopped moving object is positioned at the center of the screen. Therefore, the current moving speed (pan speed) of the camera platform 10 obtained from the camera platform controller 12 is V (unit: angle / second), the image resolution (horizontal resolution) is R (unit: pixel), and the shooting angle of view. (Horizontal angle of view)
(Unit: angle), the center coordinate in the horizontal direction on the screen is M, the barycentric coordinate of the moving object in the horizontal direction is P (unit: pixel), and the processing of one loop from step S14 to step S26 and step S28 to step S32 The required time is T (unit:
), The operation time coefficient movecount is calculated by the following equation (2), movecount = (PM) / (R × V × T / A) (2) However, due to rounding, etc., movecoun
t is an integer. By the processing of the above steps S28 to S32 using this operation time coefficient movecount,
When the moving object stops, the camera platform 10 can be stopped where the moving object is set at the center of the screen. In calculating the operation time coefficient movecount in the vertical direction, V is the tilt speed, R is the vertical resolution, A is the vertical angle of view, and P is the barycentric coordinates of the moving object in the vertical direction.

Next, the CPU 54 operates the pan head controller 1
It is determined whether or not the current moving speed (pan speed) of the camera platform 10 obtained from Step 2 has reached the pan speed instructed from the CPU 54 (Step S52). If YES, the process proceeds to Step S10. Return and repeat the above process.

If NO in step S38, that is, if it is determined that the camera platform 10 has already moved,
U54 determines the position on the screen where the center of gravity of the moving object is on the screen by the processing of the following steps S54 to S70, and determines whether the moving speed (pan speed) of the camera platform 10 should be increased or decreased. It is determined whether it should be performed. Thereby, the moving speed of the camera platform 10 is adjusted. To determine the position of the center of gravity of the moving object on the screen, the coordinate values A, B, and C shown in FIG. 11 are used. First, the CPU 54 determines whether or not the current moving direction of the camera platform 10 is rightward (step S54). Here, YES
Then, the CPU 54 determines whether or not the barycenter coordinate of the moving object is larger than A, that is, whether or not the barycenter of the moving object is within the range of the region III in FIG. S56). If the determination is YES, the pan head 10
Is slower than the moving speed of the moving object, the pan speed commanded to the pan head controller 12 is increased by a predetermined speed, and the command signal is transmitted to the pan head controller 12 (step S5).
8). Then, control goes to the step S50. on the other hand,
If NO is determined in step S56, the CPU 54 next determines whether the barycenter coordinate of the moving object is smaller than B, that is, whether the barycenter of the moving object is in the range of the region II or IV in FIG. Is determined (step S6).
0). If the determination is YES, the pan speed of the camera platform 10 is higher than the moving speed of the moving object, so the pan speed commanded to the camera platform controller 12 is decreased by a predetermined speed, and the command signal is issued. Is transmitted to the camera platform controller 12 (step S62). Then, control goes to the step S50. In this case, instead of lowering the pan speed of the camera platform 10, the pan speed may be set to 0, that is, the movement of the camera platform 10 in the pan direction may be stopped. On the other hand, if NO in step S60, that is, if it is determined that the center of gravity of the moving object is within the range of the region I in FIG. Therefore, the process proceeds to step S50 without changing the pan speed instructed to the pan head controller 12.

In the above case, the processing is performed when the current moving direction of the camera platform 10 is the right direction. However, when the current moving direction of the camera platform 10 is the left direction, the above-described step S54 is performed.
Is determined to be NO. In this case, the CPU 54
It is determined whether the coordinates of the center of gravity of the moving object are smaller than C, that is, whether the center of gravity of the moving object is within the range of the region IV in FIG. 11 (step S64). If it is determined as YES,
Since the pan speed of the camera platform 10 is lower than the moving speed of the moving object, the pan speed commanded to the camera platform controller 12 is increased by a predetermined speed.
The command signal is transmitted to the camera platform controller 12 (step S66). Then, control goes to the step S50. On the other hand, if NO is determined in step S54, the CPU 54 next determines whether the barycenter coordinate of the moving object is larger than B, that is, the barycenter of the moving object falls within the range of the region I or the region III in FIG. It is determined whether or not there is (step S68). If the determination is YES, the pan speed of the camera platform 10 is faster than the moving speed of the moving object, so the pan speed commanded to the camera platform controller 12 is decreased by a predetermined speed, and the command signal Is transmitted to the camera platform controller 12 (step S7).
0). Then, control goes to the step S50. In this case, instead of lowering the pan speed of the camera platform 10, the pan speed may be set to 0, that is, the movement of the camera platform 10 in the pan direction may be stopped. On the other hand, if NO in step S68, that is, if it is determined that the center of gravity of the moving object is within the range of the area II in FIG. 11, the pan speed of the camera platform 10 is appropriately adjusted with respect to the moving speed of the moving object. Therefore, the flow proceeds to step S50 without changing the pan speed instructed to the pan head controller 12.

Through the above processing, the moving speed of the camera platform 10 in the pan direction is appropriately adjusted according to the moving speed of the moving object. Steps S54 to S70
In the process (1), the pan speed is adjusted by raising or lowering the pan speed by a predetermined speed (variation amount). However, the present invention is not limited to this. ~ IV is further subdivided to determine the range of the area where the barycentric coordinates of the moving object are located, and as the area moves away from the center coordinates, the amount of change in the pan speed of the camera platform 10 is increased. Is also good. When the judgment area is divided into the finest parts, the pan head 1 is determined according to the distance from the center coordinates of the screen to the barycentric coordinates of the moving object.
The magnitude of the change amount of the pan speed of 0 is varied.

Next, a method of determining the shift amount shift (the shift amount of the image capturing range with respect to the image capturing range) in the setting of the comparison frame in step S16 will be specifically described. In the following description, a case where the shift amount in the horizontal direction is obtained by considering only the movement of the camera platform 10 in the pan direction will be described.

As shown in the above equation (1), the shift amount shift
Can be obtained based on the pan speed and the angle of view (focal length) of the camera platform 10. The values of the pan speed and the angle of view are determined by the CPU 54 acquiring predetermined data from the camera platform controller 12. be able to. For example, the pan speed data transmitted from the pan head controller 12 to the pan head 10 as a command signal is stored in the pan head controller 12.
, The value of the pan speed used for calculating the shift amount shift can be determined. The value of the pan speed used for calculating the shift amount shift is C
It may be a pan speed value instructed from the PU 54 to the pan head controller 12, or may be an actually measured pan speed obtained by a speed detection sensor in the pan head 10. On the other hand, the focal length is known by acquiring the value of the zoom control voltage (hereinafter, the zoom control voltage is simply referred to as the control voltage) transmitted from the camera platform controller 12 to the lens device 44 as a command signal. The value of the angle of view (horizontal angle of view) used for calculating the shift amount shift can be determined. However, since the relationship between the control voltage and the focal length differs depending on the type of the lens device 44,
In order to enable accurate calculation of the shift amount for various types of lens devices, characteristic data (a data table indicating the relationship between the control voltage and the focal length) corresponding to the type of lens device is registered in a memory in advance. In addition, it is necessary to determine the focal length from the control voltage using the characteristic data corresponding to the type of the lens device.

The shift amount shift is not theoretically obtained by using the above equation (1), but the actual shift amount shift for each value of the pan speed and the control voltage is measured before starting automatic tracking. Can be obtained in advance. In this case, for example, as shown in FIG. 12, the shift amount shift for each value of the pan speed and the control voltage acquired from the pan head controller 12
Is stored in the memory as shift amount data, the shift amount shift for the pan speed and the control voltage obtained from the pan head controller 12 is read out from the memory during the automatic tracking process, and based on this, To set a comparison frame. In general, at the time of automatic tracking, the zoom (focal length) of the lens device 44 is not changed. Therefore, before starting the automatic tracking, the value of the shift amount shift with respect to the focal length at the time of automatic tracking is set to each value of the pan speed. It is sufficient to determine the shift amount by actual measurement and store it in the memory as shift amount data.

FIG. 13 is a flowchart showing a processing procedure when the shift amount shift for a predetermined focal length and pan speed is obtained by actual measurement. During the execution of this process, it is necessary that no moving object exists in the shooting range of the camera 40. However, even if there is a moving object, there is no problem if the moving object is small with respect to the screen. First, the CPU 54 initializes the parameter minarea to a maximum value (for example, 0xffffff), the parameter i to 1, and the shift amount shift to be obtained as shift amount data to 0 (step S100). Subsequently, the CPU 54 sets the pan speed of the pan head 10 instructing the pan head controller 12 to a predetermined speed (a pan speed for which the shift amount shift is to be obtained), and sends a pan head command signal to the pan head. Transmit to controller 12 (step S
102). Then, it is determined whether or not the current pan speed of the pan head 10 obtained from the pan head controller 12 has reached the pan speed instructed by the CPU 54 (step S).
104), if the determination is YES, the process proceeds to the next step.

Next, the image processor 52 loads one frame of image data (this image is referred to as an image) into the image memory 50 in the same manner as in steps S10 to S14 in FIG. 3 (step S106). After a time corresponding to four fields has elapsed (step S108),
Further, one frame of image data (this image is referred to as an image) is loaded into the image memory 50 (step S11).
0).

When the image data for two frames is stored in the image memory 50 as described above, the CPU 54 sets the provisional shift amount sh of the photographing range of the image with respect to the photographing range of the image.
Assuming that ift 'is i, a comparison frame between images is set based on the shift amount shift' as in step S16 (step S112). Then, the comparison frame is designated to the image processor 52.

As described above, a comparison frame is set by the CPU 54, and when the comparison frame is instructed to the image processor 52, the image processor 52 and the CPU 54
Performs the same processing as the processing from step S18 to step S24 in FIG.
Perform at 20. First, the image processor 52 extracts an image in the comparison frame from the image and the image, and obtains a difference image between the images based on the extracted image (|
− |) (Step S114). Subsequently, the image processor 52 binarizes the image data of each pixel of the difference image (Step S116). Then, a process of contracting the binarized difference image is performed to remove fine noise (step S118).

Next, the image processor 52 calculates the total number of pixels having a pixel value of 1 based on the difference image, and calculates the area thereof (step S120). Then, the obtained area is given to the CPU 54. If the provisional shift amount shift 'set in step S112 appropriately corresponds to the pan speed, the area obtained here is ideally zero.

When the CPU 54 obtains the area from the image processor 52, the CPU 54 substitutes the area for the parameter area (step S122), and the parameter minarea
Whether the value of area is smaller than, that is, (minarea> ar
ea) is determined (step S124). If Y
If it is determined to be ES, the value of area is substituted for minarea (step S126), and the value of the shift amount shift finally obtained is set to i (however, it is not determined). On the other hand, if NO is determined in the step S124, the processes in the steps S126 and S128 are not performed. Subsequently, the CPU 54 increases the value of i by 1 (step S130), and determines whether i is greater than the number of pixels in the horizontal direction of the screen (horizontal resolution) (step S13).
2). If the determination is NO, the process returns to step S112. Thereby, the processing from step S112 to step S132 is repeatedly executed, and step S1 is executed.
The shift amount shift ′ = i used for setting the comparison frame when the area calculated in step S20 becomes the minimum is determined in step S12.
The shift amount obtained as the shift amount data by the processing of 8
determined by shift. If it is determined as YES in step S132, the above processing is terminated. If the above processing is performed by changing the pan speed set in step S102, the shift amount shif for each value of the pan speed
t can be obtained, and if the focal length is changed,
Cyst amount shif for each value of pan speed and focal length
t can be obtained.

As described above, the shift amount shift is obtained not by the theoretical value using the above equation (1) but by actual measurement, whereby the actual pan head of the pan head 10 acquired from the pan head controller 12 is obtained. Even when the pan speed of 10 varies, or when the focal length of the lens device 44 varies with respect to the control voltage,
An exact shift amount shift can be obtained. actually,
Since it is considered that the pan speed and the focal length vary due to differences in environment and characteristics of individual products, such a method of obtaining the shift amount shift is effective in more accurate automatic tracking. In addition, the shift amount by such actual measurement shift
Can be determined during the automatic tracking before the automatic tracking is started. That is, after capturing an image and an image during automatic tracking, the above-described step S11 is performed.
Steps 2 to S132 are performed, and the optimum shift amount shift is obtained by actual measurement. And the shift amount shif
A process of determining a panning speed by obtaining a difference image as in the flowchart shown in FIG. 3 based on t can also be performed. Also, in this case, when the moving object is large with respect to the screen, the shift amount shift may not be properly obtained. Therefore, even when the moving object is large with respect to the screen, the shift amount shift is appropriately obtained. In order to make it possible to correct the shift amount shift theoretically obtained by the above equation (1) as shown in FIG.
shift may be finally determined. That is, after taking in the image and the image, the shift amount shift is calculated by the above equation (1). Then, a comparison frame is set while increasing or decreasing the shift amount shift by one based on the shift amount shift, and a difference image is obtained based on the comparison frame to calculate the area of the moving object. If the shift amount shift is calculated to be smaller than the shift amount calculated by the above equation (1) and the minimum shift amount is detected first, the shift amount shift is set as the optimum shift amount shift. . Then, based on the shift amount shift, a process of obtaining a difference image and determining a pan speed is performed as shown in the flowchart of FIG.

The contents of the embodiment described above are based on the pan head 1
Not only can it be applied to the case where 0 moves in the pan direction and the tilt direction, but also can be applied to the case where it moves in other determined directions (for example, the case where the position of the camera is moved up, down, left and right). .

[0056]

As described above, according to the automatic tracking device of the present invention, the comparison range for extracting the image of the moving object from the image captured from the camera is determined based on the moving speed of the camera platform. Therefore, the determination can be made by using inexpensive parts as compared with the conventional case where the comparison range is determined based on the position of the camera platform. That is,
The time interval for capturing each image for two screens can be accurately determined, and since the time interval is short, for example, the resolution and accuracy of speed detection for detecting that the shooting range has been shifted by one pixel are determined by the position. It may be lower than in the case of detection, which allows the use of inexpensive sensors and A / D converters. In addition, if an image for two screens is captured while the moving speed of the camera platform is constant, it is not necessary to exactly match the time of capturing the image with the time of detecting the moving speed, and high time accuracy is not required. . Furthermore, when a stepping motor is used for pan / tilt driving, the comparison range can be obtained from the moving speed without providing a position detection sensor.

[Brief description of the drawings]

FIG. 1 is an overall configuration diagram of a remote control pan head system equipped with an automatic tracking function to which the present invention is applied.

FIG. 2 is a block diagram showing a configuration of a remote control pan head system.

FIG. 3 is a flowchart illustrating a procedure of an automatic tracking process;

FIGS. 4A to 4C are explanatory diagrams showing a relationship between a subject and a shooting range when a camera platform is stopped.

FIG. 5 is an explanatory diagram illustrating a difference image in the case of FIG. 4;

FIG. 6 is an explanatory diagram used for describing coordinates on a screen.

FIGS. 7A to 7C are explanatory diagrams showing a relationship between a subject and a shooting range when a camera platform is moving.

FIG. 8 is an explanatory diagram exemplifying a difference image in the case of FIG. 7;

FIG. 9 is an explanatory diagram showing a state of a comparison frame (comparison range) when the photographing range is shifted rightward.

FIG. 10 is an explanatory diagram showing a state of a comparison frame (comparison range) when the photographing range is shifted to the left.

FIG. 11 is an explanatory diagram showing a determination area on a screen used in setting the moving speed of the camera platform.

FIG. 12 is an explanatory diagram used to explain a case where a shift amount of a shooting range with respect to a pan speed of a camera platform and a control voltage (focal length) of zoom is stored in a memory in advance.

FIG. 13 is a flowchart illustrating a processing procedure when a shift amount of a shooting range with respect to a pan speed of a camera platform and a control voltage (focal length) of a zoom is stored in a memory in advance.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 10 ... Head, 12 ... Head controller, 14 ... Image processing apparatus, 40 ... Camera, 42 ... Camera main body, 44 ... Lens device, 46 ... Y / C separation circuit, 48 ... A / D converter, 50 ... Image memory , 52... Image processor, 5
4 ... CPU

Claims (2)

[Claims] An image captured by a camera is sequentially captured while moving a camera direction by moving a camera platform on which the camera is mounted, and the captured images are compared to extract an image of a moving object. An automatic tracking device that controls a moving speed of the camera platform so that the moving object does not fall outside a shooting range of the camera based on a position of an image of the moving object on a screen, at least an image captured from the camera. Image storage means for storing two screens; movement speed acquisition means for acquiring the movement speed of the camera platform when the images for two screens stored by the image storage means are captured from the camera; Based on the moving speed of the camera platform acquired by the means, a range where the photographing ranges when the images of the two screens are photographed by the camera overlaps with each other. A comparison range determining means for determining as a comparison range; an image within the comparison range determined by the comparison range determining means from each of the two screen images; and comparing the extracted images with the moving object. Image processing means for extracting an image of the camera, based on a position on the screen of the image of the moving object extracted by the image processing means, so that the moving object does not fall outside the shooting range of the camera An automatic tracking device, comprising: moving speed control means for controlling a moving speed. 2. A photographing angle of view for acquiring a photographing angle of view of said camera when images of two screens stored by said image storage means are fetched from said camera when the photographing angle of view of said camera is variable. 2. The apparatus according to claim 1, further comprising an acquisition unit, wherein the comparison range determination unit determines the comparison range based on a shooting angle of view and a moving speed of the camera platform obtained by the shooting angle of view obtaining unit. Automatic tracking device.