JP2017098723A - Tracking device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To continue tracking without losing sight of a subject even in the case where there is control delay in tracking control.SOLUTION: An automatic tracking device is configured to automatically track and image a subject that should be tracked, by operating any one of zooming, panning and tilting. The automatic tracking device comprises: subject recognition means for recognizing the subject; subject speed calculation means for calculating a speed of the subject that is recognized by the subject recognition means; target value determination means for determining target values of a position and a size with which the subject recognized by the subject recognition means should be held on a screen; and drive speed determination means for determining a drive speed of zooming, panning or tilting based on the position or the size of the subject recognized by the subject recognition means within the screen, the target values determined by the target value determination means and the speed of the subject calculated by the subject speed calculation means. At least either the target value determination means or the drive speed determination means changes a determination method in accordance with an approach/separation speed of the subject to/from the automatic tracking device.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an automatic tracking device that automatically tracks a tracking target, and more particularly to an automatic tracking pan head camera that turns in accordance with the operation of the tracking target.

  In recent years, there has been proposed an auto-tracking pan / tilt head camera system that automatically tracks a subject by zoom / pan / tilt control of the pan / tilt head camera. Japanese Patent Application Laid-Open No. 2004-133867 reports a method of automatically performing zoom control so as to keep the size of a subject in a shooting screen constant in the system as described above. In the prior art disclosed in Patent Document 2, when the subject speed in the shooting screen exceeds a preset reference speed, the subject speed and the reference speed are increased to the wide angle side until the subject speed is approximately the same. A method for performing automatic zoom control has been reported.

Japanese Patent No. 3546117 Japanese Patent No. 4293236

  However, in the conventional technique disclosed in Patent Document 1 described above, if there is a delay in automatic zoom control when the subject approaches the camera platform, the size of the subject in the screen increases, and accordingly the screen of the subject is increased. The internal speed may become relatively fast. If the speed of the subject in the screen increases, the camera platform cannot follow the subject and the subject cannot be kept at the center of the screen, and the subject may go out of the screen and be lost. In the prior art disclosed in Patent Document 2, zoom control is performed in accordance with the subject speed, so that the possibility of losing the subject is reduced.

  However, since the control delay in the zoom control is not taken into consideration as in the technique of Patent Document 1, if the zoom control delay occurs when the subject approaches the camera platform, tracking can be performed due to the increase in the relative speed of the subject. There is no possibility. Since zoom control is performed so that the speed of the subject matches the reference speed, the size of the subject on the screen may become unstable if the speed of the subject is not constant.

  Therefore, the present invention is a tracking system equipped with automatic zoom control that keeps the size of the subject on the screen constant. The purpose is to enable tracking without losing sight.

In order to achieve the above object, an automatic tracking device according to the present invention includes:
In an automatic tracking device that operates at least one of zoom, pan, and tilt, automatically tracks and shoots the subject to be tracked,
Subject recognition means for recognizing the tracking subject;
Subject speed calculation means for calculating the speed of the subject recognized by the subject recognition means;
Target value determining means for determining a target value of a position and a size at which the subject recognized by the subject recognizing means should be stopped on the screen;
Zoom / pan / tilt based on the position and size of the subject recognized by the subject recognition means, the target value determined by the target value determination means, and the subject speed calculated by the subject speed calculation means Drive speed determining means for determining the drive speed of
At least one of the target value determination means and the drive speed determination means changes the determination method according to the approaching / leaving speed of the subject with respect to the automatic tracking device.

  With the automatic tracking camera control device and the automatic tracking camera control method according to the present invention, automatic tracking can be performed without losing sight of the subject even if a delay occurs in zoom control.

It is a block diagram of a tracking system. It is the monitor screen which showed the area | region which recognizes a tracking object. It is a flowchart which shows the flow of the whole tracking process in an Example. 5 is a flowchart of tracking processing in the first embodiment. It is explanatory drawing which showed the effect of the method of Example 1. FIG. 9 is a flowchart of tracking processing in the second embodiment. It is explanatory drawing which showed the effect of the method of Example 2. FIG. FIG. 10 is a configuration diagram of a tracking system according to a third embodiment. 10 is a flowchart of tracking processing in the third embodiment. It is explanatory drawing which showed the effect of the method of Example 3. FIG.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[Example 1]
The configuration according to the first embodiment of the present invention will be described below with reference to FIG.

  The tracking system according to this embodiment includes an operating device 1, a camera platform 10, a camera 20, a lens 30, and a tracking processing device 40. The pan head 10 includes a pan head CPU 11, a tracking processing device communication unit 12, a pan control unit 13a, a tilt control unit 13b, a pan motor 14a, a tilt motor 14b, a zoom control unit 15a, a focus control unit 15b, and a camera control unit 16. The

  At the time of subject tracking, the camera platform 10 first receives an operation signal from the tracking processing device 40 by the tracking processing device communication unit 12. The pan head CPU 11 interprets the received operation signal and performs an operation suitable for the operation signal. When the operation signal is a pan / tilt drive command, the pan / tilt motors 14a / b are driven by controlling the pan / tilt control units 13a / b. In the case of a zoom / focus drive command, the lens 30 is driven by controlling the zoom / focus control units 15a and 15b. If the control command is for various cameras, the camera control unit 16 is controlled to control the camera 20.

  In the pan head control during non-tracking, an operation signal from the operating device 1 is transmitted to the pan head 10 via the tracking processing device 40, and the same control as described above is performed.

  The tracking processing device 40 includes an image processing unit 41, an image processing memory 42, an image processing processor 43, a template memory 44, a tracking processing device CPU 45, an inter-operator communication unit 46, and a pan-head communication unit 47. The inner tracking processing device CPU 45 includes a subject recognition unit 45a, a subject speed calculation unit 45b, a target value determination unit 45c, and a drive speed determination unit 45d as functions, and performs overall control of these functions.

  In the tracking processing device 40, first, the image processing unit 41 converts the video signal received from the camera 20 into image data for each frame and stores the image data in the image processing memory 42. The image processor 43 uses the image stored in the template memory 44 as a reference image, and performs template matching based on the luminance pattern on the image stored in the image processing memory 42 to determine the presence or absence of the subject. If there is a region where the similarity with the reference image in the image processing memory 42 exceeds a preset subject recognition threshold value TH, it is determined that a subject exists and the result is sent to the tracking processing unit CPU 45. .

  On the other hand, if there is no region exceeding the threshold TH, the image processing processor 43 assumes that no subject exists and sends the result to the tracking processing device CPU 45. When the subject exists, the subject recognition unit 45a in the tracking processing device CPU 45 calculates the coordinates of the four vertices of the subject region recognized as a rectangular shape in the screen shot by the camera 20. Then, the extracted subject information is sent to the subject speed calculation unit 45b and the drive speed determination unit 45d.

For example, FIG. 2 shows a monitor screen during recognition and tracking of a subject. In this case, the subject recognizing unit 45a coordinates (X ₁ , Y ₁ ), (X ₁ , Y ₂ ), (X ₂ , Y ₁ ) of the four vertices of the area recognized as the subject as shown in FIG. ), (X ₂ , Y ₂ ) are recognized as subject information. As a procedure for subject tracking control, first, the target value determination unit 45c sets the position and size at which the subject is to be tracked on the screen. Then, the driving speed determination unit 45d determines the pan / tilt driving speed of the camera platform 10 so that the subject position / size recognized by the subject recognition unit 45a approaches the set target value.

  Further, the tracking processing device CPU 45 can acquire the current pan head position information from the pan head CPU 11 through the pan head communication unit 47 and the tracking processing unit communication unit 12. The pan / tilt head position information is stored by the pan / tilt head CPU 11 receiving feedback from the pan control unit 13a, the tilt control unit 13b, and the zoom control unit 15b. The subject speed calculation unit 45b calculates the approaching / leaving speed of the subject with respect to the pan head 10 from the pan head position information and the amount of change in the size of the subject in the screen. In the first embodiment, the drive speed determination unit 45d switches the pan head control method according to the calculated approach / leaving speed.

  The basic processing flow of the tracking processing device CPU 45 in this embodiment will be described below with reference to the flowchart of FIG.

  First, it is determined whether or not there is an operation command from the controller 1 from the inter-operator communication unit 46 (S101). Next, it is determined whether tracking is currently in progress (S103). If tracking is not in progress, it is determined whether tracking is to be started (S104). If the tracking start condition is satisfied, the positions of the camera 20, the lens 30 and the camera platform 10 are shifted to the initial state (S105). When tracking is not in progress and the tracking start condition is not satisfied, the process returns to the operation signal reception process of S101.

  If tracking is in progress, the process proceeds to tracking processing (S106). Finally, a tracking end determination is made (S107), and a tracking end command is received from the controller 1, or the tracking end condition is satisfied, such as when the subject to be tracked has moved outside the screen. In step S108, the process shifts to tracking stop. The subject tracking operation is performed by repeating the processes of S101 to S108.

  Next, details of the tracking process S106 will be described with reference to the flowchart of FIG. First, the subject recognition unit 45a, which is a part of the tracking processing device CPU 45, receives subject information from the image processor 43 described above, and calculates whether or not the subject exists in the screen (if it exists) ( S201). Next, it is determined from the received subject information whether the subject exists in the screen (S202). If there is a subject in the screen, the subject speed calculation unit 45b calculates the approaching / leaving speed of the subject with respect to the camera platform 10 (S203).

  Then, the drive speed determining unit 45d determines the drive speed v of the camera platform 10 according to the approach / leaving speed (S204). If there is no subject in the screen, a tracking end flag is set (S205), and v = 0 is set so as to stop the pan head 10 (S206). Finally, an automatic tracking operation is performed by transmitting an operation signal to the camera platform 10 so as to operate at the determined driving speed (S207).

  The drive speed v determined in S204 is determined so that the zoom control gain increases as the speed at which the subject approaches the camera platform 10 increases, and the size of the subject converges quickly with respect to a preset target value. Is done.

  As described above, in this embodiment, when the subject is approaching the camera platform 10, the control gain for determining the zoom control speed is changed according to the approach speed.

  FIG. 5 shows the effect when this embodiment is used.

  Assuming a situation in which the subject approaches the camera platform 10 suddenly, in the conventional automatic zoom that does not consider the zoom control delay, the size of the subject on the screen is larger than the target size indicated by the dotted line as shown in FIG. As a result, the speed of the subject on the screen also increases.

  However, when this embodiment is applied, the size of the subject on the screen can be quickly converged to the target size as shown in FIG. 5B, so that an increase in the speed of the subject on the screen can be suppressed. I can do it.

  In this way, even when the subject suddenly approaches the camera platform 10, by suppressing the change in size of the subject on the screen, the increase in the speed of the subject on the screen can be reduced, and tracking can be continued without losing sight of the subject. I can do it.

  In this embodiment, the method for recognizing the subject is image recognition. However, other recognition means such as subject recognition using a radar may be used. Moreover, although the area | region which recognizes a to-be-photographed object was made into the rectangle, this shape is not specifically limited. Furthermore, although the speed of the subject is calculated using the size of the subject on the screen and the pan head position information, it may be calculated by another method.

  Further, in the tracking processing device 40, the image processing processor 43, the platform / operator communication units 46 and 47, and the tracking processing device CPU 45 are separated, but they may be processed by a single CPU or a template memory. For 44, a built-in memory of the CPU may be used. Further, the function of the tracking processing device 40 may be incorporated in the pan head 10 or may be integrated into the pan head CPU 11.

[Example 2]
The second embodiment of the present invention will be described below.

  The configuration is the same as that of the first embodiment shown in FIG. The basic tracking process is the same as the flowchart shown in FIG. However, since the processing of the tracking processing device CPU 45 in the tracking processing S106 in the overall flow is different, a flowchart of the processing of the tracking processing device CPU 45 in the second embodiment is shown in FIG.

  In the second embodiment, the target value calculation unit 45c switches the pan head control method according to the approaching / leaving speed calculated by the subject speed calculation unit. The overall flow is the same as in the first embodiment.

  First, as in the first embodiment, the subject recognition unit 45a receives subject information, calculates the subject position (S301), and determines whether the subject exists (S302). If there is a subject, the subject speed calculation unit 45b calculates the approaching / leaving speed of the subject with respect to the camera platform 10 (S303).

  Then, the target value determining unit 45c determines a target value of a size that should control the subject in the screen according to the approaching / leaving speed (S304), and the driving speed determining unit 45d determines the driving speed v of the camera platform 10. To do. If there is no subject in the screen, a tracking end flag is set (S305), and v = 0 is set so as to stop the pan head 10 (S306).

  Finally, an automatic tracking operation is performed by transmitting an operation signal to the camera platform 10 so as to operate at the determined driving speed (S307).

  The target value determined in S304 is set smaller as the speed at which the subject approaches the camera platform 10 is faster.

  As described above, in this embodiment, when the subject is approaching the camera platform 10, the target value for controlling the subject is changed according to the approach speed.

  FIG. 7 shows the effect when this embodiment is used.

  Assuming a situation in which the subject approaches the camera platform 10 suddenly, in the conventional automatic zoom that does not consider the zoom control delay, the size of the subject on the screen is larger than the target size indicated by the dotted line as shown in FIG. As a result, the speed of the subject on the screen also increases.

  However, when the present embodiment is applied, the size of the subject on the screen is equal to the target size even if a zoom control delay occurs by reducing the target size in advance as shown in FIG. 7B. And an increase in the speed of the subject on the screen can be suppressed.

  In this way, even when the subject suddenly approaches the camera platform 10, by suppressing the change in size of the subject on the screen, the increase in the speed of the subject on the screen can be reduced, and tracking can be continued without losing sight of the subject. I can do it.

  The derived system of the present embodiment is the same as the derived system described in the first embodiment.

[Example 3]
The third embodiment of the present invention will be described below.

  Since the functional configuration in the tracking processing device CPU 45 is different from those in the first and second embodiments, the configuration of the third embodiment is shown in FIG. As a function in the tracking processing device CPU 45, a tracking operation prohibited area determining unit 45e is added. The basic tracking process is the same as the flowchart shown in FIG. However, since the processing of the tracking processing device CPU45 in the tracking processing S106 in the overall flow is different, a flowchart of the processing of the tracking processing device CPU45 in the third embodiment is shown in FIG. The overall flow is the same as in the first embodiment.

  In the automatic tracking system of the third embodiment, a tracking operation prohibition area is provided in a screen for photographing a subject. The tracking operation prohibited area is an area in which the tracking operation of the camera platform 10 is stopped when the position of the subject is within a predetermined range in the screen shot by the camera 20. By providing the tracking operation prohibition region, it is possible to prevent a sensitive tracking operation from being performed with respect to a slight change in the position of the subject, and to achieve a stable operation with few hunting operations.

  Hereinafter, the flow of processing of the tracking processing device CPU 45 in the third embodiment will be described with reference to the flowchart of FIG.

  First, as in the first and second embodiments, the subject recognition unit 45a receives subject information, calculates the subject position (S401), and determines whether the subject exists (S402). If there is a subject, the subject speed calculation unit 45b calculates the approaching / leaving speed of the subject with respect to the camera platform 10 (S403). Then, the tracking operation prohibition area determination unit 45e determines a tracking operation prohibition area in the screen according to the calculated approaching / leaving speed (S404), and the driving speed determination unit 45d determines the driving speed v of the camera platform 10. (S405). If there is no subject in the screen, a tracking end flag is set (S406), and v = 0 is set so as to stop the pan head 10 (S407).

  Finally, an automatic tracking operation is performed by transmitting an operation signal to the camera platform 10 so as to operate at the determined driving speed (S408).

  The tracking operation prohibited area determined in S404 is set to a smaller range as the speed at which the subject approaches the camera platform 10 is faster.

  As described above, in the present embodiment, when the subject is approaching the camera platform 10, the tracking operation prohibited area is changed according to the approach speed.

  FIG. 10 shows the effect when this embodiment is used.

  Assuming a situation in which the subject is rapidly approaching the camera platform 10, the size and speed of the subject on the screen are increased. Therefore, in the conventional method in which the tracking operation prohibited area is not changed, the tracking operation starts slowly as shown in FIG. 10a. , Subject supplementation will not be in time.

  However, when this embodiment is applied, it can respond sensitively to the movement of the subject in the traveling direction as shown in FIG.

  In this way, in a situation where the speed of the subject on the screen has increased, it becomes possible to quickly respond to the movement of the subject and start the tracking operation, so that tracking can be performed without losing sight of the subject. Can be done.

  In this embodiment, it is assumed that the tracking operation prohibition area is provided at the center of the screen, but the position of this area may be set to any position within the shooting screen. In the method of the present embodiment, the size of the tracking operation prohibited area is changed according to the approach speed. However, when the area is set to other than the center of the screen, the position of the area is displayed instead of the size. You may make it change so that it may approach a center. The system derived from the present embodiment other than the above is the same as the system derived from the first embodiment.

1 operation unit, 10 pan head, 20 camera, 30 lens, 40 tracking processing device

Claims (5)

In an automatic tracking device that operates at least one of zoom, pan, and tilt, automatically tracks and shoots the subject to be tracked,
Subject recognition means for recognizing the tracking subject;
Subject speed calculation means for calculating the speed of the subject recognized by the subject recognition means;
Target value determining means for determining a target value of a position and a size at which the subject recognized by the subject recognizing means should be stopped on the screen;
Zoom / pan / tilt based on the position and size of the subject recognized by the subject recognition means, the target value determined by the target value determination means, and the subject speed calculated by the subject speed calculation means Drive speed determining means for determining the drive speed of
An automatic tracking device, wherein at least one of the target value determining means and the driving speed determining means changes a determination method according to an approaching / leaving speed of the subject with respect to the automatic tracking device. 2. The automatic tracking device according to claim 1, wherein the driving speed determining unit changes a zoom driving speed according to an approaching / leaving speed of the subject with respect to the automatic tracking device. 3. The automatic tracking device according to claim 1, wherein the target value determination unit changes a target size of the subject in accordance with an approaching / leaving speed of the subject with respect to the automatic tracking device. The driving speed determining means has a tracking operation prohibition area that stops driving when the subject is within a predetermined range, and the tracking operation is performed according to the approaching / leaving speed of the subject with respect to the automatic tracking device. The automatic tracking device according to any one of claims 1 to 3, wherein the prohibited area is changed. 5. The drive speed determination unit moves the tracking operation prohibited area toward the center of the screen in accordance with the approaching / leaving speed of the subject with respect to the automatic tracking device. An automatic tracking device according to claim 1.