JP2017204795A - Tracking apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an automatic-tracking imaging system which acquires an excellent image at the start of tracking.SOLUTION: A tracking processing apparatus 40 recognizes a subject to be tracked, on the basis of an image output from a first camera 20a which monitors a preset area, calculates a speed of the recognized subject, estimates a position to which the subject is to move, from the calculated speed, and controls an imaging optical axis of a second camera 20b, which can vary imaging optical axis or imaging angle-of-view, to be turned to the subject, on the basis of the estimated subject position. The tracking processing apparatus recognizes the subject to be tracked, on the basis of an image output from the second camera, and controls the imaging optical axis or the imaging angle-of-view with a universal head 10 so that the recognized subject may be captured at the center of an imaging screen of the second camera. After the second camera captures the subject, the imaging optical axis or the imaging angle-of-view of the second camera is controlled to track the subject for imaging.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an automatic tracking device that automatically tracks a tracking object, and more particularly to an automatic tracking camera system that tracks a tracking object by linking a plurality of cameras.

  In recent years, there has been proposed an auto-tracking pan / tilt head camera system that automatically tracks a subject by zoom / pan / tilt control of the pan / tilt head camera.

  As a prior art in the above-mentioned field, Patent Document 1 discloses a cloud in which an operator sets a target position of a tracking subject on the screen from an operating unit, and the subject is relatively stopped in the screen at the position during automatic tracking. Controlling the table is disclosed.

  In Patent Document 2, tracking shooting is performed by linking a fixed camera with a wide-angle lens and a wide shooting range with a telephoto camera that has a narrower angle of view than the fixed camera and can tilt the optical axis by drive control. A method has been proposed.

JP-A-8-74296 Japanese Patent No. 4188394

  By recognizing the subject with a fixed camera and controlling the telephoto camera based on the coordinate information of the recognized subject, it is possible to achieve both a wide search range and an enlarged image of the subject.

  However, in the conventional technique disclosed in Patent Document 1 described above, tracking may not be started when the subject is not within the angle of view of the camera. In particular, when a new subject appears from another direction while tracking one subject, the new subject cannot be recognized by image recognition, and thus tracking cannot be started.

  In the prior art disclosed in Patent Document 2, since a subject can be recognized from an image of a wide-angle fixed camera, a wide range of subjects can be searched.

  However, if the subject moves greatly while performing the operation of capturing the recognized subject within the angle of view of the telephoto camera, useless movement may occur in the capturing operation. Furthermore, if the subject goes out of the angle of view of the wide-angle camera during the capturing operation, there is a possibility that tracking shooting by the telephoto camera cannot be started.

  Therefore, the present invention is a useless operation even when a recognized subject moves during an operation of capturing with a camera platform in an automatic tracking camera platform system that tracks a subject by linking a plurality of cameras. The aim is to start tracking in a short time.

In order to achieve the above object, an automatic tracking device according to the present invention includes:
At least one first camera for monitoring a preset range;
A second camera that is mounted on at least one pan / tilt head device that drives at least one of zoom, pan, and tilt, and that can change a photographing optical axis or a photographing angle of view;
A tracking processing device for recognizing a subject to be tracked from image signals output from the first and second cameras and operating the second camera to perform control calculation for tracking the subject;
In an automatic tracking device composed of
The tracking processing device includes:
First camera subject recognition means for recognizing a subject to be tracked based on an image output from the first camera;
Subject speed calculation means for calculating the speed of the subject recognized by the first camera subject recognition means;
Subject position prediction means for predicting the position where the subject moves from the speed calculated by the subject speed calculation means;
Subject capturing means for controlling the camera platform device to direct the photographing optical axis of the second camera toward the subject based on the subject position predicted by the subject position predicting means;
Second camera subject recognition means for recognizing a subject to be tracked based on an image output from the second camera;
Second camera tracking control means for driving and controlling the camera platform so that the subject recognized by the second camera subject recognition means is captured at the center of the shooting screen of the second camera;
After the second camera subject recognizing unit recognizes the subject, the pan head device is controlled by the second camera tracking control unit instead of the subject capturing unit to perform tracking shooting of the subject.

  According to the control device for the automatic tracking camera and the control method for the automatic tracking camera according to the present invention, even when the recognized subject moves during the operation of capturing with the camera platform camera, the tracking can be performed in a short time without wasteful operation. Can start.

1 is a configuration diagram of a tracking system in Embodiment 1. FIG. FIG. 3 is an explanatory diagram showing the arrangement of cameras in Embodiment 1. 3 is a flowchart illustrating a flow of processing in the first embodiment. It is explanatory drawing which showed the effect of the method of Example 1. FIG. It is a block diagram of the tracking system in Example 2. 10 is a flowchart illustrating a processing flow in the second embodiment. It is explanatory drawing which showed the effect of the method of Example 2. FIG.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[Example 1]
The configuration according to the first embodiment of the present invention will be described below with reference to FIG.

  The tracking system of the present embodiment includes a pan head 10, a first camera 20a, a second camera 20b, a first lens 30a, a second lens 30b, and a tracking processing device 40. As shown in FIG. 2, in this embodiment, the first camera 20 a uses a fixed wide-angle camera, and the second camera 20 b uses a camera that can be pan / tilt / zoom driven by the camera platform 10. I use it. A and B in FIG. 2 indicate the angles of view of the first camera 20a and the second camera 20b, respectively. Each of the two cameras is connected to the tracking processing device 40 through a signal line, and communicates an image signal and a control signal.

  The pan head 10 includes a pan head CPU 11, a tracking processing device communication unit 12, a pan control unit 13a, a tilt control unit 13b, a pan motor 14a, a tilt motor 14b, a zoom control unit 15a, a focus control unit 15b, and a camera control unit 16. The

  At the time of subject tracking, the camera platform 10 first receives an operation signal from the tracking processing device 40 by the tracking processing device communication unit 12. The pan head CPU 11 interprets the received operation signal and performs an operation suitable for the operation signal. When the operation signal is a pan / tilt drive command, the pan / tilt motors 14a / b are driven by controlling the pan / tilt control units 13a / b. In the case of a zoom / focus drive command, the lens 30 is driven by controlling the zoom / focus control units 15a and 15b. If the control command is for various cameras, the camera control unit 16 is controlled to control the camera 20.

  The tracking processing device 40 includes an image processing unit 41, an image processing memory 42, an image processing processor 43, a template memory 44, a tracking processing device CPU 45, an inter-operator communication unit 46, and a pan-head communication unit 47. The inner tracking processing device CPU 45 functions as a first camera subject recognition unit 45a, a second camera subject recognition unit 45b, a subject speed calculation unit 45c, a subject position prediction unit 45d, a subject capture control unit 45e, and a subject tracking control unit 45f. And control these functions.

  In the tracking processing device 40, first, the image processing unit 41 converts the video signal received from the first camera 20 a and the second camera 20 b into image data for each frame and stores it in the image processing memory 42. The image processor 43 uses the image stored in the template memory 44 as a reference image, and performs template matching based on the luminance pattern on the image stored in the image processing memory 42 to determine the presence or absence of the subject. If there is a region where the similarity with the reference image in the image processing memory 42 exceeds a preset subject recognition threshold value TH, it is determined that a subject exists and the result is sent to the tracking processing unit CPU 45. .

  On the other hand, if there is no region exceeding the threshold TH, the image processing processor 43 assumes that no subject exists and sends the result to the tracking processing device CPU 45. The first camera subject recognition unit 45a and the second camera subject recognition unit 45b in the tracking processing device CPU 45 recognize the subject recognized as a rectangular shape in the screen to be photographed when the subject exists in the screen of the corresponding camera. Calculate the coordinates of the four vertices of the region. When the subject is observed in the first camera 20a, the extracted subject information is sent to the subject speed calculation unit 45c and the subject position prediction unit 45d. When a subject is observed by the second camera 20b, the extracted subject information is sent to the subject tracking control unit 45f.

  Further, the tracking processing device CPU 45 can acquire the current pan head position information from the pan head CPU 11 through the pan head communication unit 47 and the tracking processing unit communication unit 12. The pan / tilt head position information is stored by the pan / tilt head CPU 11 receiving feedback from the pan control unit 13a, the tilt control unit 13b, and the zoom control unit 15b. When the subject is observed with the first camera 20a, the subject speed calculation unit 45c calculates the speed of the subject from the pan head position information and the amount of movement of the subject within the screen. The subject position prediction unit 45d performs subject movement prediction based on the calculated subject speed and pan head position information, and subject information extracted by the first camera subject recognition unit 45a.

  Then, the subject capturing control unit 45e calculates a driving speed necessary to capture the subject within the screen of the second camera 20b from the standing subject movement prediction, and sends a control command to the pan head CPU 11. When the subject is observed with the second camera 20b, the subject tracking control unit 45f uses the subject information extracted by the second camera subject recognition unit 45b to track the subject so as to keep the subject at the center of the screen. And a control command is sent to the pan / tilt head CPU 11.

  The basic processing flow of the tracking processing device CPU 45 in this embodiment will be described below with reference to the flowchart of FIG.

  When the tracking process is started, first, the first camera subject recognition unit 45a receives subject information from the image processor 43 described above, and determines whether or not a subject exists in the screen (S101). If there is no subject, the detection process is repeated. If it exists, the position is calculated (S102). Next, the subject speed calculation unit 45c calculates the movement speed of the subject (S103), and the subject position prediction unit 45d predicts the movement of the subject based on the information (S104). Then, the subject capture control unit 45e calculates the driving speed of the camera platform 10 for capturing the subject on the screen of the second camera 20b in the shortest time based on the predicted motion of the subject, and performs capture control. It performs (S105).

  The second camera subject recognition unit 45b determines whether the subject has been captured on the screen of the second camera 20b (S106), and if not captured, repeats the processing of S102 to S105. If capture is successful, the subject capture control unit 45e performs capture control. Instead, the second camera subject recognition unit 45b acquires subject information (S107), and the subject tracking control unit 45f captures the subject to the second camera. Tracking control is performed so as to remain at the center of the screen 20b (S108). Finally, it is determined whether or not the tracking is to be ended (S109). If the tracking is not to be ended, S107 to S109 are repeated. When the tracking is finished, the process returns to S101 to search again for the subject.

  As described above, in the present embodiment, the movement of the subject is predicted from the image of the first camera 20a, the camera platform is controlled based on the prediction, and the second camera screen is displayed in the shortest time. Control to capture the subject. FIG. 4 shows the effect when this embodiment is used. First, after the subject is recognized by the first camera 20a, capture control is performed so that the subject is photographed by the second camera 20b. In the prior art shown in FIG. 4a, since capture control is always performed toward the current position of the subject, if the subject moves during the capture control operation, the subject cannot be captured at the shortest distance.

  Further, if the subject goes out of the screen of the first camera 20a before the capture is completed, the current position of the subject cannot be acquired, and thus capture control cannot be performed. However, when this embodiment is applied, since the pan head 10 is driven and controlled at the shortest distance using the movement prediction of the subject as shown in FIG. 4B, the subject is captured by the second camera 20b without useless operation. be able to. In addition, even when the subject goes out of the screen of the first camera 20a before the capture is completed, the subject capture control is continued by controlling the camera platform 10 using the movement prediction of the held subject. can do.

  In the present embodiment, the first camera 20a is a camera fixed at a wide angle. However, the first camera 20a may be driven using a driving device such as a pan head. Although two cameras are used, a plurality of cameras corresponding to the roles of the first and second cameras may be used. Although the method for recognizing the subject is image recognition, other recognition means such as subject recognition using a radar may be used. Moreover, although the area | region which recognizes a to-be-photographed object was made into the rectangle, this shape is not specifically limited.

  Furthermore, although the speed of the subject is calculated using the size of the subject on the screen and the pan head position information, it may be calculated by another method. In the tracking processing device 40, the image processing processor 43, the pan / tilt head communication unit 46, and the tracking processing device CPU 45 are separate from each other. However, the processing may be performed by a single CPU. Alternatively, a built-in memory may be used. Further, the function of the tracking processing device 40 may be incorporated in the pan head 10 or may be integrated into the pan head CPU 11.

[Example 2]
The second embodiment of the present invention will be described below.

  FIG. 5 shows the system configuration in this embodiment. Compared to the first embodiment, a subject size prediction unit 45f is added to the functional configuration in the tracking processing device CPU45. The flow of the basic tracking process is shown in FIG.

  First, as in the first embodiment, the first camera subject recognition unit 45a determines the presence or absence of a subject (S201), and repeats the detection process when the subject does not exist. If it exists, its position and size are calculated (S202). Next, the subject speed calculation unit 45c calculates the moving speed of the subject (S203), and the subject size change speed calculation unit 45d calculates the size change speed of the subject on the screen (S204). Then, the subject position predicting unit 45e predicts the subject movement based on the moving speed of the subject (S205), and the subject size predicting unit 45f captures an image with the second camera 20b based on the subject size change speed. The size of the subject is predicted (S206).

  Then, the subject capture control unit 45g captures the subject in the shortest time based on the set motion prediction and size prediction of the subject, and allows the subject to be tracked and captured with the optimum size from the point of capture. The drive speed of the table 10 is calculated and capture control is performed (S207). The subsequent processes S208 to S211 are the same as S206 to S209 in the first embodiment.

  As described above, in this embodiment, in addition to the subject movement prediction from the image of the first camera 20a, the subject size prediction on the screen is used for subject capture control. As a result, it is possible to start tracking shooting of the subject with an optimum size from the time when it is captured by the second camera. FIG. 7 shows the effect when this embodiment is used. When the subject is recognized by the first camera subject recognition means 45a, capture control is performed so that the second camera 20b captures the subject.

  In the prior art shown in FIG. 7a, since the subject size information is not acquired, the subject size may not match the angle of view when the subject is captured by the second camera 20b. When the subject is too small or too large on the screen, the subject image does not match the template, and there is a possibility that the subject recognition by the second camera subject recognition unit 45b may be hindered. However, when the present embodiment is applied, as shown in FIG. 7b, the capture control is performed while performing zoom driving of the camera platform using the subject size prediction, so that it is optimal from the time when captured by the second camera 20b. You can shoot with a wide angle of view.

  The derived system of the present embodiment is the same as the derived system described in the first embodiment.

1 operation unit, 10 pan head, 20a / 20b camera, 30a / 30b lens,
40 Tracking processing device

Claims (2)

At least one first camera for monitoring a preset range;
A second camera that is mounted on at least one pan / tilt head device that drives at least one of zoom, pan, and tilt, and that can change a photographing optical axis or a photographing angle of view;
A tracking processing device for recognizing a subject to be tracked from image signals output from the first and second cameras and operating the second camera to perform control calculation for tracking the subject;
In an automatic tracking device composed of
The tracking processing device includes:
First camera subject recognition means for recognizing a subject to be tracked based on an image output from the first camera;
Subject speed calculation means for calculating the speed of the subject recognized by the first camera subject recognition means;
Subject position prediction means for predicting the position where the subject moves from the speed calculated by the subject speed calculation means;
Subject capturing means for controlling the camera platform device to direct the photographing optical axis of the second camera toward the subject based on the subject position predicted by the subject position predicting means;
Second camera subject recognition means for recognizing a subject to be tracked based on an image output from the second camera;
Second camera tracking control means for driving and controlling the camera platform so that the subject recognized by the second camera subject recognition means is captured at the center of the shooting screen of the second camera;
After the second camera subject recognition means recognizes the subject, the second camera tracking control means controls the pan head device to perform tracking shooting of the subject instead of the subject capturing means. Tracking device. The tracking processing device includes:
Subject size change speed calculating means for calculating a change speed of the size of the subject recognized by the first camera subject recognition means;
Subject size prediction means for predicting the size of the subject from the speed calculated by the subject size change speed calculation means;
The subject capturing means includes
2. The automatic tracking device according to claim 1, wherein the angle of view of the second camera is controlled so as to match the predicted size of the subject predicted by the subject size predicting unit.