JP2016076791A - Object tracking device, object tracking method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a possibility for imaging means to fail in automatic tracking of a tracking target.SOLUTION: Information indicating a relation between a tracking continuity value indicating a possibility (difficulty) of automatic tracking of a tracking target 102 and a target part is stored. When a tracking continuity value is acquired, a moving amount of pan/tilt is calculated in such a manner that a target part relating to the tracking continuity value is displayed on the center of a screen, a universal head of a PTZ camera 201 is moved just by the calculated moving amount, and an imaging direction of an imaging part 400 is changed. At such a time, a position where a failure of automatic tracking of the tracking target 102 is more unlikely to occur as the tracking continuity value indicates a lower possibility of automatic tracking of the tracking target 102, is determined as a target part. For example, in the case where the imaging part 400 is installed vertically downwards at a position higher than the tracking target 102, as the tracking continuity value indicates a lower possibility of automatic tracking of the tracking target 102, a lower position within the tracking target 102 is determined as the target part.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an object tracking device, an object tracking method, and a program, and is particularly suitable for use by an imaging unit to automatically track an object.

  In the monitoring and monitoring system, an automatic tracking technique using a camera called a PTZ camera that can automatically adjust pan, tilt, and zoom is known. In such automatic tracking technology, pan / tilt / zoom of the PTZ camera is automatically controlled so that the tracking target is continuously imaged. Here, panning means that the direction of the camera is shaken from side to side. Tilt means to shake the camera up and down. Zooming means changing the angle of view to a telephoto or wide angle.

  Patent Document 1 discloses a technique for detecting a moving body using a difference between frames when the PTZ camera is stationary, and tracking the detected moving body with the PTZ camera. In Patent Document 1, the moving direction and speed of the moving body are estimated from the difference image, and a pan / tilt value necessary for capturing the moving body near the center of the imaging screen is calculated.

  In Patent Document 2, in order to capture a face image in an optimal state with a camera at the entrance of the store, the face image is extracted and tracked, and a camera direction control unit, zoom control unit, and exposure control unit are provided. Techniques for controlling are disclosed.

  Patent Document 3 discloses a method for calculating control parameters of a PTZ camera necessary for displaying the tracking target at the center of the screen based on the coordinates of the tracking target displayed on the screen.

JP 2011-77617 A JP 2009-86932 A JP-A-9-181961

  However, in the techniques described in Patent Documents 1 to 3, depending on the position and movement of the tracking target, the imaging unit may fail to automatically track the tracking target. For example, when the imaging direction of the imaging means is moved so that the center of gravity of the person area as the tracking target is always at the center of the screen, there is a risk that automatic tracking will fail.

  The present invention has been made in view of such problems, and an object of the present invention is to reduce the possibility that the imaging means will fail to automatically track the tracking target.

  The object tracking device according to the present invention includes an extraction unit that extracts a moving object that is automatically tracked by the imaging unit from an image captured by the imaging unit that can automatically change the imaging direction, and the extraction unit extracts the moving object. Acquisition means for acquiring an index for evaluating the degree of difficulty of continuation of automatic tracking for the moving object, and determination means for determining a target portion in the image based on the index acquired by the acquisition means; And an instruction means for instructing the change of the imaging direction by the imaging means based on the target portion determined by the determination means, and the determination means has a relative degree of difficulty evaluated by the index. The position where the failure of automatic tracking of the moving object is less likely to occur when the target is higher than the case where the degree of difficulty evaluated by the index is relatively low is defined as the target unit. Characterized by being adapted to determine Te.

  According to the present invention, it is possible to reduce the possibility that the imaging unit will fail to automatically track the tracking target.

It is a figure explaining the setting method of a target part. It is a figure explaining the situation where it becomes easy to fail in automatic tracking. It is a figure which shows the structure of a tracking system. It is a figure which shows the structure of a PTZ camera. It is a figure which shows the relationship between a tracking continuity value and the position of a target part. It is a figure which shows an example of the position of a target part. It is a flowchart explaining an automatic tracking process.

Before describing the embodiment of the present invention, the premise of the embodiment of the present invention will be described.
In the automatic tracking technology, it is common to keep track of a tracking target by repeating the search for the tracking target in the screen and the calculation of the control parameters of the camera so that the searched tracking target appears in the screen.

  At this time, the tracking target is often approximated by a simple figure such as a rectangle or an ellipse. Also, a certain part in the figure that approximates the tracking target is selected as a target part, and the camera control parameters are calculated so that the target part is located at the center of the screen. Which part is preferably selected as the target part depends on the purpose of automatic tracking. Normally, the target portion is often selected from the upper half area of the approximate figure so that the tracking target is imaged at the center of the screen or slightly below the center of the screen.

FIG. 1 is a diagram illustrating an example of a method for setting a target portion.
FIG. 1 shows a state in which a tracking target 102 is projected in an image 101 captured by a PTZ camera. An arrow line indicating the movement direction 106 indicates that the tracking target 102 is moving in the direction of the arrow line. The shape of the tracking target 102 is approximated to the approximate figure 103. FIG. 1 shows an example in which the approximate figure 103 is a rectangle. The center point 104 represents the center point of the approximate figure 103. The upper 1/3 point 105 has a horizontal position that is the center position of the approximate figure 103 in the horizontal direction, and a vertical position that is the height of the approximate figure 103 from the upper end of the approximate figure 103 downward. This is a point at a position separated by 1/3 of the distance. Note that the approximate figure 103, the center point 104, and the upper 1/3 point 105 are not normally displayed in the image 101.

The PTZ camera can project the whole body of the tracking target 102 in the center of the screen by always setting the center point 104 as a target portion. In addition, the PTZ camera can project the upper half of the tracking target 102 in the center of the screen by always setting the upper 1/3 point 105 as the target portion.
In the example illustrated in FIG. 1, a case where the target portion is a point is described as an example. However, a region larger than a point can be used as the target portion. For example, the entire face area can be selected as the target portion.

  If some video processing is performed at a later stage using an image captured by the PTZ camera, it is conceivable to select a portion that functions most effectively as the target portion. For example, when there is a face authentication function or a glasses discrimination function as the video processing function in the subsequent stage, the tracking target face area can be selected as the target portion. Further, when there is a clothing discrimination function as a subsequent video processing function, the body area of the tracking target can be selected as the target portion.

However, by always selecting the same area as the target part, automatic tracking tends to fail in a specific situation.
FIG. 2 is a diagram illustrating an example of a situation in which automatic tracking is likely to fail. FIG. 2 shows an example in which the PTZ camera 201 (the direction of the zenith) is installed vertically downward at a position higher than the tracking target 102. The tracking target 102 is assumed to move from a point 203 to a point 205 via a point 204 on a straight line 202 several meters away from the PTZ camera 201. At this time, when the upper position in the approximate figure 103, for example, the center point 104 or the upper 1/3 point 105 is selected as the target portion, the automatic tracking is likely to fail.

The reason will be described below.
When the tracking target 102 moves to a position close to the PTZ camera 201, the pan angle / tilt angle of the PTZ camera 201 changes greatly instantaneously. In the example shown in FIG. 2, when the tracking target 102 moves near the point 203 or the point 205, the angular velocity of the pan angle / tilt angle is small, and the tilt angle becomes an angle close to the horizontal direction. On the other hand, when the tracking target 102 moves near the point 204, the angular velocity of both the pan angle and the tilt angle increases, and the tilt angle sinks deeply toward the zenith.

At this time, when selecting the upper position in the approximate figure 103 as the target portion and the case of selecting the lower position, the former case is more preferable than the latter case. The distance between the target unit and the PTZ camera 201 in the real world (real space) is shortened. Therefore, in the former case, the angular velocity of the pan angle and the tilt angle of the PTZ camera 201 necessary for continuing the automatic tracking is larger than in the latter case. Therefore, when the upper position in the approximate figure 103 is selected as the target portion, the automatic tracking is more likely to fail than when the lower position is selected.
The present inventors pay attention to the fact that the likelihood of failing in automatic tracking differs according to the distance between the target unit and the PTZ camera 201 in the real world (real space), and will be described below. It came to the embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 3 is a diagram illustrating an example of the configuration of the tracking system according to the present embodiment.
In FIG. 3, the tracking system includes a PTZ camera 201, a display device 301, and a recording device 302. The PTZ camera 201, the display device 301, and the recording device 302 are connected via a network 303 so that they can communicate with each other. The PTZ camera 201 transmits an image 304 captured by the PTZ camera 201 in response to requests from the display device 301 and the recording device 302. The recording device 302 stores the image 304 acquired from the PTZ camera 201 in a recording medium. The display device 301 reads and displays the image 304 acquired from the PTZ camera 201 and the image 304 stored in the recording device 302, receives a request from the user through an input device (not shown), and transmits the request to the PTZ camera 201. To do.

FIG. 4 is a diagram illustrating an example of the configuration of the PTZ camera 201.
The PTZ camera 201 includes an imaging unit 400 and a control unit 450.
The imaging unit 400 includes a solid-state imaging device such as a CMOS sensor or a CCD sensor, and a camera platform. The imaging unit 400 can capture the image 304 by changing at least one of pan, tilt, and zoom.

The communication I / F unit 451 transmits and receives information based on a communication method such as HTTP or TCP / IP.
The video input unit 452 receives the imaging data D from the imaging unit 400. Here, the imaging data D is data including an image 304 captured by the imaging unit 400 and posture information (pan value, tilt value, zoom value) of the imaging unit 400 when the image 304 is captured.

  The video encoding unit 453 receives the imaging data D from the video input unit 452 and encodes the image 304 by an appropriate encoding method. Here, as encoding methods, JPEG, MPEG-2, MPEG-4, and H.264 are available. There are methods such as H.264. The encoded image 304 is transmitted from the video transmission unit 454 to the outside via the communication I / F unit 451.

The overall control unit 455 controls the modules that constitute the PTZ camera 201.
The imaging data D acquired by the video input unit 452 is also transmitted to the tracking target determination unit 456. The tracking target determination unit 456 receives an automatic tracking start request and an automatic tracking end request from the display device 301 or the like via the communication I / F unit 451 and the overall control unit 455. The tracking target determination unit 456 determines and outputs position information of the tracking target 102 from the time when the automatic tracking start request is received until the automatic tracking end request is received.

  The tracking target determination unit 456 determines the tracking target 102 after receiving the automatic tracking start request. As the tracking target 102, either a moving object automatically detected from the image 304 by moving image processing or a moving object explicitly selected by the user is generally used. When the moving object is detected as the tracking target 102 as in the former case, the detection result of the moving object region by the background subtraction method or the motion vector detection, or the specific region of the moving object such as the human body or the face using machine learning is used. The detection result of the detector can be used. However, the method of selecting the tracking target 102 is not limited to these, and any method may be used. The method described in Patent Document 1 is an example.

  After determining the tracking target 102, the tracking target determining unit 456 determines and outputs the position of the tracking target 102 in the screen as long as the tracking target 102 continues to be displayed in the screen. As a method for determining the position of the tracking target 102 within the screen, for example, there are the following methods. First, the tracking target determination unit 456 extracts a template and an image feature amount from the imaging data D when the tracking target 102 is determined. Then, the tracking target determination unit 456 defines an appropriate index for the template and the image feature amount, so that the position with the highest probability that the tracking target 102 exists is determined from the current imaging data D as the tracking target. It searches for the position in the screen 102 and outputs it. As a result of this search, when it is determined that the tracking target 102 does not exist in the screen, the tracking target determination unit 456 outputs that fact.

A search method using a template will be described as a more specific example of a method for searching the position of the tracking target 102 on the screen.
First, the tracking target determination unit 456 extracts a template based on the luminance of the image 304 from the approximate figure 103 including the tracking target 102 when the tracking target 102 is determined. When the next image 304 comes in, the tracking target determination unit 456 moves the template little by little in the sliding window method on the screen, and searches for a position where the SSD (Sum of Squared Difference) is minimized. The tracking target determination unit 456 determines that the tracking target 102 exists at that position. Here, the SSD is an index that quantifies the degree of difference between images. When there is no location where the SSD is below a certain threshold, the tracking target determination unit 456 determines that the tracking target 102 does not exist in the screen.

  The method for determining the position of the tracking target 102 within the screen is not limited to the method described above. For example, the position can be determined by estimating the direction and amount of motion using the motion vectors of feature points, extracting multiple moving objects using inter-frame differences, and selecting one of the multiple conductors. Any method may be used.

  The tracking target determination unit 456, for example, approximates the tracking target 102 with a simple approximate figure 103 such as a rectangle or an ellipse, and then expresses the position information of the tracking target 102 based on the coordinate information on the screen of the approximate figure 103. . However, the method for expressing the position information of the tracking target 102 is not limited to this method. For example, any expression method for specifying the position of the tracking target 102 on the screen, such as the outline of the tracking target 102 or a set of pixels or blocks constituting the tracking target 102, may be used.

  Further, as a method for determining the position of the tracking target 102, a method for estimating the position of a future tracking target may be used. For example, the moving direction and moving speed of the tracking target 102 may be estimated, the position of the tracking target 102 on the screen after a predetermined time may be predicted, and the position information of the tracking target 102 may be created based on the position. .

  The tracking continuity estimation unit 457 receives the imaging data D and the position information of the tracking target 102 from the tracking target determination unit 456. Then, the tracking continuity estimation unit 457 estimates tracking continuity. Here, the tracking continuity is a high or low probability of succeeding in continuing the automatic tracking of the tracking target 102. In the following description, a situation where the difficulty level of the automatic tracking with respect to the tracking target 102 is relatively low and the automatic tracking is likely to continue is expressed as high tracking continuity as necessary. On the other hand, if the degree of difficulty of automatic tracking with respect to the tracking target 102 is relatively high and the automatic tracking fails and the tracking target 102 may not be kept on the screen, the tracking is performed as necessary. Expressed as low continuity.

In this embodiment, it is assumed that the higher the tracking continuity is, the larger the tracking continuity value indicating the possibility (difficulty) of continuing automatic tracking for the tracking target 102 is. The tracking continuity value is an index for evaluating the possibility (difficulty) of continuation of automatic tracking with respect to the tracking target 102. For example, the installation status of the PTZ camera 201, the position of the PTZ camera 201, and the position of the tracking target 102 It is estimated using an index determined based on the relationship between
For example, assuming that the zoom values of the PTZ camera 201 are the same, it is determined that the tracking continuity value is smaller (the tracking continuity is lower) as the tracking target 102 moves closer to the PTZ camera 201. it can.

  When determining tracking continuity in this way, the tracking continuity estimation unit 457 determines the distance (in real space) between the tracking target 102 and the PTZ camera 201, for example, the size of the tracking target 102 on the screen. Can be guessed. Further, the tracking continuity estimation unit 457 can measure the distance (in real space) between the tracking target 102 and the PTZ camera 201 using a device (not shown) such as a distance sensor or a stereo camera. As described above, as an index for evaluating the possibility (difficulty) of continuation of automatic tracking with respect to the tracking target 102, for example, the size of the tracking target 102 on the screen or the tracking target 102 and the PTZ camera 201 The distance between can be used.

  Further, when the PTZ camera 201 is installed vertically downward at a position higher than the tracking target 102, the tilt angle of the PTZ camera 201 sinks deeper toward the zenith as the tracking target 102 is closer to the PTZ camera 201. . Therefore, the tracking continuity estimation unit 457 can also approximate the distance (in real space) between the tracking target 102 and the PTZ camera 201 using the magnitude of the tilt angle. In this case, for example, it can be estimated that the tracking continuity value is smaller (the tracking continuity is lower) as the tilt angle of the PTZ camera 201 is closer to the direction farther from the zenith (horizontal direction). As described above, for example, the tilt angle of the PTZ camera 201 can be used as an index for evaluating the possibility (degree of difficulty) of continuing automatic tracking with respect to the tracking target 102.

  Also, the tracking continuity estimation unit 457 increases the tracking continuity value as the pan angle / tilt angle change rate (moving angle per unit time) increases in the most recent predetermined number of frames captured by the PTZ camera 201. Can be estimated to be small (the tracking continuity is low). As described above, as an index for evaluating the possibility (difficulty) of continuing the automatic tracking with respect to the tracking target 102, for example, the change speed of the pan angle / tilt angle (the angle that moves per unit time) can be used. .

The method for estimating the tracking continuity value is not limited to these methods. For example, any method may be used such as a method of determining that tracking continuity is reduced because the detection capability of a specific object detector such as a human body detector, a spectacles detector, or a clothes detector is reduced.
The tracking continuity value may be expressed as a discrete value such as “large”, “medium”, or “small”, or may be expressed as a continuous value.

Here, an example of the tracking continuity value when the tracking continuity value is expressed as a discrete value will be described.
First, appropriate threshold values a and b (a <b) are prepared.
In this case, the tracking continuity value is set to “small” when the estimated distance between the tracking target 102 and the PTZ camera 201 is “less than a”, “a or more and less than b”, and “b or more”. , “Medium” and “large”.
Further, it is assumed that the tilt angle φ of the PTZ camera 201 can take a continuous value of 0 to 90 °. In this case, when the tilt angle φ of the PTZ camera 201 is “less than a”, “when more than a and less than b”, and “when more than b”, the tracking continuity values are “large” and “medium”, respectively. , “Small”. However, here, the tilt angle φ is determined on the assumption that the PTZ camera 201 faces the horizontal direction when the tilt angle φ is 0 °, and the PTZ camera 201 faces the zenith direction when the tilt angle φ is 90 °. The case is shown as an example.

  Next, an example of the tracking continuity value when the tracking continuity value is expressed as a continuous value will be described. For example, the tracking continuity value can be expressed as 90−φ using the tilt angle φ. In this case, it is continuously expressed that the tracking continuity is higher as the tilt angle φ is closer to 0 °, and the tracking continuity is lower as the tilt angle φ is closer to 90 °.

  The target unit determination unit 458 receives the imaging data D, the position information of the tracking target 102, and the tracking continuity value from the tracking continuity estimation unit 457. And the target part determination part 458 determines a target part from a screen based on a tracking continuity value.

  First, it is assumed that the PTZ camera 201 is installed vertically downward at a position higher than the tracking target 102. In this case, as described with reference to FIG. 2, by selecting the lower position in the tracking target 102 displayed on the screen as the target portion, the real world (real space) between the PTZ camera 201 and the target portion. The distance at will increase. Therefore, the angular velocity of the pan angle / tilt angle required for the PTZ camera 201 to automatically track the tracking target 102 is reduced. For this reason, the PTZ camera 201 is less likely to fail to automatically track the tracking target 102.

FIG. 5 is a diagram for explaining an example of a method for determining a target part when the tracking continuity value is given as a discrete value. Specifically, FIG. 5 is a diagram showing an example of the relationship between the tracking continuity value and the position of the target portion in a table format. FIG. 6 is a diagram showing an example of the position of the target portion shown in FIG.
In the example shown in FIG. 5, when the tracking continuity value is “large”, the target part determining unit 458 determines the upper third point 105 of the approximate figure 103 of the tracking target 102 as the target part (FIG. 5). 6). When the tracking continuity value is “medium”, the target part determining unit 458 determines the center point 104 of the approximate figure 103 of the tracking target 102 as the target part (see FIG. 6). When the tracking continuity value is “medium”, the target part determining unit 458 determines the lower quarter point 601 of the approximate figure 103 of the tracking target 102 as the target part (see FIG. 6). The center point 104 and the upper 1/3 point 105 are points described with reference to FIG. The lower 1/4 point 601 has a horizontal position that is the center position of the approximate figure 103 in the horizontal direction, and a vertical position that extends from the lower end of the approximate figure 103 upward. It is a point at a position separated by a distance of 1/4 of the height of.

  As described above, when the face authentication function or the glasses discrimination function is executed as the subsequent video processing function, the face region (the position of the face in the tracking target 102 (approximate figure 103)) is determined as the target part. It is preferable. In addition, when executing the clothing discrimination function, it is desirable to determine the body region (particularly the outer jacket portion) as the target portion. As shown in FIG. 5, the higher the tracking continuity value (the higher the possibility that automatic tracking of the tracking target 102 can be continued), the higher the position within the tracking target 102 is set as the target portion. This is because it is possible to determine a target part suitable for the video processing function in the subsequent stage.

When the tracking continuity value is given as a continuous value, the target part determining unit 458 can determine the position of the target part depending on the range of the tracking continuity value. Further, the target part determination unit 458 can determine the position of the target part by substituting the tracking continuity value into a relational expression describing the relation between the tracking continuity value and the position of the target part. For example, the tracking continuity value is t (where t can be a continuous value from 0 to 90). In this case, the target part determination unit 458 separates the position of the target part of the tracking target 102 from the upper end of the approximate figure 103 of the tracking target 102 by L (t) percent of the height of the approximate figure 103. It can be determined as a position. Here, L (t) is expressed by the following equation (1).
L (t) = A−t × (A−B) / 90 (1)
In the formula (1), A and B are constants. For example, if A = 75 and B = 33, if the tracking continuity value is 0, the lower 1/4 point 601 is set, and if the tracking continuity value is 90, the upper 1/3 point 105 is set, respectively. It corresponds to determining as a target part. As the tracking continuity value changes continuously, the position of the target portion also changes continuously.

  As described above, when the PTZ camera 201 is installed vertically downward at a position higher than the tracking target 102, the lower the tracking continuity value, the lower the position on the screen is determined as the target portion. . By doing so, the distance between the target unit and the PTZ camera 201 in the real world (real space) is increased, and the pan angle / tilt angle necessary for the PTZ camera 201 to automatically track the tracking target 102. The angular velocity of can be increased. Therefore, the automatic tracking of the tracking target 102 can be made difficult to fail.

  The method for determining the target portion differs depending on the installation position of the PTZ camera 201. For example, it is assumed that the PTZ camera 201 is installed at a position close to the ground and is installed so as to look up at the tracking target 102. In this case, as the tracking continuity value is smaller, the upper position on the screen is determined as the target portion, so that the PTZ camera 201 can be made less likely to fail in automatic tracking.

  The control command generation unit 459 receives the imaging data D and the target unit from the target unit determination unit 458. Then, the control command generation unit 459 calculates the amount of pan / tilt movement necessary to display the target unit at the center of the screen. Then, the control command generation unit 459 generates a control command C for the camera platform and transmits it to the imaging unit 400. The control command C includes the calculated pan / tilt movement amount. Note that any known method can be used as the calculation method. As an example of the calculation method, the method described in Patent Document 3 can be used.

  The control command generation unit 459 may further include a necessary zoom amount in the control command C so that the size of the tracking target 102 on the screen is constant. Furthermore, the control command generation unit 459 may include the pan, tilt, and zoom change rates in the control command. The changing speed of pan, tilt, and zoom is determined based on, for example, the moving speed of the tracking target 102 within the screen.

In the present embodiment, the hardware configuration for realizing the configuration shown in FIG. 4 is not limited to a specific hardware configuration. For example, the imaging unit 400 and the control unit 450 may be separated.
Each unit in the control unit 450 may be implemented by hardware, or implemented by software realized by the control unit 450 executing a program stored in a memory or the like in the control unit 450. Also good.

FIG. 7 is a flowchart for explaining an example of automatic tracking processing (processing for automatically performing object tracking) by the tracking system. Immediately after the start of the automatic tracking process, the overall control unit 455 keeps the mode in the automatic tracking process in the standby mode.
In step S701, the overall control unit 455 waits until an automatic tracking start request is given from the outside through the communication I / F unit 451. When an automatic tracking start request is given, the process advances to step S702, and the overall control unit 455 changes the mode in the automatic tracking process from the standby mode to the automatic tracking mode.

  Next, in step S703, the overall control unit 455 makes an inquiry to the tracking target determination unit 456 and requests the tracking target 102 to be determined. When information from the outside is required, such as when an object explicitly selected by the user using the input device is used as the first tracking target 102, the overall control unit 455 also displays the information as a tracking target. Tell the decision unit 456. The tracking target determination unit 456 determines the tracking target 102 based on this inquiry.

Next, in step S704, the overall control unit 455 makes an inquiry to the tracking target determination unit 456 and requests determination of the position of the tracking target 102 within the screen. The tracking target determination unit 456 determines the position of the tracking target 102 in the screen based on this inquiry.
Next, in step S705, the tracking target determination unit 456 determines whether the tracking target 102 has not been found in the screen. As a result of this determination, if the tracking target 102 is found in the screen, the process proceeds to step S706. On the other hand, if the tracking target 102 is not found in the screen, the process proceeds to step S711 described later.

In step S706, the tracking continuity estimation unit 457 derives a tracking continuity value.
Next, in step S707, the target part determination unit 458 determines a target part.
Next, in step S708, the control command generation unit 459 generates a control command C for the camera platform.

Next, in step S709, the imaging unit 400 operates the camera platform based on the control command C, and changes the posture (imaging direction) of the PTZ camera 201.
Next, in step S710, the overall control unit 455 determines whether or not an automatic tracking end request is given from the outside through the communication I / F unit 451. If the result of this determination is that an automatic tracking end request has not been given, processing returns to step S704.

On the other hand, if an automatic tracking end request is given, the process proceeds to step 711.
In step S711, the overall control unit 455 ends the automatic tracking mode, and changes the mode in the automatic tracking process from the automatic tracking mode to the standby mode.

<Second Embodiment>
Next, a second embodiment will be described. In the first embodiment described above, the case where the information regarding the installation conditions of the imaging unit 400 is preset in the control unit 450 and is not changed has been described as an example. In contrast, in the present embodiment, a case where the user teaches the PTZ camera 201 through the display device 301 or the like to teach the PTZ camera 201 of installation information that is information regarding the installation conditions of the imaging unit 400 will be described as an example. As described above, the present embodiment is added to the first embodiment in the configuration and processing for determining the installation condition of the imaging unit 400 based on an instruction from the user and determining the target unit based on the determined installation condition. It is a thing. Therefore, in the description of the present embodiment, detailed description of the same parts as those of the first embodiment is omitted.

  The installation information indicating the installation conditions of the imaging unit 400 is taken into the control unit 450 as follows, for example. First, the display device 301 uses a GUI (such as a radio button) to select whether the installation condition of the imaging unit 400 is “install vertically from the ceiling”, “install on the wall”, or “install on the feet”. Graphic user interface). When the user selects an installation condition of the imaging unit 400 using this GUI, the display device 301 transmits installation information indicating the installation condition of the selected imaging unit 400 to the control unit 450 via the network 303.

  The target unit determination unit 458 receives the installation information via the communication I / F unit 451 and the overall control unit 455. The target part determination unit 458 determines an appropriate target part according to the received installation information. For example, based on the installation information that the installation position of the imaging unit 400 is installed vertically downward from the ceiling, the area below the approximate figure 103 is selected as the target unit, so that the automatic tracking of the tracking target 102 is successfully performed. Judge that the probability increases. Then, the target part determining unit 458 appropriately determines the target part with reference to the tracking continuity value.

As described above, in the above-described embodiment, information indicating the relationship between the tracking continuity value indicating the possibility (difficulty) of automatic tracking of the tracking target 102 and the target portion is stored. When the tracking continuity value is acquired, the pan / tilt movement amount is calculated so that the target portion related to the tracking continuity value is displayed at the center of the screen, and the PTZ camera 201 is calculated by the calculated movement amount. The imaging direction of the imaging unit 400 is changed by moving the pan head. At this time, as the tracking continuity value is a value indicating that the possibility of automatic tracking of the tracking target 102 is lower, a position where the automatic tracking failure of the tracking target 102 is less likely to occur is determined as a target portion. For example, when the imaging unit 400 is installed vertically downward at a position higher than the tracking target 102, the tracking continuity value is a value indicating that the possibility of automatic tracking of the tracking target 102 is low. The lower position in the target 102 is determined as the target portion.
Therefore, even if the probability of failure of automatic tracking of the tracking target 102 is high, the possibility that the automatic tracking of the tracking target 102 will fail can be reduced as compared with the case where the position of the target portion in the screen is fixed. it can.

  In the above-described embodiment, the position suitable for the video processing function at the subsequent stage is set as the target portion as the tracking continuity value is a value indicating that the tracking target 102 is less likely to be automatically tracked. For example, when the imaging unit 400 is installed vertically downward at a position higher than the tracking target 102, the tracking continuity value is a value indicating that the tracking target 102 is highly likely to be automatically tracked. The upper position in the tracking target 102 is determined as a target portion. Therefore, in a situation where the probability of the automatic tracking failure of the tracking target 102 is low, the tracking target 102 can be reflected in a position suitable for the subsequent processing.

  In the embodiment described above, an index for evaluating the degree of difficulty in continuing automatic tracking with respect to the tracking target 102 is acquired (detected), a tracking continuity value is derived from the acquired (detected) information, and the tracking continuity value is determined. The case where the target part corresponding to is determined has been described as an example. As described above, for example, the index for evaluating the difficulty of continuing the automatic tracking with respect to the tracking target 102 is the size of the tracking target 102 on the screen, the distance between the tracking target 102 and the PTZ camera 201, and PTZ. For example, the tilt angle of the camera 201.

  However, it is not always necessary to derive (explicitly) a tracking continuity value. That is, when an index for evaluating the degree of difficulty in continuing the automatic tracking with respect to the tracking target 102 is acquired (detected), the position of the target portion corresponding to the acquired (detected) information can be determined (without using the tracking continuity value). ) You may decide.

Note that, as described with reference to FIG. 5, when the value indicated by the acquired (detected) information is within a predetermined range, the same target portion can be determined. Therefore, when the value indicated by the acquired (detected) information changes, the position of the target portion does not necessarily have to be changed. That is, when the acquired (detected) information indicates that the degree of difficulty in continuing the automatic tracking with respect to the tracking target 102 is relatively low, the automatic tracking of the tracking target 102 fails. It suffices if the position where the occurrence of the occurrence of the problem is difficult can be determined as the target portion. More specifically, the position of the target portion is made the same in the former case and the latter case, or the automatic tracking of the tracking target 102 is more failed in the latter case than in the former case. A position where it is difficult to occur is determined as a target portion. However, in the latter case, a position where the automatic tracking failure of the tracking target 102 is less likely to occur is determined as a target portion than in the former case.
On the other hand, when the value indicated by the acquired (detected) information changes, the position of the target portion may be changed.

<Other examples>
The present invention can take an embodiment as a system, apparatus, method, program, recording medium (storage medium), or the like. Specifically, the present invention may be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or may be applied to a device composed of a single device. good.

  The present invention can also be realized by executing the following processing. That is, first, software (computer program) for realizing the functions of the above embodiments is supplied to a system or apparatus via a network or various storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the computer program.

  In addition, the above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

  201: PTZ camera, 400: imaging unit, 450: control unit

Claims (15)

An extraction means for extracting a moving object automatically tracked by the imaging means from an image taken by the imaging means capable of automatically changing the imaging direction;
Obtaining means for obtaining an index for evaluating a difficulty level of continuation of automatic tracking for the moving object extracted by the extracting means;
Determining means for determining a target portion in the image based on the index acquired by the acquiring means;
Instruction means for instructing change of the imaging direction by the imaging means based on the target portion determined by the determining means;
The determination unit is more likely to fail in automatic tracking of the moving object when the difficulty level evaluated by the index is relatively higher than when the difficulty level evaluated by the index is relatively low. An object tracking device characterized in that a position where it is difficult to occur can be determined as the target portion.   The object tracking device according to claim 1, wherein the index is an index determined based on a relationship between a position of the imaging unit in real space and a position of the moving object in real space. The index is a distance between the imaging means and the moving object,
The determination unit is configured such that when the distance between the imaging unit and the moving object is relatively small, compared to when the distance between the imaging unit and the moving object is relatively large, The object tracking device according to claim 1, wherein a position where the failure of the automatic tracking of the moving object is less likely to occur can be determined as the target unit. The indicator is the size of the moving object in the image,
The determination means performs automatic tracking of the moving object when the size of the moving object in the image is relatively larger than when the size of the moving object in the image is relatively small. The object tracking device according to claim 1, wherein a position where the failure is less likely to occur is determined as the target unit. The index is a tilt angle of the imaging means,
The determination means moves the movement when the tilt angle of the imaging means is relatively closer to the zenith than when the tilt angle of the imaging means is relatively far from the zenith. The object tracking device according to claim 1, wherein a position at which failure of automatic tracking of an object hardly occurs can be determined as the target unit. The index is an angle at which the imaging means moves per unit time to change the imaging direction,
In the case where the angle at which the imaging unit moves in unit time to change the imaging direction is a relatively large angle, the determination unit is configured to change the imaging direction in unit time. The position where the failure of automatic tracking of the moving object is less likely to occur than when the moving angle is a relatively small angle can be determined as the target portion. 2. The object tracking device according to 2.   The position where the failure of the automatic tracking of the moving object is less likely to occur is a position where a distance in the real space between the imaging unit and the target unit becomes large. The object tracking device according to the item. The imaging means is installed vertically downward,
The determining means determines the lower position in the image when the difficulty level evaluated by the index is relatively higher than when the difficulty level evaluated by the index is relatively low. The object tracking device according to claim 1, wherein the object tracking device can be determined as a target portion.   The determination means determines, as the target unit, a position at which the failure of automatic tracking of the moving object is less likely to occur as the degree of difficulty evaluated by the index is higher. The object tracking device according to claim 1.   In the determination means, the target when the degree of difficulty evaluated by the index is relatively lower than the case where the degree of difficulty evaluated by the index is relatively high is the target. The object tracking device according to any one of claims 1 to 9, wherein a position suitable for processing performed on the image after a part is determined can be determined as the target part. . Based on the index, further comprising a derivation means for deriving a value indicating a difficulty level of continuation of automatic tracking for the moving object;
The determining means determines the position of the target portion based on the relationship between the value and the position of the target portion and the value derived by the derivation means,
The relationship is a relationship in which, as the value is a value indicating that the degree of difficulty is high, the position of the target unit becomes a position where the failure of automatic tracking of the moving object is less likely to occur. The object tracking device according to any one of claims 1 to 10.   The said instruction | indication means instruct | indicates the change of the said imaging direction by the said imaging means so that the said target part may become the center position of the said image, The any one of Claims 1-11 characterized by the above-mentioned. Object tracking device.   The object according to any one of claims 1 to 12, wherein the determination unit changes a position at which failure of automatic tracking of the moving object is less likely to occur according to an installation condition of the imaging unit. Tracking device. An extraction step of extracting a moving object to be automatically tracked by the imaging unit from an image captured by the imaging unit capable of automatically changing the imaging direction;
An acquisition step of acquiring an index for evaluating the difficulty level of continuation of automatic tracking for the moving object extracted by the extraction step;
A determination step of determining a target portion in the image based on the index acquired by the acquisition step;
An instruction step for instructing the change of the imaging direction by the imaging means based on the target portion determined by the determination step;
In the determining step, when the degree of difficulty evaluated by the index is relatively high, a failure in automatic tracking of the moving object occurs than when the degree of difficulty evaluated by the index is relatively low. An object tracking method characterized in that a position to be difficult can be determined as the target portion.   A program that causes a computer to function as each unit of the object tracking device according to any one of claims 1 to 13.

Images