JP2015194915A - Moving body tracking apparatus, moving body tracking method, moving body tracking system, and camera - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving body tracking apparatus which can precisely track a moving body moving against an overall flow of moving bodies.SOLUTION: A moving body tracking apparatus includes a movement vector acquisition unit 23 which acquires a movement vector indicating the movement of a moving body between images constituting a dynamic image, an overall direction acquisition unit 24 which acquires an overall direction indicating an overall movement direction of the moving bodies included in the dynamic image, and a counter movement tracking unit 25 which tracks one of the moving bodies that moves counter in relation to the overall direction on the basis of the moving vector having a direction different from the overall direction by a predetermined angle or more.

Description

  The present invention relates to a moving body tracking apparatus that tracks a moving body that goes backward in the general flow of the moving body from an image captured by a monitoring camera or the like.

  2. Description of the Related Art Conventionally, a method for analyzing the movement of a moving body by processing an image captured by a camera is known. For example, by calculating the x coordinate of a moving object from a video composed of a plurality of images that are continuous in time series, and tracking the moving object by creating an x-t map, it is against the specified direction. A method for detecting a retrograde person walking is known (see, for example, Non-Patent Document 1).

Hiroyuki Arai, Takayuki Anno, Midori Mizukami, Miki Haseyama, “Method for Detecting Retrograde Person from Video”, The Institute of Electronics, Information and Communication Engineers, IEICE Technical Report ITS2005-84, IE2005-291 (2006-02), pp. 29-34

  However, in the method described in Non-Patent Document 1, tracking of the moving object is performed ignoring the y coordinate of the moving object. For this reason, the moving direction of the moving body regarded as retrograde is limited. For example, even if there is a mobile body in which the global flow of the mobile body is in the positive y-axis direction and moves in the negative y-axis direction, it is not determined that the mobile body is going backwards.

  Further, in the method described in Non-Patent Document 1, tracking of a moving object is performed by detecting a straight line on an xt map using Hough transform. On the x-t map, a moving body that moves at a constant speed appears as a straight line, whereas a moving body that moves at an accelerated speed appears as a curve. For this reason, in the method described in Non-Patent Document 1, it is possible to detect a moving body that moves backward while moving at a constant speed, but it is not possible to detect a moving body that moves backward while performing an acceleration movement.

  The present invention has been made in view of such circumstances, and accurately tracks a moving body that goes against the general flow of the moving body without limiting the moving direction, speed, or acceleration of the moving body. An object of the present invention is to provide a movable body tracking device and the like.

  In order to achieve the above object, a moving body tracking device according to an aspect of the present invention includes a motion vector acquisition unit that acquires a motion vector indicating a motion of a moving body between images constituting a moving image, and the moving image. A global direction acquisition unit that acquires a global direction indicating a global movement direction of the mobile body included, and a motion vector having a direction different from the global direction by a predetermined angle or more among the motion vectors acquired by the motion vector acquisition unit And a retrograde tracking unit that traces a mobile body that runs backward in the general direction.

  Note that the present invention can be realized not only as a mobile tracking device including such a characteristic processing unit, but also as a step that includes a process executed by a characteristic processing unit included in the mobile tracking device. It can be realized as a body tracking method. Further, the present invention can be realized as a program for causing a computer to function as a characteristic processing unit included in the mobile body tracking device or a program for causing a computer to execute characteristic steps included in the mobile body tracking method. Such a program can be distributed via a computer-readable non-transitory recording medium such as a CD-ROM (Compact Disc-Read Only Memory) or a communication network such as the Internet. . Further, the present invention can be realized as a semiconductor integrated circuit that realizes part or all of the mobile tracking device, or as a mobile tracking system including mobile tracking.

  According to the present invention, it is possible to accurately track a moving body that goes against the general flow of the moving body without limiting the moving direction, speed, or acceleration of the moving body.

It is a figure which shows the structure of the mobile body tracking system containing the mobile body tracking apparatus which concerns on embodiment of this invention. It is a block diagram which shows the hardware constitutions of a mobile body tracking system. It is a figure which shows an example of the image which comprises the moving image imaged by the imaging part. It is the figure which showed the motion vector which shows the motion of the person between two continuous images superimposed on the image. It is a figure for demonstrating how to obtain | require the motion vector which has direction different from a global direction more than a predetermined angle. It is a figure which shows a cluster with an ellipse cluster frame. It is a figure which shows the average vector of the motion vector contained in a cluster frame. It is a figure which shows the average vector of the motion vector contained in a cluster frame after removing noise. It is a figure which shows the tracking result of the person who goes back to the general direction in 10 continuous images. It is a figure which shows the result of having tracked the average vector calculated from the motion vector which generate | occur | produced by the person's leg swing. It is a figure which shows an example of the image which shows that the person is retrograde. It is a figure which shows another example of the image which shows that the person is retrograde. It is a figure which shows an example of the enlarged image of the person who goes retrograde. It is a flowchart of the retrograde determination process which a moving body tracking device performs.

  First, embodiments of the present invention will be listed and described.

  (1) A moving body tracking apparatus according to an aspect of the present invention includes a motion vector acquisition unit that acquires a motion vector indicating a motion of a moving body between images constituting the moving image, and the moving body included in the moving image. Based on a motion vector having a direction different from the global direction by a predetermined angle or more among the motion vectors acquired by the motion vector acquisition unit. And a retrograde tracking unit that tracks a mobile object that runs in the general direction.

  According to this configuration, a mobile object traveling in the general direction is tracked based on a motion vector having a direction different from the general direction by a predetermined angle or more without limiting the general direction. In addition, it is possible to track a moving body going backward in the general direction without limiting the speed of the moving body. For this reason, it is possible to accurately track the moving body that goes against the general flow of the moving body without limiting the moving direction, speed, or acceleration of the moving body.

  (2) Further, the retrograde tracking unit determines a motion vector having a direction different from the general direction by a predetermined angle or more among the motion vectors acquired by the motion vector acquisition unit based on a distribution of the motion vector in the image. And determining that the moving body corresponding to the cluster has moved in the direction of the vector representing the motion vector included in the cluster for each cluster classified by the clustering unit and the cluster classified by the clustering unit, And a tracking unit that tracks a mobile body that moves backward in the general direction.

  According to this configuration, the clustering unit can cluster the motion vectors in the direction reverse to the general direction so that the motion vectors whose positions are close to each other are classified into the same cluster. For this reason, by tracking the moving object for each cluster, it is possible to track the moving object that is moving backward in units of moving objects or in units of groups of moving objects.

  (3) In addition, the tracking unit extracts, for each cluster classified by the clustering unit, a motion vector whose direction difference from the direction of the average vector of motion vectors included in the cluster is within a predetermined angle, and extracts the extracted motion By determining that the mobile object corresponding to the cluster has moved in the direction of the average vector of the vectors, the mobile object going backward in the global direction may be tracked.

  According to this configuration, motion vectors whose direction difference from the average vector is within a predetermined angle in the cluster are extracted. That is, a motion vector having a direction difference larger than a predetermined angle is removed as noise. For this reason, it is possible to track the moving body in the direction of the average vector of the extracted motion vectors after removing the noise. Therefore, it is possible to accurately track a moving object that is moving backward.

  (4) Further, the above-described moving body tracking device further includes a mobile station that can track the moving body when the tracking unit can track the moving body over a predetermined number of continuous images constituting the moving image. You may provide the retrograde determination part which determines that it is retrograde in the direction.

  According to this configuration, when a moving body cannot be tracked continuously for a long time, it is not determined that the moving body is going backward. In other words, even if a part of the body moves backwards due to a hand or leg swing of a person moving in the general direction, these movements are not continuous, so this person goes backwards. It is not determined to be. Therefore, it can be accurately determined whether or not the moving body is moving backward in the general direction.

  (5) Further, the retrograde determination unit is configured such that the tracking unit can track the moving body over the predetermined number of continuous images constituting the moving image, and the size of the moving body is a first threshold value. When it is above, you may determine with the said mobile body going backward in the said big direction.

  According to this configuration, when the size of the moving object is too small, it is determined that the moving object is not a moving object assumed by the moving object tracking device and can be excluded from the tracking target.

  (6) Further, the retrograde determination unit is configured such that the tracking unit can track the moving body over the predetermined number of continuous images constituting the moving image, and the size of the moving body is a second threshold value. When it is below, you may determine with the said mobile body going backward in the said big direction.

  According to this configuration, when the size of the moving body is too large, it is determined that the moving body is not the moving body assumed by the moving body tracking device, and can be excluded from the tracking target.

  (7) Moreover, the above-described moving body tracking apparatus further displays an image indicating that the moving body is moving backward when the backward moving determination unit determines that the moving body is moving backward in the general direction. Or you may provide the presentation part shown to a user with an audio | voice.

  According to this configuration, the user can know that the moving object is moving backward by looking at the image or listening to the sound. Therefore, the user can pay attention to the moving body that is moving backward so as not to go backward.

  (8) In addition, the presenting unit displays (i) a message indicating that the moving body is moving backward, and (ii) shows the moving body moving backward in a manner different from that of the moving body not moving backward. (Iii) display an image composed of a retrograde moving object, (iv) display an image in which a region including the retrograde moving object is enlarged, or (v) a mobile object A warning sound may be output indicating that is reverse.

  According to this configuration, the user can be easily informed that the moving body is going backward. For this reason, the user can promptly pay attention to the moving body that is going backward so as not to go backward.

  (9) The retrograde tracking unit may not track the moving body when the general direction is not within a predetermined angle range.

  For example, when a sidewalk whose direction of traffic is switched depending on the time zone is imaged, there are cases where people heading in various directions are mixed when the direction of traffic is switched, and the global direction indicates an unexpected direction. According to this configuration, the mobile object is not tracked when the general direction indicates an unexpected direction. Thereby, it is possible to prevent the reverse determination unit 27 from erroneously determining the reverse moving object when the traffic direction is switched.

(10) The global direction acquisition unit may determine the direction of the average vector of motion vectors included in all or a part of the region of the motion vector acquired by the motion vector acquisition unit. May be calculated as
According to this configuration, the global direction can be calculated by simple calculation.

  Note that the method of calculating the global direction is not limited to this. For example, instead of the direction of the average vector of the motion vectors, the direction of the mode vector of the motion vector may be calculated as the global direction.

(11) A moving body tracking method according to another aspect of the present invention includes a step of obtaining a motion vector indicating movement of a moving body between images constituting a moving image, and the moving body included in the moving image. A step of acquiring a global direction indicating a global movement direction, and tracking a moving body that runs backward in the global direction based on a motion vector having a direction different from the global direction by a predetermined angle or more among the acquired motion vectors. Steps.
According to this configuration, the same operation and effect as the moving body tracking device shown in (1) above can be achieved.

(12) A program according to still another aspect of the present invention includes a step of obtaining a motion vector indicating a motion of a moving body between images constituting a moving image, and a global view of the moving body included in the moving image. Acquiring a global direction indicating a specific direction of movement, and tracking a mobile body going backward in the global direction based on a motion vector having a direction different from the global direction by a predetermined angle or more among the acquired motion vectors. Is a program for causing a computer to execute.
According to this configuration, the same operation and effect as the moving body tracking device shown in (1) above can be achieved.

(13) A moving body tracking system according to another aspect of the present invention includes a camera that captures a moving image, and the above-described moving body tracking device that processes the moving image captured by the camera.
According to this configuration, it is possible to determine whether or not the moving body is moving backward based on the moving image captured by the camera.

(14) A camera according to still another aspect of the present invention includes an imaging unit that captures a moving image, and the above-described moving body tracking device that processes the moving image captured by the imaging unit.
According to this configuration, it is possible to perform the moving image capturing process and the moving object retrograde determination process based on the captured moving image with one camera.

(15) A camera according to another aspect of the present invention includes an imaging unit that captures a moving image, the imaging direction and the angle of view being adjustable, the above-described moving body tracking device, and a moving body using the moving body tracking device. And an adjustment unit that adjusts the imaging direction and the angle of view of the imaging unit so as to magnify and image the moving body that is traveling in reverse.
According to this configuration, it is possible to perform a retrograde determination process for a moving object based on a moving image and an imaging process for an image obtained by enlarging the retrograde moving object.

[Details of the embodiment of the present invention]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Each of the embodiments described below shows a preferred specific example of the present invention. Numerical values, shapes, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present invention. The invention is specified by the claims. Therefore, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept of the present invention are not necessarily required to achieve the object of the present invention. It will be described as constituting a preferred form.

[1. Overall configuration of mobile tracking system]
FIG. 1 is a diagram showing a configuration of a mobile tracking system including a mobile tracking device according to an embodiment of the present invention.

  The moving body tracking system 1 is a system that tracks a moving body that goes against the general flow of the moving body from an image captured by the camera, and includes a camera 10, a moving body tracking device 20, a display device 30, and the like. And a speaker 40. In the following description, the moving body is described as being a person, but the present disclosure can be applied to a moving body such as an animal or a vehicle other than a person.

  The camera 10 is a surveillance camera that is installed outdoors or indoors for the purpose of crime prevention and captures time-series images (hereinafter referred to as “moving images”) of an imaging region. FIG. 1 shows an example in which the camera 10 is installed on the road as a monitoring target. The camera 10 is usually installed on a utility pole, a dedicated columnar jig, or the like in order to overlook a place where there is a flow of human movement.

  The mobile tracking device 20 is connected to the camera 10 via a computer network such as the Internet 50 or a LAN (Local Area Network), and moves backward from the moving image captured by the camera 10 to the general flow of the mobile body. A device that tracks the body. The mobile body tracking device 20 is configured by, for example, a computer including a CPU (Central Processing Unit) and a RAM (Random Access Memory). In addition, the mobile body tracking device 20 and the camera 10 may be directly connected by a cable.

The display device 30 is connected to the moving body tracking device 20 and displays a moving image captured by the camera 10 and a processing result by the moving body tracking device 20. The display device 30 is, for example, a liquid crystal display.
The speaker 40 is connected to the mobile tracking device 20 and outputs a processing result by the mobile tracking device 20 as a sound.

  The mobile tracking device 20, the display device 30, and the speaker 40 are installed, for example, indoors or the like different from the installation location of the camera 10, and the mobile tracking device 20 displays the display device 30 to the user of the mobile tracking system 1. Alternatively, the processing result by the mobile tracking device 20 is presented through the speaker 40.

[2. Hardware configuration of mobile tracking system]
FIG. 2 is a block diagram showing a hardware configuration of the mobile tracking system 1.

  The camera 10 includes an imaging unit 11 and an adjustment unit 12. The imaging unit 11 includes a charge coupled device (CCD), a lens, and the like to capture an image to be monitored. The imaging unit 11 has a pan function, a tilt function, and a zoom function, and can capture an enlarged image of the imaging target. In other words, the imaging unit 11 changes the imaging direction of the imaging target with the pan function and the tilt function, adjusts the angle of view with the zoom function, and captures an enlarged image of the imaging target.

  The adjustment unit 12 controls the imaging direction and zoom magnification by the imaging unit 11 based on a control signal output from a camera control unit 29 of the moving body tracking device 20 described later. In addition, the adjustment unit 12 adjusts the focus and shutter speed of the imaging unit 11.

  The moving body tracking device 20 includes an image reception unit 21, an image storage unit 22, a motion vector acquisition unit 23, a global direction acquisition unit 24, a retrograde tracking unit 25, a retrograde determination unit 27, a presentation unit 28, And a camera control unit 29.

  The image reception unit 21 receives a moving image captured by the imaging unit 11 of the camera 10 via the Internet 50. FIG. 3 is a diagram illustrating an example of an image 112 constituting the moving image captured by the imaging unit 11. The image 112 shows ten people 101 to 110 walking. Among these, the persons 101 and 103 to 110 other than the person 102 are moving in the direction from right to left on the page, but the person 102 is in a direction opposite to the moving direction of these persons 101 and 103 to 110. It moves in the direction from the left to the right of the page. That is, the person 102 goes against the general flow of the person.

  The image storage unit 22 stores the image 112 received by the image reception unit 21. The image storage unit 22 is configured by a storage device such as an HDD (Hard Disk Drive).

  The motion vector acquisition unit 23 acquires a motion vector indicating the motion of the moving body between the images constituting the moving image. FIG. 4 is a diagram in which a motion vector indicating a person's motion between two consecutive images is superimposed on the image 112. In FIG. 4, it is assumed that the motion vectors 114 and 116 are indicated by arrows, the direction of the arrow indicates the direction of the motion vector, and the length of the arrow indicates the length of the motion vector. In the figure, reference symbols are assigned to only representative motion vectors. The motion vector acquisition unit 23 calculates a motion vector for feature points such as edge portions and corner portions in the image 112. That is, the motion vector acquisition unit 23 calculates a motion vector of a human feature point by calculating an optical flow indicating a motion vector distribution using a block matching method or a gradient method.

  The global direction acquisition unit 24 acquires a global direction indicating the global movement direction of the moving object included in the moving image. That is, the global direction acquisition unit 24 calculates the average vector of all the motion vectors 114 and 116 included in the image 112, and calculates the direction indicated by the calculated average vector as the global direction. In FIG. 4, the global direction 118 is indicated by an arrow, and the direction from right to left on the paper is the global direction 118.

  The retrograde tracking unit 25 tracks a mobile body that runs backward in the global direction 118 based on the motion vector 116 having a direction different from the global direction 118 by a predetermined angle or more among the motion vectors 114 and 116 acquired by the motion vector acquisition unit 23. . In order to realize this tracking process, the backward tracking unit 25 includes a clustering unit 25A and a tracking unit 25B.

  The clustering unit 25A classifies motion vectors having directions different from the global direction 118 by a predetermined angle or more among the motion vectors acquired by the motion vector acquisition unit 23 based on the distribution of the motion vectors in the image 112.

  FIG. 5 is a diagram for explaining how to obtain the motion vector 116 having a direction different from the global direction 118 by a predetermined angle or more. In FIG. 5, it is assumed that the global direction 118 is a 180 ° direction. At this time, the clustering unit 25A extracts, for example, a motion vector 116 having a direction different from the global direction 118 by 120 ° or more. That is, the clustering unit 25A extracts a motion vector 116 having a hatched direction (−60 ° (300 °) or more and 60 ° or less) as a motion vector 116 having a direction different from the global direction 118 by 120 ° or more. In FIG. 4, the motion vector 116 is indicated by a solid arrow. The other motion vectors 114 are indicated by dashed arrows.

  The clustering unit 25A clusters the motion vectors 116 by classifying the motion vectors 116 whose distance between the start points of the motion vectors 116 is equal to or less than a predetermined distance threshold into the same cluster. That is, the motion vectors 116 located in the vicinity of each other are classified into the same cluster. FIG. 6 is a diagram showing clusters in an ellipse cluster frame. A cluster frame 120 is generated from the motion vector 116 of the person 102. The cluster frame 120 is, for example, an ellipse having a size including the motion vector 116 classified into the same cluster with the center of gravity 122 of the distribution of the motion vectors 116 classified into the same cluster as the center. The clustering unit 25A classifies the motion vectors 114 included in the cluster frame 120 into the same clusters as the motion vectors 116.

  The motion vector 116 is also generated by the leg swing of the person 105. Therefore, the clustering unit 25A performs clustering on the motion vector 116 of the person 105 in the same manner as the motion vector 116 of the person 102, and sets the cluster frame 124 and the centroid 126. The clustering unit 25A similarly sets the cluster frames 128 and 132 and the centroids 130 and 134 for the people 107 and 108. Note that the shape of the cluster frame is not limited to an ellipse, and may be a circle, a rectangle, or a polygon.

For each cluster classified by the clustering unit 25A, the tracking unit 25B determines that the moving object corresponding to the cluster has moved in the direction of the vector representing the motion vector included in the cluster, thereby moving the tracking unit 25B in the global direction 118. Track retrograde moving objects.
Hereinafter, the tracking process of the moving body by the tracking unit 25B will be specifically described with reference to FIGS.

  Referring to FIG. 7, tracking unit 25B calculates average vector 136 of all motion vectors 114 and 116 included in cluster frame 120. The tracking unit 25B extracts a motion vector whose direction difference from the calculated direction of the average vector 136 is within a predetermined angle. For example, when the predetermined angle is 60 °, only the motion vector 116 out of the motion vectors 114 and 116 is extracted by the tracking unit 25B. By such processing, the motion vector 114 having a different orientation from the cluster frame 120 can be removed as noise.

  Referring to FIG. 8, tracking unit 25B calculates average vector 138 of extracted motion vectors 116. The tracking unit 25B tracks the person 102 traveling backward in the global direction 118 by determining that the person 102 corresponding to the cluster frame 120 (cluster) has moved in the direction of the average vector 138.

  FIG. 9A is a diagram illustrating a tracking result of the person 102 going backward in the global direction 118 in 10 consecutive images. It can be seen that the average vector 138 continuously moves from the left to the right of the page.

  On the other hand, FIG. 9B shows a result of tracking the average vector 138 calculated from the motion vector 116 generated by the leg swing of the person 107. Although the leg temporarily moves in the direction opposite to the general direction 118 by swinging the leg, the time does not last long. For this reason, the average vector 138 having a direction reverse to the global direction 118 can be tracked only with a few continuous images.

  Referring again to FIG. 2, when the tracking unit 25 </ b> B can track the moving body over a predetermined number of consecutive images constituting the moving image, the backward moving determination unit 27 moves backward in the global direction 118. It is determined that For example, when the predetermined number is 10, the tracking unit 25B can track the person 102 over 10 images, as shown in FIG. 9A. For this reason, the retrograde determination unit 27 determines that the person 102 is retroactive in the general direction 118. On the other hand, as shown in FIG. 9B, the tracking unit 25B can track the leg of the person 107 only over two images. For this reason, the retrograde determination unit 27 does not determine that the person 107 is retroactive. Similarly, the retrograde determination unit 27 does not determine that the person 105 or the person 108 or other people are retroactive.

  The retrograde determination unit 27 may further determine whether or not the mobile object is retrograde using a size criterion. When the size of the moving object is too small or too large, the retrograde determination unit 27 determines that the moving object is not a moving object assumed by the moving object tracking device 20, that is, a person. can do. For this reason, the retrograde determination unit 27 can track the moving body over a predetermined number of times, and the condition that the size of the moving body is equal to or larger than the first threshold and the size of the moving body is equal to or smaller than the second threshold. It may be determined that a person is going backward only when one or both of the conditions are satisfied. Here, the size of the cluster frame may be regarded as the size of the moving object, or the area of the moving object is separated from the cluster frame by dividing the area having the motion vector 116 to calculate the size of the moving object. May be.

  When the retrograde determination unit 27 determines that the moving body is retrograde in the general direction 118, the presentation unit 28 presents to the user by image or sound that the mobile body is retrograde. For example, the presentation unit 28 displays an image 142 as illustrated in FIG. 10 on the display device 30. The presentation unit 28 displays a message 144 indicating that the moving body is going backwards, such as “There is a person going backwards in the flow!”. In addition, the presentation unit 28 displays the backward person 102 in a manner different from that of other persons. For example, the person 102 is highlighted with a thick frame, or the person 102 is surrounded with a frame 148.

  The presentation unit 28 may display an image 150 as shown in FIG. In the image 150, the retrograde person 102 is cut out and displayed. The person 102 may be cut out by separating a region having a similar motion vector 116 based on the motion vector 116 going backward in the global direction 118.

  Further, the presentation unit 28 cuts out a region including the person 102 who is going backward from the image 112 shown in FIG. 3, and displays an image 152 obtained by enlarging the cut out region as shown in FIG. 12 on the display device 30. Also good.

  Further, as shown in FIG. 1, the presentation unit 28 outputs a sound such as “There is a person who goes back to the flow” or outputs a warning sound from the speaker 40, and then goes back to the flow. The presence of a person may be presented to the user.

  The camera control unit 29 controls the imaging direction of the camera 10 according to the position of the person in the image so that an enlarged image of the person who is going backward is captured, and according to the size of the person in the image, The zoom magnification of the camera 10 is controlled. That is, the camera control unit 29 controls the imaging direction of the camera 10 so that the person who is going backward comes to the center of the image, and the zoom magnification so that the angle of view becomes narrower as the size of the person going backward is smaller. Increase The camera control unit 29 outputs a signal for controlling the imaging direction and zoom magnification of the camera 10 to the adjustment unit 12 of the camera 10 via the Internet 50. The adjustment unit 12 of the camera 10 that has received the control signal controls the imaging direction and zoom magnification of the imaging unit 11. Thereby, the imaging part 11 can image the image 152 which expanded the person 102 who is going backward as shown in FIG.

[3. Processing flow of mobile tracking device]
Next, the flow of determination processing for determining whether or not a person is moving backward by the mobile tracking device 20 will be described. FIG. 13 is a flowchart of the retrograde determination process executed by the mobile tracking device 20.
The image reception unit 21 receives an image 112 as shown in FIG. 3 among the moving images captured by the imaging unit 11 of the camera 10 and writes the image 112 in the image storage unit 22 (S1).

  The motion vector acquisition unit 23 calculates an optical flow from a plurality of temporally continuous images stored in the image storage unit 22, thereby calculating motion vectors 114 and 116 as shown in FIG. 4 for each image. (S2).

  The global direction acquisition unit 24 calculates the average vector of all the motion vectors 114 and 116 included in the image 112, and calculates the direction indicated by the calculated average vector as the global direction 118 (FIG. 4) (S3).

  The retrograde tracking unit 25 determines whether or not the global direction 118 is within a predetermined angle range determined in advance (S4). For example, when the direction of human traffic is restricted to the left, an angle range from 150 ° to 210 ° (FIG. 5) may be set as the predetermined angle range.

  If the global direction 118 is not within the predetermined angle range (NO in S4), the retrograde tracking unit 25 ends the process without performing the tracking process of the moving object. For example, it is assumed that the camera 10 images a sidewalk whose direction of travel is switched depending on the time zone, and there are a time zone in which the right direction is the passing direction and a time zone in which the left direction is the passing direction in FIG. At this time, when the traffic direction is switched, people heading in various directions may be mixed, and the global direction 118 may indicate an unexpected direction. If the moving object is tracked using the global direction 118 indicating an unexpected direction, it is erroneously assumed that the person is going backwards even though the person is moving in the direction of travel. It may be judged. By executing the determination process (S4), an erroneous retrograde determination can be prevented.

  When the global direction 118 is within the predetermined angle range (YES in S4), the clustering unit 25A determines the global direction 118 and the predetermined angle (for example, 120 °) out of the motion vectors 114 and 116 acquired by the motion vector acquisition unit 23. ) The motion vectors 116 having different directions are classified into clusters based on the distribution of the motion vectors 116 in the image 112 (S5). That is, as described above, the motion vectors 116 located in the vicinity of each other are classified into the same cluster, and a cluster frame 120 as shown in FIG. 6 is set.

  As shown in FIG. 7, the tracking unit 25B calculates an average vector 136 of all the motion vectors 114 and 116 included in the cluster frame 120 (S6).

The tracking unit 25B extracts the motion vector 116 having a different direction from the cluster frame 120 by extracting the motion vector 116 whose direction difference from the calculated direction of the average vector 136 is within a predetermined angle (for example, 60 °). Are removed as noise (S7).
Referring to FIG. 8, tracking unit 25B calculates average vector 138 of extracted motion vectors 116 (S8).

  The tracking unit 25B tracks the moving body that runs backward in the global direction 118 by determining that the moving body corresponding to the cluster frame 120 (cluster) has moved in the direction of the average vector 138 (S9).

  The retrograde determination unit 27 can track the moving object over a predetermined number of images, satisfies the condition that the size of the moving object is equal to or larger than the first threshold value, and the size of the moving object is equal to or smaller than the second threshold value. It is determined whether or not (S10). If the above condition is not satisfied (NO in S10), it is determined that there is no person going backwards, and the subsequent processing is not executed.

  When the above conditions are satisfied (YES in S10), the presentation unit 28 presents to the user by image or voice that the moving body is moving backward (S11). For example, the presentation unit 28 displays an image as shown in FIG. 10 or 11 on the display device 30 or outputs a voice message indicating that a person is going backward from the speaker 40.

  The camera control unit 29 controls the imaging direction and zoom magnification of the camera 10 so that an enlarged image of the person 102 who is going backward is captured (S12).

[4. Effects of the embodiment]
As described above, according to the present embodiment, the following effects can be achieved.
That is, the mobile body tracking device 20 tracks a mobile body that travels backward in the global direction based on a motion vector that has a direction different from the general direction by a predetermined angle or more without limiting the global direction. Moreover, the mobile body tracking device 20 can track a mobile body that moves backward in the general direction without limiting the speed of the mobile body. For this reason, it is possible to accurately track the moving body that goes against the general flow of the moving body without limiting the moving direction, speed, or acceleration of the moving body.

  Further, the clustering unit 25A clusters the motion vectors in the direction reverse to the general direction so that the motion vectors whose positions are close to each other are classified into the same cluster. For this reason, by tracking the moving object for each cluster, it is possible to track the moving object that is moving backward in units of moving objects or in units of groups of moving objects.

  As described with reference to FIGS. 7 and 8, the tracking unit 25B extracts a motion vector 116 whose direction difference from the direction of the average vector 136 is within a predetermined angle in the cluster. That is, the motion vector 114 whose direction difference is larger than the predetermined angle is removed as noise. Therefore, it is possible to track the moving body in the direction of the average vector 138 of the extracted motion vectors 116 after removing noise. Therefore, it is possible to accurately track a moving object that is moving backward.

  In addition, the retrograde determination unit 27 does not determine that the mobile object is retrograde when the mobile object cannot be tracked continuously for a long time. In other words, even if a part of the body moves backwards due to a hand or leg swing of a person moving in the general direction, these movements are not continuous, so this person goes backwards. It is not determined to be. Therefore, it can be accurately determined whether or not the moving body is moving backward in the general direction.

  Further, the retrograde determination unit 27 determines whether or not the size of the moving object satisfies the threshold condition. For this reason, even if the moving object can be tracked continuously for a long time, if the size of the moving object is too small or too large, the moving object tracking device 20 assumes the moving object. It can be determined that it is not a moving object and excluded from the tracking target.

  In addition, when it is determined that the moving body is going backward, the presentation unit 28 presents the determination result to the user with an image or sound. For this reason, the user can know that the moving body is going backward by looking at the image or listening to the sound. Therefore, the user can pay attention to the moving body that is moving backward so as not to go backward.

  Further, when it is determined that the moving body is moving backward, the presentation unit 28 presents an image as illustrated in FIGS. 10 to 12 or outputs a warning sound to the user. It is possible to easily inform that the moving body is going backwards. For this reason, the user can promptly pay attention to the moving body that is going backward so as not to go backward.

  In addition, when a sidewalk whose direction of traffic is switched depending on the time zone is imaged, there are cases where people heading in various directions are mixed when the direction of traffic is switched, and the general direction indicates an unexpected direction. However, the retrograde tracking unit 25 does not track the moving object when the global direction indicates an unexpected direction. Thereby, it is possible to prevent the reverse determination unit 27 from erroneously determining that the moving body is moving backward, even when the moving body is not moving backward when the traveling direction is switched.

  Also, the average vector of motion vectors is calculated as the general direction. Therefore, the general direction can be calculated by simple calculation.

  Further, the moving body tracking system 1 can determine whether or not the moving body is moving backward based on the moving image captured by the camera 10.

[5. Addendum]
The mobile tracking system including the mobile tracking device according to the embodiment of the present invention has been described above, but the present invention is not limited to this embodiment.

  For example, the global direction acquisition unit 24 calculates the average vector of motion vectors as the global direction, but may acquire the global direction from the outside. For example, when the mobile tracking device 20 determines a person who goes backward in the traveling direction of the escalator, the global direction acquisition unit 24 may acquire a predetermined traveling direction of the escalator as the global direction.

  Further, the calculation method of the global direction is not limited to the above-described method. For example, instead of the direction of the average vector of the motion vectors, the direction of the mode vector of the motion vector may be calculated as the global direction.

  The global direction may be set for each area of the image captured by the camera 10. For example, the processing may be divided into the right half and the left half of the image, and the general direction may be set for each. Thereby, for example, it is possible to detect a person who goes back on each escalator from an image obtained by simultaneously imaging the up escalator and the down escalator.

Further, the global direction acquisition unit 24 may calculate the global direction only from the motion vectors included in a predetermined region in the image, not the motion vectors included in the entire image.
The motion vector acquisition unit 23 may acquire a motion vector from the outside.

  In the configuration of the mobile tracking device 20 illustrated in FIG. 2, some functions may be provided in a server on a computer network. For example, the mobile tracking device 20 may transmit an input image to the server and obtain a reverse determination result from the server.

  The camera 10 may include a plurality of imaging units 11. For example, the camera 10 may include an imaging unit 11 that captures wide-angle video and an imaging unit 11 that includes a pan function, a tilt function, and a zoom function. In this case, the former imaging unit 11 captures a video of the monitoring target area at a wide angle, and the mobile tracking device 20 performs a retrograde determination process based on the video. When the retrograde is detected, the adjustment unit 12 of the camera 10 adjusts the imaging direction and the angle of view of the latter imaging unit 11 in accordance with an instruction from the moving body tracking device 20, so that the latter imaging unit 11 performs the retrograde. An image obtained by enlarging the person who is performing the operation may be captured.

  Further, the function shown in the mobile tracking device 20 may be provided in the camera 10. According to this configuration, it is possible to perform the moving image capturing process and the moving object retrograde determination process based on the captured moving image with one camera. Furthermore, it is possible to perform a moving object retrograde determination process based on a moving image and an imaging process of an image obtained by enlarging the retroactive moving object with one camera.

  Further, the mobile tracking device 20 may be specifically configured as a computer system including a microprocessor, a ROM (Read Only Memory), a RAM, a hard disk drive, a display unit, a keyboard, a mouse, and the like. . A computer program is stored in the RAM or hard disk drive. The mobile tracking device 20 achieves its functions by the microprocessor operating according to the computer program. Here, the computer program is configured by combining a plurality of instruction codes indicating instructions for the computer in order to achieve a predetermined function.

  Furthermore, some or all of the constituent elements constituting the mobile tracking device 20 may be configured by a single system LSI (Large Scale Integration). The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip, and specifically, a computer system including a microprocessor, ROM, RAM, and the like. . A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating according to the computer program.

  Further, the computer program is a computer-readable non-transitory recording medium, for example, a flexible disk, a hard disk drive, a CD-ROM, an MO (Magneto-Optical disc), a DVD (Digital Versatile Disc), a DVD-ROM (Digital). It may be recorded on a Versatile Disc Read Only Memory (DVD), a Digital Versatile Disc Random Access Memory (BD-RAM), a BD (Blu-ray (registered trademark) Disc), a semiconductor memory, or the like.

Further, the computer program or the digital signal may be transmitted via an electric communication line, a wireless or wired communication line, a network represented by the Internet, a data broadcast, or the like.
Furthermore, it is possible to arbitrarily combine at least a part of the embodiment and the modified example.

The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the meanings described above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  The present invention is useful when used in a monitoring system or the like that determines a person who is reversing the flow of people from an image captured by a monitoring camera.

DESCRIPTION OF SYMBOLS 1 Mobile tracking system 10 Camera 11 Imaging part 12 Adjustment part 20 Mobile tracking apparatus 21 Image reception part 22 Image storage part 23 Motion vector acquisition part 24 Global direction acquisition part 25 Reverse tracking part 25A Clustering part 25B Tracking part 27 Reverse determination part 28 Presentation unit 29 Camera control unit 30 Display device 40 Speaker 50 Internet 101-110 Person 112, 142, 150, 152 Image 114, 116 Motion vector 118 Global direction 120, 124, 128 Cluster frame 122, 126, 130 Center of gravity 136, 138 Average vector 144 Message 148 Frame

Claims (15)

A motion vector acquisition unit that acquires a motion vector indicating a motion of a moving body between images constituting a moving image;
A global direction acquisition unit that acquires a global direction indicating a global movement direction of the moving body included in the moving image;
A moving body tracking comprising: a backward tracking section that tracks a moving body that travels backward in the global direction based on a motion vector having a direction different from the general direction by a predetermined angle or more among the motion vectors acquired by the motion vector acquisition section. apparatus. The retrograde tracking unit
A clustering unit that classifies motion vectors having a direction different from the global direction by a predetermined angle or more among the motion vectors acquired by the motion vector acquisition unit based on a distribution of the motion vectors in the image;
For each cluster classified by the clustering unit, by determining that the moving body corresponding to the cluster has moved in the direction of the vector representing the motion vector included in the cluster, the moving body going backward in the global direction is The mobile body tracking device according to claim 1, further comprising a tracking unit that tracks. The tracking unit extracts, for each cluster classified by the clustering unit, a motion vector whose direction difference from the direction of the average vector of motion vectors included in the cluster is within a predetermined angle, and the average vector of the extracted motion vectors The mobile body tracking device according to claim 2, wherein the mobile body tracking device moves backward in the general direction by determining that the mobile body corresponding to the cluster has moved in the direction. Further, when the tracking unit can track the moving body over a predetermined number of continuous images constituting the moving image, a reverse determination unit that determines that the moving body is going backward in the general direction. The moving body tracking device according to claim 2 or 3. The retrograde determination unit is configured so that the tracking unit can track the moving body over the predetermined number of consecutive images constituting the moving image, and the size of the moving body is equal to or larger than a first threshold value. The mobile tracking device according to claim 4, wherein it is determined that the mobile is going backward in the general direction. The retrograde determination unit is configured so that the tracking unit can track the moving body over the predetermined number of consecutive images constituting the moving image, and the size of the moving body is equal to or smaller than a second threshold value. The mobile body tracking device according to claim 4, wherein the mobile body is determined to be retrograde in the general direction. Furthermore, when the retrograde determination unit determines that the mobile body is retrograde in the general direction, the retrograde determination unit includes a presentation unit that presents to the user by image or sound that the mobile body is retrograde. The moving body tracking device according to any one of claims 4 to 6. The presenting unit displays (i) a message indicating that the moving object is moving backward, and (ii) displays an image shown in a mode different from that of the moving object not moving backward. , (Iii) displaying an image composed of a moving body that is retrograde, (iv) displaying an image in which an area including the moving body that is retroactive is enlarged, or (v) a moving body is retrograde The moving body tracking device according to claim 7, wherein a warning sound indicating that is output. The mobile tracking device according to claim 1, wherein the backward tracking unit does not track the mobile when the general direction is not within a predetermined angle range. The global direction acquisition unit calculates, as the global direction, an average vector direction of motion vectors included in all or part of the region in the image among the motion vectors acquired by the motion vector acquisition unit. The moving body tracking device according to any one of claims 1 to 9. Obtaining a motion vector indicating the motion of the moving body between images constituting the moving image;
Obtaining a global direction indicating a global movement direction of the moving body included in the moving image;
Tracking a moving body that runs backward in the global direction based on a motion vector having a direction different from the global direction by a predetermined angle or more among the acquired motion vectors. Obtaining a motion vector indicating the motion of the moving body between images constituting the moving image;
Obtaining a global direction indicating a global movement direction of the moving body included in the moving image;
A program for causing a computer to execute a step of tracking a moving body that runs backward in the global direction based on a motion vector having a direction different from the general direction by a predetermined angle or more among the acquired motion vectors. A camera that captures moving images;
A moving body tracking system comprising: the moving body tracking device according to claim 1, which processes the moving image captured by the camera. An imaging unit that captures a moving image;
A moving object tracking device given in any 1 paragraph of Claims 1-10 which processes said moving picture imaged by said image pick-up part. An imaging unit that captures a moving image, the imaging direction and angle of view being adjustable;
The mobile tracking device according to any one of claims 4 to 8,
An adjustment unit that adjusts an imaging direction and an angle of view of the imaging unit so as to enlarge and capture the moving body that is moving backward when the moving body tracking device determines that the moving body is moving backward A camera with and.