JP2012108798A - Moving object tracking device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem in which, in a device for tracking a moving object in a monitoring space based on plural prediction positions generated at each time, tracking accuracy is deteriorated when a prediction position of the moving object at the current time is set in an obstacle area in the monitoring space.SOLUTION: A prediction position setting unit performs motion prediction using past position information stored in a storage unit to calculate plural prediction positions of a moving object at the current time. At that time, a prediction position correction unit 53 corrects, among the plural prediction positions, a prediction position 221 with an obstacle area 42a existing between the prediction position 221 and a prediction position 220 preceding one time being their movement source to a prediction position 241 being a position nearer to the movement source position than the obstacle area 42a. An object position calculation unit 56 determines a movement destination position of the moving object at the current time based on an image feature of the moving object which each of the plural prediction positions including the corrected prediction position has.

Description

  The present invention relates to a moving object tracking apparatus that tracks an object moving in a space, and more particularly to a moving object tracking apparatus that performs tracking processing using an image obtained by imaging a space.

  For the purpose of monitoring or the like, the position of a person or the like is tracked using a monitoring image obtained by imaging a monitoring space. At the time of tracking, the predicted position at the current time is obtained from the past movement of the tracked object, and whether the part corresponding to the predicted position in the monitoring image has the image feature of the tracked object is evaluated to track at the current time. A method for specifying the position of an object is sometimes used, and one of such methods is a particle filter (Patent Document 1).

  Since the particle filter can compensate for the misalignment of prediction by setting a large number of predicted positions for each tracking target, it is possible to track with high accuracy.

JP 2010-49296 A

  In the surveillance space, objects such as walls and furniture may exist as obstacles that obstruct the progress of the person who is the tracking target. In the prior art, there is a problem in that a predicted position is set at a position in or across an obstacle that prevents the tracking object from proceeding in the space, and the predicted position may be evaluated in the tracking process. there were. That is, since the tracking object does not actually enter or pass through the obstacle, the evaluation with respect to these predicted positions is invalid. These invalid prediction positions not only cause unnecessary processing, but also reduce the effective prediction positions, so that the effect of compensating for the misalignment of the prediction is diluted, which causes a deterioration in tracking accuracy. In addition to the obstacles, there are areas in the monitoring space that block people from reaching, such as moats, and it is basically difficult to enter such areas. Therefore, such a region can cause the same problem as the obstacle. Hereinafter, a spatial region in which the progress is inhibited including an obstacle is referred to as an obstacle region.

  The present invention has been made in view of the above problems, and an object of the present invention is to realize highly accurate tracking of a moving object even in the vicinity of an obstacle area existing in a space.

  The moving object tracking device according to the present invention uses a time-series image obtained by photographing a predetermined space to track an object moving in the space, and obstructs the movement in the space. Is stored in advance, the storage unit that stores information on the past position of the object in the past from the attention time, and the motion prediction of the object is performed using the past position, and the movement destination candidate of the object at the attention time is determined. A plurality of position predicting units for predicting and an object position determining unit that determines a moving destination of the object at the attention time based on an image feature of the object included in a corresponding portion of the movement destination candidate in the image at the attention time. And the position predicting unit determines an invalid destination candidate in which the failure area exists between the movement destination candidates and the past position that is the movement source, rather than the failure area. Comprising a correction unit for correcting the Domoto the side position.

  In the moving object tracking device according to another aspect of the present invention, the correcting unit transforms a route from the moving source to the invalid moving destination candidate while maintaining its route length, and converts the invalid moving destination candidate. It is corrected to the end point of the route after the deformation.

  A preferred aspect of the present invention is the moving object tracking device in which the correction unit performs the deformation by changing the direction of the route at the point where the route to the invalid destination candidate reaches the obstacle area. is there.

  Another preferable aspect of the present invention is the moving object tracking device in which the correction unit performs the deformation by changing a direction of a route to the invalid movement destination candidate at the movement source.

  In the moving object tracking device according to another aspect of the invention, the correction unit calculates the nearest point that is the shortest distance from the invalid destination candidate on the boundary of the source in the obstacle area, and The destination candidate is corrected to a position within a predetermined distance from the nearest point.

  According to the present invention, the predicted positions set in the area within or beyond the failure area are excluded, and the number of effective predicted positions is maintained by correcting these predicted positions before the failure area. -Since it can be secured, it is possible to track a moving object with high accuracy even around the obstacle area.

It is a block block diagram of the moving object tracking apparatus which concerns on embodiment of this invention. It is a perspective view which shows an example of a three-dimensional model typically. It is a schematic diagram of the obstacle map created corresponding to the three-dimensional model of FIG. It is a general | schematic flowchart of the tracking process of the moving object tracking apparatus which concerns on embodiment of this invention. It is a typical top view which shows an example of the prediction position set for every time by the prediction position setting part. It is a typical top view explaining the example of processing about the 1st form of correction processing. It is a typical top view explaining the example of processing about the 2nd form of correction processing. It is a typical top view explaining the example of a process regarding the 3rd form of correction processing. It is a typical top view which shows an example of the object position in the 4th-6th form of a correction process, and the predicted position of the present time. It is a typical top view explaining the process example regarding the 4th-6th form of a correction process.

  Hereinafter, a moving object tracking device 1 according to an embodiment of the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. The moving object tracking device 1 sets a room in which furniture is placed as a monitoring target space, and sets a person moving in the monitoring space as a tracking target (hereinafter referred to as a moving object). The moving object tracking device 1 processes a monitoring image obtained by imaging a monitoring space to track a moving object. Here, the area where the fixture is installed in the monitoring space is an obstacle area that hinders the movement of the moving object. Other examples of obstacle areas include walls, pillars and water heaters. The position of the obstacle area is known in advance without moving. The monitoring space is not limited to indoors but may be outdoor. In the case of outdoors, ponds, moats, etc., as well as houses, fences and various exteriors are obstacle areas. The moving object tracking device 1 is configured to perform tracking processing in consideration of the presence of an obstacle region, and improves tracking accuracy.

[Configuration of moving object tracking device]
FIG. 1 is a block diagram of a moving object tracking device 1 according to the embodiment. The moving object tracking device 1 includes an imaging unit 2, a setting input unit 3, a storage unit 4, a control unit 5, and an output unit 6. The imaging unit 2, the setting input unit 3, the storage unit 4, and the output unit 6 are connected to the control unit 5.

  The imaging unit 2 is a monitoring camera, is installed so as to face the monitoring space, and images the monitoring space at a predetermined time interval. The captured monitoring images of the monitoring space are sequentially output to the control unit 5. In order to grasp the position and movement of a person who moves along a reference plane such as the floor surface or the ground surface, the imaging unit 2 is basically installed at a height that allows a bird's-eye photography, for example, in this embodiment, The moving object tracking device 1 is used for indoor monitoring, and the imaging unit 2 is installed on the ceiling. The time interval at which the monitoring image is captured is 1/5 second, for example. Hereinafter, the unit of time recorded at the time interval of imaging is referred to as time.

  The setting input unit 3 is an input means for the administrator to perform various settings for the control unit 5, and is a user interface device such as a touch panel display, for example.

  The storage unit 4 is a storage device such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage unit 4 stores various programs and various data, and inputs / outputs such information to / from the control unit 5. The various data includes a three-dimensional model 40, a camera parameter 41, an obstacle map 42, a predicted position 43, and an object position 44.

  The three-dimensional model 40 is data describing a state in which a moving object model that approximates the three-dimensional shape of a moving object and / or an obstacle model that approximates the three-dimensional shape of an obstacle is arranged in a virtual space that simulates a monitoring space. An obstacle is an object or terrain that forms an obstacle area. In the present embodiment, the monitoring space and the virtual space are represented by a right-handed orthogonal coordinate system including the X, Y, and Z axes, and the vertically upward direction is set to the positive direction of the Z axis. A reference surface such as a floor surface is horizontal and is defined by an XY plane represented by Z = 0. The arrangement of the obstacle model in the virtual space is matched with the actual arrangement of the obstacle in the monitoring space. On the other hand, the arrangement of the moving object model can be set at an arbitrary position.

  The moving object model is, for example, data describing a partial model representing a three-dimensional shape for each of a plurality of constituent parts constituting the moving object, and an arrangement relationship between the partial models. The moving object to be monitored by the moving object tracking device 1 is a standing person. In this embodiment, for example, a spheroid approximating the three-dimensional shape of the three parts of the person's head, torso, and leg is used. A moving object model stacked in the Z-axis direction is set. The height from the reference plane to the center of the head is represented by H, and the maximum width of the trunk (the trunk minor axis diameter) is represented by W. In this embodiment, in order to simplify the description, the height H and the width W are standard human sizes and are common to any moving object. The center of the head is set as the representative position of the moving object. Note that the moving object model may be further simplified and approximated by one spheroid.

  The obstacle model is set for each obstacle in advance prior to the detection / tracking process of the moving object. The obstacle model includes data on the shape and dimensions of obstacles actually existing in the monitoring space, and data on the arrangement of obstacles in the monitoring space. The shape and dimensions of the obstacle can be obtained by actually measuring the obstacle placed in the monitoring space, and for example, three-dimensional data of a product provided by a fixture manufacturer or the like can be used. The arrangement of the obstacle can be obtained by actual measurement. For example, when designing the furniture layout of the room on the computer, the design data can be used. Data on the obstacle thus obtained is input to the control unit 5 from the setting input unit 3 or a connection interface (not shown) with an external device, and the control unit 5 stores the storage unit 4 based on the input data. The obstacle model is stored in.

  The camera parameter 41 includes information regarding a projection condition when the imaging unit 2 captures a monitoring image that projects the monitoring space. For example, external parameters such as the installation position and imaging direction of the imaging unit 2 in the actual monitoring space, focal length, angle of view, lens distortion, and other lens characteristics of the imaging unit 2, and internal parameters such as the number of pixels of the imaging element are included. These parameters obtained by actual measurement or the like are input in advance from the setting input unit 3 and stored in the storage unit 4. The three-dimensional model 40 can be projected onto the coordinate system of the monitoring image (imaging surface of the imaging unit 2; xy coordinate system) by a coordinate transformation formula in which the camera parameter 41 is applied to a camera model such as a known pinhole camera model.

  The obstacle map 42 (obstacle area) is information for specifying an area where an obstacle exists in the monitoring space. The moving object basically moves horizontally along the XY plane in the monitoring space. Therefore, in the present embodiment, as described above, the constraint condition that the lower end of the moving object model is in contact with the reference plane (Z = 0) is imposed. Correspondingly, the obstacle map 42 represents the obstacle area in the XY coordinate system of the reference plane, that is, the floor plane coordinates of the monitoring space. Here, the obstacle map 42 may be configured to specify the obstacle model itself, that is, the existence range of the obstacle with the three-dimensional coordinates in the monitoring space. As described above, the obstacle area is represented by the horizontal coordinate (XY coordinate system) of the monitoring space, and the obstacle area represented in two dimensions is referred to as an obstacle map. The data format of the obstacle map 42 is digital image data corresponding to the reference plane (Z = 0), and corresponds to an image when the monitoring space is viewed from directly above. Each pixel corresponds to a quantized position on the reference plane, and constitutes image data composed of N × M pixels quantized with, for example, N pixels in the X-axis direction of the reference plane and M pixels in the Y-axis direction. To do. The pixel value of each pixel represents whether or not the pixel is a position where an obstacle exists. In this embodiment, a pixel with a pixel value “1” is an obstacle location, while a pixel value “0” is defined as a location where no obstacle exists.

  The predicted position 43 (hypothesis) is information regarding the predicted value (predicted position) of the position of the moving object at each time. In order to determine the position (object position) of a moving object stochastically, a large number of predicted positions are set for each moving object (the number is represented by P. For example, 200 per moving object). Specifically, the predicted position 43 is time-series data in which the identifier of the moving object is associated with the predicted position index (0 to P-1) and the position coordinates (XYZ coordinate system) at each time.

  The object position 44 is information regarding the position of the moving object at each time, and specifically is time-series data in which the identifier of the moving object is associated with the position coordinates (XYZ coordinate system).

  Note that the processing can be speeded up as a configuration in which the predicted position 43 and the object position 44 are specified by the horizontal coordinate (XY coordinate system) of the monitoring space. In the present embodiment, the configuration will be described as an example for easy understanding.

  The control unit 5 is configured using an arithmetic device such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit), and reads out and executes a program from the storage unit 4 to create an obstacle map. Functions as a unit 50, a changed pixel extraction unit 51, a predicted position setting unit 52, a predicted position correction unit 53, an object region estimation unit 54, a likelihood calculation unit 55, an object position calculation unit 56, and an abnormality determination unit 57.

  The obstacle map creation unit 50 projects all obstacle models in the three-dimensional model 40 onto the horizontal plane of the monitoring space, calculates an obstacle region, creates an obstacle map 42, and stores it in the storage unit 4. For example, a reference surface (Z = 0) such as a floor surface or the ground can be set as a horizontal plane to be projected.

  The change pixel extraction unit 51 extracts a change pixel from the monitoring image newly input from the imaging unit 2, and outputs information about the extracted change pixel to the likelihood calculation unit 55. Therefore, the change pixel extraction unit 51 performs a difference process between the newly input monitoring image and the background image, and extracts a pixel whose difference is equal to or greater than a preset difference threshold as a change pixel. Note that the changed pixels may be extracted by a correlation process between the newly input monitoring image and the background image instead of the difference process, or the background model is learned instead of the background image and the difference process with the background model is performed. The change pixel may be extracted by.

  The predicted position setting unit 52 (hypothesis setting unit) and the predicted position correcting unit 53 perform motion prediction of the moving object using past position information (object position or predicted position) of the moving object, and at the current time of the moving object. It functions as a position prediction unit that obtains a plurality of predicted positions that are candidate movement destination positions.

  First, the predicted position setting unit 52 performs motion prediction from the object position of each moving object determined in the past or the predicted position of each moving object set in the past, and moves at the time when a new monitoring image is input. The position where the object exists is predicted, and the predicted position (predicted position) is output to the object region estimation unit 54, the likelihood calculation unit 55, and the object position calculation unit 56. As described above, P predicted positions are set for each moving object. A method of determining a position (object position) of a moving object in a probabilistic manner by sequentially setting a large number of predicted positions in this manner is called a particle filter or the like, and the set predicted position is called a hypothesis or the like. The predicted position can be set in the xy coordinate system of the monitoring image, but in the present embodiment, it is set in the XYZ coordinate system of the monitoring space. Motion prediction is performed by applying a predetermined motion model or applying a predetermined prediction filter to past position data.

  In the present embodiment, the average speed vector is calculated from the predicted positions for T times (for example, T = 5) before the attention time. This average speed vector is added to the predicted position one time ago to predict a new predicted position at the time of interest. Prediction is performed for each of the P predicted positions, and the same identifier is assigned to the new predicted position and the previous predicted position that is the basis of the new predicted position, and is circularly stored. In addition, the prediction position whose likelihood is lower than a preset likelihood threshold value is deleted from the predicted position one hour ago. On the other hand, in order to compensate for this deletion, a new predicted position one hour before the same number as the deleted number is randomly selected within a predetermined range (for example, within a circle with a radius r) centered on the predicted object position one hour before. And the predicted positions two hours before before corresponding to these predicted positions are extrapolated in accordance with the motion of the past object position. Therefore, in addition to the past predicted position, the object position for the past T time is also circulated and stored in the storage unit 4.

  When a new moving object is detected, the predicted position setting unit 52 randomly sets P predicted positions within a predetermined range centered on the detected position.

  The predicted position correcting unit 53 has a failure region between the predicted position (movement destination candidate position) newly set by the predicted position setting unit 52 and the predicted position (movement source position) at the previous time corresponding to the predicted position. If it exists, the movement destination candidate position is corrected to a position before the obstacle area when viewed from the movement source position. By this correction, the invalid prediction position set in the failure area and the invalid prediction position set across the failure area are excluded, and instead, the effective prediction position is set in front of the failure area.

  For each predicted position, the object region estimation unit 54 generates a three-dimensional model 40 in which a moving object model is virtually arranged at the predicted position in the monitoring space. Then, the camera parameter 41 is used to project the 3D model 40 onto a projection plane that is virtually set corresponding to the imaging surface of the imaging unit 2, and an area where the moving object is imaged in the monitoring image ( (Object region) is estimated, and the estimation result is output to the likelihood calculating section 55. The object region estimation unit 54 hides the moving object model whose object region is to be estimated by an obstacle model or other moving object model existing in front of the moving object model as viewed from the camera in the three-dimensional model 40 during projection. The non-hidden part (non-hidden area) is obtained as the object area of the moving object.

The likelihood calculating unit 55 extracts the feature amount of the moving object in the object region estimated for each predicted position from the monitoring image, and the likelihood as the object position of the predicted position according to the degree of feature amount extraction. Is output to the object position calculation unit 56. The following (1) to (4) are examples of likelihood calculation methods.
(1) The object region is superimposed on the change pixel extracted by the change pixel extraction unit 51, and the ratio (inclusion degree) in which the change pixel is included in the object region is obtained. Inclusion degree increases as the predicted position is closer to the position where the moving object actually exists, and tends to decrease as the distance increases. Therefore, the likelihood corresponding to the high degree of inclusion is calculated.
(2) Extract an edge from a portion corresponding to the contour of the object region in the monitoring image. The closer the predicted position is to the position where the moving object actually exists, the more the contour of the object area matches the edge position, so the degree of edge extraction (for example, the sum of the extracted edge strengths) increases, while the degree of extraction increases as the distance increases. Tends to be low. Therefore, the likelihood corresponding to the degree of edge extraction is calculated.
(3) The feature amount extracted from the monitoring image at the past object position of each moving object is stored in the storage unit 4 as reference information of the moving object. As the predicted position is closer to the position where the moving object actually exists, the feature quantity of the background and other moving objects will not be mixed, so the similarity between the feature quantity extracted from the object area and the reference information will be higher, but away from it The degree of similarity tends to be low. Therefore, the feature amount in the object region is extracted from the monitoring image, and the similarity between the extracted feature amount and the reference information is calculated as the likelihood. As the feature amount here, for example, various image feature amounts such as an edge distribution, a color histogram, or both of them can be used.
(4) The likelihood may be calculated according to a weighted addition value of a plurality of degrees of the above-described inclusion degree, edge extraction degree, and similarity degree.

  Here, the object region estimation unit 54 estimates a non-concealed region as an object region and uses it for likelihood calculation, whereby the likelihood suitable for the concealment state of each target can be calculated, and the tracking reliability can be improved. .

The object position calculation unit 56 determines the position (object position) of the moving object from each predicted position of the moving object and the likelihood calculated for each predicted position, and the determination result is stored in the storage unit 4 for each moving object. Accumulate in time series. When all the likelihoods are less than a predetermined lower limit (likelihood lower limit), it is determined that there is no object position, that is, disappeared. The following (1) to (3) are examples of the object position calculation method.
(1) For each moving object, a weighted average value of predicted positions weighted by likelihood is calculated, and this is set as the object position of the moving object.
(2) For each moving object, a predicted position where the maximum likelihood is calculated is obtained and set as the object position.
(3) For each moving object, an average value of predicted positions at which likelihoods equal to or higher than a preset likelihood threshold value are calculated is set as the object position. Here, the likelihood threshold> the likelihood lower limit value.

  The likelihood calculating unit 55 and the object position calculating unit 56 determine the destination position of the moving object at the current time based on the image feature of the moving object that the image has at each destination candidate position. As a function.

  The abnormality determination unit 57 refers to the time-series object positions accumulated in the storage unit 4, determines that a suspicious movement that stays for a long time or a suspicious movement that deviates from the normal flow line is abnormal, and determines the abnormality. And output an abnormal signal to the output unit 6.

  The output unit 6 includes a sound output unit such as a speaker or a buzzer that outputs a warning sound, a display unit such as a liquid crystal display or a CRT that displays a monitoring image determined to be abnormal, and an abnormality signal is output from the abnormality determination unit 57. When input, the fact that an error has occurred is output to the outside. The output unit 6 may include a communication unit that transmits an abnormal signal to a center device installed in a monitoring center of a security company via a communication line.

[Operation of Moving Object Tracking Device and First Form of Correction Process]
Next, the tracking operation of the moving object tracking device 1 will be described. Prior to the tracking process, the camera parameter 41 and various data of the obstacle model are input from the setting input unit 3 and stored in the storage unit 4 as pre-processing, and the obstacle map creation unit 50 uses these data to check the obstacle map. 42 is created.

  FIG. 2 is a perspective view schematically showing an example of the three-dimensional model 40, and shows an arrangement example of the reference plane 100 of N × M pixels, the camera position 101, the obstacle model 102, and the moving object model 103.

  FIG. 3 is a schematic diagram of an obstacle map 42 created corresponding to the three-dimensional model 40 of FIG. In FIG. 2, a white portion is an obstacle region and a pixel value “1” is set, and a shaded portion is a region where no obstacle exists and a pixel value “0” is set.

  FIG. 4 is a schematic flowchart of the tracking process of the moving object tracking device 1.

  The imaging unit 2 inputs a monitoring image to the control unit 5 every time the monitoring space is imaged (S1). Hereinafter, the time when the latest monitoring image is input is called the current time, and the latest monitoring image is called the current image.

  The current image is compared with the background image by the change pixel extraction unit 51, and the change pixel extraction unit 51 extracts the change pixel (S2). Here, the isolated change pixel is excluded from the extraction result as being caused by noise. Immediately after the start of the operation without a background image, the current image is stored in the storage unit 4 as a background image, and there is no change pixel for convenience.

  The predicted position setting unit 52 sets P predicted positions based on the motion prediction for each moving object being tracked (S3). In addition, since the predicted position of the moving object determined to be a new appearance in step S15 described later cannot be predicted, P predicted positions are set in a wider range centering on the appearance position. Further, the predicted position of the moving object determined to be lost in step S15 described later is deleted.

  FIG. 5 is a schematic plan view showing an example of a predicted position set for each time by the predicted position setting unit 52, and shows a change in the predicted position from time (t-3) to time t. In FIG. 5, “◯” indicates a predicted position, and “Δ” indicates an object position, and the centers of “◯” and “Δ” correspond to the predicted position and the object position, respectively. P predicted positions 200 were set at a time (t-3) three hours before the current time t, and the object position 210 was determined based on these. At the subsequent time (t−2), P predicted positions 201 corresponding to the predicted position group 200 are set, and the object position 211 is determined based on these. Similarly, at time (t−1), P predicted positions 202 are set and the object position 212 is determined. At the current time t, the predicted position group 200, the predicted position group 201, and the predicted position group 202 correspond to each. P predicted positions 203 to be set are set.

  Here, the predicted position 223 set in front of the failure area 42a in the predicted position group 203 is a position where the moving object can move, but straddles the predicted position 221 and the failure area 42a set in the failure area 42a. The predicted position 222 set in (1) is a position that cannot be reached by a moving object, and is an invalid setting. Setting a large number of prediction positions has an effect of compensating for the misalignment of prediction, but an invalid setting dilutes the effect of compensating for the misalignment of prediction and decreases the accuracy of the tracking result. Such a decrease in accuracy becomes more serious if the moving speed of the moving object is high or the frame period of the monitoring image is long.

  If the change pixel is not extracted in step S2 and the predicted position is not set in step S3 (there is no moving object being tracked) (if “YES” in S4), the control unit 5 performs step. Returning to S1, the input of the next monitoring image is awaited.

  On the other hand, if “NO” in the step S4, the processes of the steps S5 to S15 are performed. The control unit 5 sequentially sets each moving object being tracked as a target object (S5). Subsequently, the control unit 5 sequentially sets each predicted position of the target object as the target position (S6). However, the predicted position outside the field of view of the monitoring image is excluded from the target position setting targets, the object region at the predicted position is not estimated, and the likelihood is set to zero.

  The predicted position correcting unit 53 reads the predicted position one hour before corresponding to the target position from the storage unit 4, and moves a straight line connecting the read predicted position (movement source position) and the target position (movement destination candidate position) to the moving object. (S7).

  Next, the predicted position correction unit 53 refers to the obstacle map 42 and determines whether or not an obstacle area exists on the route (S8). If there is an obstacle area (in the case of “YES” in S8), the attention position is corrected to a position before the obstacle area when viewed from the movement source position side (S9). On the other hand, if there is no failure area (in the case of “NO” in S8), the correction process S9 is not executed.

  FIG. 6 is a schematic plan view for explaining an example of the correction process S9, and FIG. 6A is a diagram for explaining the correction process S9 taking the correction to the predicted position 221 at the current time shown in FIG. FIG. 6B is a diagram illustrating a change in the spatial distribution of the predicted position due to the correction. The predicted position 220 at time (t−1) is the predicted position at the previous time corresponding to the predicted position 221 that is the target position. The predicted position correcting unit 53 calculates a straight line 230 connecting the predicted position 220 of the previous time and the predicted position 221 of the current time newly set corresponding thereto as a movement route. If any pixel on the straight line representing the movement path is a pixel value representing the obstacle region 42a on the obstacle map 42, the predicted position correcting unit 53 determines the newly set predicted position as the correction target. On the other hand, if there is no pixel on the straight line representing the movement path having a pixel value representing the failure area 42a, it is determined that the newly set predicted position does not require correction. In this way, by determining using the positional relationship with the predicted position of the previous time, not only the predicted position set in the failure area 42a but also the predicted position set across the failure area 42a are detected as correction targets. can do. By this determination processing, the predicted position 221 at the current time shown in FIG.

  The predicted position correcting unit 53 changes the direction of the moving path at the point where the moving path from the predicted position at the previous time to the target position reaches the obstacle area, and corrects the end point of the moving path whose direction has been corrected Let it be a position (moving destination candidate position).

  Specifically, in the determination of the correction target, the pixel values on the straight line 230 are sequentially scanned from the predicted position 220 at the previous time, and the pixels in which the pixel value representing the faulty area 42a is first detected are the straight line 230 and the faulty area 42a. The contact point 231 is determined. A pixel (contour pixel) on the contour line of the obstacle region 42a continuing from the contact point 231 within a circle having a predetermined radius ρ centered on the contact point 231 is extracted, and a straight line (contour straight line) approximating the contour pixel is derived. In view of the movement of the moving object to avoid the obstacle region 42a, it is presumed that it is appropriate to grasp the shape of the contour line with a resolution about the size of the moving object. Therefore, the contour line in the vicinity of the contact point 231 is assumed. It is appropriate that the range for linearly approximating is also a size corresponding to the resolution. That is, the radius ρ may be, for example, W / 2. When the incident angle of the straight line 230 to the obstacle region 42a is defined using the derived contour straight line and expressed by φ, the angle θ formed by the straight line 230 and the contour straight line is calculated as θ = 90 ° −φ. The predicted position correction unit 53 obtains the θ and sets an acute angle Δ (where Δ ≧ 0) at random, and a contour line that forms an angle θ with the straight line 230 centering on the contact point 231 on the moving path ahead from the contact point 231. Is rotated by an angle (θ + Δ) to correct the predicted position 221 within a region outside the obstacle region 42a on the predicted position 220 side. As a result, the predicted position 221 is corrected to the predicted position 241. When this correction is performed on all correction targets in the predicted position group 203, the predicted position group 203 is corrected to the predicted position group 253 as shown in FIG.

  The movement path length from the predicted position 220 at the previous time to the corrected predicted position 241 via the contact point 231 is equal to the moving path length from the predicted position 220 at the previous time to the predicted position 221 before the correction. . The corrected predicted position is located outside the obstacle area on the predicted position side of the previous time and is within a range in which the moving object can move, and can move at the speed obtained by motion prediction at the previous time. This is preferable in terms of the path length.

  As described above, an invalid predicted position set to a position where a moving object cannot reach, such as a predicted position 221 set in the fault area 42a or a predicted position 222 set across the fault area 42a, is predicted. The effect of compensating for the gap was diluted. Since the predicted position correcting unit 53 corrects these useless predicted positions to positions where the moving object can move, the predicted position correction unit 53 can increase a significant predicted position even in the vicinity of the obstacle, and can track in the vicinity of the obstacle. Accuracy can be improved.

  The object region estimation unit 54 arranges the moving object model at a position corresponding to the target position, further creates a three-dimensional model 40 in which the obstacle model is arranged at each installation position, and uses the three-dimensional model 40 as a camera parameter. 41 is used to project onto the imaging surface of the imaging unit 2. Then, the object area estimation unit 54 extracts the area where the moving object model is projected as the object area corresponding to the target position. If a part of the moving object is concealed by an obstacle or the like, an object region lacking the concealed portion is extracted, and if it is not concealed, an object region representing the whole body is extracted.

  When the object region at the target position is obtained, the likelihood calculating unit 55 calculates the likelihood for the target position (S11).

  The control part 5 repeats step S5-S11, when the prediction position in which the likelihood is not calculated remains (in the case of "NO" in S12). When the likelihood is calculated for all P predicted positions (in the case of “YES” in S12), the object position calculation unit 56 calculates the likelihood calculated for each predicted position of the target object and each of the predicted positions. Is used to calculate the object position of the object of interest (S13). The object position calculated for the current time is added in association with the object position of the object of interest stored in the storage unit 4 one hour before. In the case of a newly appearing moving object, a new identifier is assigned and registered. If the likelihood at all predicted positions is less than the lower limit of likelihood, it is determined that there is no object position.

  When the unprocessed moving object remains (“NO” in S14), the control unit 5 repeats the processes S5 to S13 for determining the object position of the moving object. On the other hand, when the object positions are determined for all the moving objects, new appearance and disappearance of the objects are determined (S15). Specifically, the control unit 5 synthesizes the object regions estimated for the respective object positions, detects change pixels outside the synthesis region among the change pixels extracted by the change pixel extraction unit 51, and detects them. Among the changed pixels, the adjacent changed pixels are labeled. If the label is of a size that can be regarded as a moving object, a new appearance is recorded in the storage unit 4 together with the label position (appearance position). If there is a moving object without an object position, the fact that the moving object has disappeared is recorded in the storage unit 4. When the above processing is completed, the processing returns to step S1 in order to perform processing on the monitoring image at the next time.

[Second form of correction processing]
The form of the correction process of the predicted position in the overall operation of the moving object tracking device 1 described above is an example, and another form of the correction process will be described below.

  The second and third forms of the correction process described below are the same as those described with reference to FIG. 6 in that the predicted position at the current time is set in correspondence with the predicted position at the previous time as shown in FIG. Although common to the first mode, the prediction position correction method in the prediction position correction unit 53 is different.

  FIG. 7 is a schematic diagram for explaining a second form of the correction process S9, and FIG. 7A is a diagram for explaining the correction process S9 taking as an example the correction to the predicted position 221 at the current time shown in FIG. . FIG. 7B is a diagram showing a change in the spatial distribution of the predicted position due to the correction.

  In the present embodiment, the predicted position correcting unit 53 changes the direction of the moving route from the predicted position at the previous time to the target position, and sets the end point of the moving route as the corrected predicted position. Specifically, the predicted position correction unit 53 performs the detection of the correction target, the calculation of the straight line 230, the calculation of the contact point 231 and the calculation of the contour straight line of the obstacle region 42a in the same manner as in the first embodiment. Further, a distance L0 from the predicted position 220 of the previous time to the predicted position 221 of the current time to be corrected, and a distance L1 from the predicted position 220 of the previous time to the contact point 231 are calculated.

When the perpendicular 300 from the predicted position 220 of the previous time to the contour straight line is derived and the distance from the predicted position 220 to the intersection 301 of the contour straight line and the perpendicular 300 is represented by L2, the angle φ0 formed by the perpendicular 300 and the straight line 230 is φ0 = cos ⁻¹ {L2 / L1}. Considering a straight line 302 connecting the predicted position 220 with the intersection of the circle with the radius L0 centered on the predicted position 220 and the contour straight line, which is located in the same direction as the contact point 231 when viewed from the intersection 301, An angle φ1 formed by the straight line 302 and the perpendicular 300 is given by φ1 = cos ⁻¹ {L1 · cos φ0 / L0}.

  The predicted position correcting unit 53 randomly sets an acute angle Δ (where Δ ≧ 0), and a straight line 302 is positioned when a straight line 303 passing through the predicted position 220 and forming an angle of (φ1 + Δ) with the vertical line 300 is viewed from the vertical line 300. Derived on the side to do. Then, a point located on the straight line 303 at a distance L0 from the predicted position 220, that is, the intersection of the circle with the radius L0 centering on the predicted position 220 and the straight line 303 is set as the corrected predicted position 304.

  When this correction is performed on all correction targets in the predicted position group 203, the predicted position group 203 is corrected to a predicted position group 305 as shown in FIG.

  In the second form of the correction process, the moving path of the moving object from the predicted position 220 at the previous time to the corrected predicted position 304 is the moving path from the predicted position 220 at the previous time to the predicted position 221 before the correction. The path length is equal. Similar to the first embodiment, the corrected predicted position is located outside the obstacle region on the predicted position side of the previous time and is in a range where the moving object can move, and is obtained by motion prediction at the previous time. This is preferable in that the length of the path can be moved at a high speed. And also in this 2nd form similarly to the 1st form, since the prediction position which was wasted conventionally is corrected to the position where a moving object can move, increasing a significant prediction position also in the vicinity of an obstacle Tracking accuracy can be improved in the vicinity of the obstacle.

  In addition, while changing the path length from the predicted position at the previous time to the initial predicted position to be corrected, the moving path of the moving object from the predicted position at the previous time by motion prediction is changed, and the moving path The configuration in which the end point is the predicted position after correction is possible in addition to the first and second modes of the correction processing described above. For example, the direction of the movement path from the predicted position 220 is changed as in the first form, and the movement path is refracted as in the second form when hitting the obstacle area in the changed direction. Exists. Further, the angle Δ is randomly set in a range of 0 ° to less than 90 °, but the method of generating random numbers is not limited to uniform random numbers, and may have a non-uniform distribution such as a normal distribution. Good. For example, in the first embodiment, the angle (reflection angle) formed between the path from the contact point 231 to the predicted position 241 and the normal to the contour line is the same as the incident angle φ from the predicted position 220 to the contact point 231. By using a random number having a distribution peak in the direction, correction can be made in consideration of conservation of momentum of the moving object in the direction along the contour straight line. In addition to this, the upper limit of Δ and the method of generating random numbers can be changed according to the speed of the moving object. For example, in the second embodiment, the higher the speed of the moving object, the more difficult it is for the moving object to change direction, so the upper limit of Δ may be lowered.

[Third form of correction processing]
FIG. 8 is a schematic diagram for explaining a third form of the correction process S9, and FIG. 8A is a diagram for explaining the correction process S9 taking as an example the correction to the predicted position 221 at the current time shown in FIG. . FIG. 8B is a diagram showing a change in the spatial distribution of the predicted position due to the correction.

  In the third embodiment, the predicted position correcting unit 53 is a point of a part that can be seen from the predicted position at the previous time on the boundary line of the failure area, and the point that is the shortest distance from the initial predicted position for the current time. The corrected predicted position is set in a region having a size equal to or smaller than the distribution range of the predicted position at the immediately preceding time with the nearest point as the center. Specifically, the predicted position correction unit 53 performs the detection of the correction target, the calculation of the straight line 230, the calculation of the contact point 231 and the calculation of the contour line of the obstacle region 42a in the same manner as in the first and second embodiments. Do. Further, a perpendicular 400 from the initial predicted position 221 at the current time to the contour straight line is derived to obtain the intersection 401 of the contour straight line and the perpendicular 400 as the nearest point, and within the circle of a predetermined radius from the intersection 401, the obstacle region 42a The predicted position 402 after correction is randomly set in the area outside the area. The predetermined radius may be set to about half the radius r used for setting the predicted position by the predicted position setting unit 52.

  Thus, the predicted position can be corrected within a range in which the moving object can move by being located outside the obstacle region on the predicted position side of the previous time by simple processing.

  Also in this case, since the predicted position correcting unit 53 corrects the predicted position that was previously wasted to a position where the moving object can move, the predicted position can be increased even in the vicinity of the obstacle. Tracking accuracy can be improved in the vicinity.

  In addition, when the initial predicted position is set across the obstacle area 42a and outside of the obstacle area 42a, the outline of the obstacle area can be seen from the predicted position 220 at the previous time, that is, a position where the moving object can reach through the straight path. There may also be situations where the normal 400 cannot be drawn. In this case, the predicted position correction unit 53 can be seen from the predicted position at the previous time in the outline of the failure area and the shortest distance from the initial predicted position at the current time by a process different from the process for obtaining the perpendicular 400. Is determined as a point to replace the intersection 401, and a corrected predicted position is set in a circle having a predetermined radius with this point as the center.

[Fourth Modification Process]
In the first to third forms of the correction processing, unique associations are managed between the predicted positions at each time, and the predicted position setting unit 52 starts new motions based on the movements of the past predicted positions corresponding to each other. Set the predicted position. Therefore, in the first to third embodiments, the predicted position correcting unit 53 corrects the predicted position at a position before the obstacle area as viewed from the predicted position one hour before, and the first and second forms are one hour before. The movement path length from the predicted position to the predicted position at the current time is corrected so that it does not change before and after the correction.

  FIG. 9 is a schematic plan view showing an example of the object position and the predicted position of the current time in the fourth to sixth modes of the correction process described below, and from time (t-3) to time (t− The change in the object position leading to 1) and the predicted position at the current time t are shown. As in FIG. 5, in FIG. 9, “◯” indicates a predicted position, and “Δ” indicates an object position. In the fourth to sixth embodiments, the predicted position setting unit 52 sets P predicted positions one hour before in a predetermined range centered on the object position 502 one time before, and each predicted one time before The predicted position 510 of the current time is set by moving the position according to the movement of the object position. Incidentally, this method is equivalent to predicting the object position at the current time according to the movement of the object position and then setting the predicted position at the current time within a predetermined range centered on the position.

  In this predicted position setting method, the predicted position correcting unit 53 according to the fourth form of the correction process prevents the movement path length from the object position 502 one time before to each predicted position 510 at the current time before and after the correction. Make corrections. FIG. 10 is a schematic plan view for explaining examples of the fourth to sixth modes of the correction process S9. FIG. 10A is a diagram relating to the fourth embodiment. This figure is a diagram for explaining the correction processing S9 by taking the correction to the predicted position 521 at the current time shown in FIG. 9 as an example. The position one hour before as the starting point of the movement path is replaced with the predicted position by the object position 502. Except for this, it is basically the same as the first embodiment described with reference to FIG.

  The predicted position correcting unit 53 detects a contact point 531 between the straight line 530 from the object position 502 one time before to the predicted position 521 to be corrected and the obstacle region 42a, and approximates the boundary of the obstacle region near the contact point 531. The angle θ between the contour straight line and the straight line 530 is calculated. The predicted position correction unit 53 randomly determines an angle Δ within an acute angle range, rotates the predicted position 521 by (θ + Δ) around the contact point 531, and moves the predicted position 521 to an area outside the obstacle area 42 a on the object position 502 side. Correct in. As a result, the predicted position 521 is corrected to the predicted position 532.

  Also in this case, since the predicted position correcting unit 53 corrects the predicted position that was previously wasted to a position where the moving object can move, the predicted position can be increased even in the vicinity of the obstacle. Tracking accuracy can be improved in the vicinity.

[Fifth form of correction processing]
FIG. 10B is a schematic plan view for explaining an example of the fifth form of the correction process S9. This figure is a diagram for explaining the correction processing S9 by taking the correction to the predicted position 521 at the current time shown in FIG. 9 as an example. The position one hour before as the starting point of the movement path is replaced with the predicted position by the object position 502. Except for this, it is basically the same as the second embodiment described with reference to FIG.

  The predicted position correcting unit 53 is a corrected circle that is rotated by an angle (φ1 + Δ−φ0) from the predicted position 521 on a circle having a radius L0 from the object position 502 one time before the predicted position 521 to be corrected. A predicted position 540 is calculated.

  Also in this case, since the predicted position correcting unit 53 corrects the predicted position that was previously wasted to a position where the moving object can move, the predicted position can be increased even in the vicinity of the obstacle. Tracking accuracy can be improved in the vicinity.

[Sixth form of correction processing]
FIG. 10C is a schematic plan view for explaining an example of the sixth form of the correction process S9. This figure is a diagram for explaining the correction processing S9 by taking the correction to the predicted position 521 at the current time shown in FIG. 9 as an example. The position one hour before as the starting point of the movement path is replaced with the predicted position by the object position 502. Except for this, it is basically the same as the third embodiment described with reference to FIG.

  The predicted position correcting unit 53 derives a perpendicular line from the newly set predicted position 521 to the contour straight line, obtains the intersection 550 of the contour straight line and the perpendicular as the nearest point, and is within a circle with a predetermined radius from the intersection 550 and has a fault. The corrected predicted position 551 is randomly set in a region outside the region 42a.

  Also in this case, since the predicted position correcting unit 53 corrects the predicted position that was previously wasted to a position where the moving object can move, the predicted position can be increased even in the vicinity of the obstacle. Tracking accuracy can be improved in the vicinity.

  DESCRIPTION OF SYMBOLS 1 Moving object tracking apparatus, 2 Imaging part, 3 Setting input part, 4 Storage part, 5 Control part, 6 Output part, 40 3D model, 41 Camera parameter, 42 Obstacle map, 42a Obstacle area, 43 Predicted position, 44 Object position, 50 Obstacle map creation unit, 51 Changed pixel extraction unit, 52 Prediction position setting unit, 53 Prediction position correction unit, 54 Object region estimation unit, 55 Likelihood calculation unit, 56 Object position calculation unit, 57 Abnormality determination unit , 100 Reference plane, 101 Camera position, 102 Obstacle model, 103 Moving object model, 200 to 203, 510 Predicted position group, 210 to 212, 500 to 502 Object position, 220 to 223, 241, 304, 402, 521 532, 540, 551 predicted position, 231, 531 contact point, 401, 550 nearest point.

Claims (5)

A moving object tracking device for tracking an object moving in the space using a time-series image obtained by photographing a predetermined space,
A storage unit that stores in advance the obstacle region that prevents the movement in the space, and stores information on the past position of the object in the past from the attention time;
A position prediction unit that performs motion prediction of the object using the past position and predicts a plurality of movement destination candidates of the object at the time of interest;
An object position determination unit that determines a movement destination of the object at the attention time based on an image feature of the object included in a corresponding portion with each of the movement destination candidates in the image at the attention time;
Have
The position prediction unit corrects an invalid movement destination candidate in which the failure area exists between the movement destination candidates and the past position that is the movement source to a position closer to the movement source than the failure area. Having a correction part,
A moving object tracking device. The moving object tracking device according to claim 1,
The correction unit transforms the route from the source to the invalid destination candidate while maintaining the route length, and corrects the invalid destination candidate to the end point of the transformed route. Feature moving object tracking device. The moving object tracking device according to claim 2,
The moving object tracking device, wherein the correction unit performs the deformation by changing a direction of the route at a point where the route to the invalid destination candidate reaches the obstacle area. The moving object tracking device according to claim 2,
The moving object tracking device, wherein the correction unit performs the deformation by changing a direction of a route to the invalid movement destination candidate at the movement source. The moving object tracking device according to claim 1,
The correction unit calculates a nearest point that is at the shortest distance from the invalid destination candidate on the border on the source side in the obstacle area, and the invalid destination candidate is a position within a predetermined distance from the nearest point. A moving object tracking device characterized by:

Images