JP2012159957A - Moving object tracking device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve such a problem that, in a device for tracking a moving object in a monitoring space based on a plurality of prediction positions generated for every interval time, when uniformly removing the prediction positions set in an obstacle region in a monitoring space, a determination accuracy of the object position is deteriorated.SOLUTION: A penalty setting unit 50 sets penalty values corresponding to obstacle regions according to the heights of obstacles. An object position determining unit 53 calculates an evaluation value according to the amount of feature of the moving body extracted from an image corresponding to each prediction position and determines the object position based on the evaluation value. In the determination, the object position determining unit 53 performs a correction to decrease the evaluation value of each prediction position according to the penalty value at the position.

Description

  The present invention relates to a moving object tracking apparatus that tracks a moving object by image processing, and more particularly to a moving object tracking apparatus that tracks a moving object that moves in a space where an obstacle exists.

  Conventionally, in order to accurately track the position of an object, it has been performed to eliminate erroneous detection of the object in a region where the object cannot exist. For example, in the person position estimation method described in Patent Literature 1, information on the shooting space such as the size of a room or a desk / shelf is used to exclude a person position estimated in an area where a person cannot exist. That is, it has been determined whether or not a person can exist depending on the presence or absence of an obstacle.

JP 2007-233523 A

  However, when various obstacles such as walls, desks, and shelves are handled uniformly, there is a problem that it is difficult to accurately track the position of the person. Specifically, when a desk lower than a person is an obstacle, the person's position may protrude over the obstacle as the person passes near or leans on it, but is higher than the person When a wall or the like is an obstacle, the person position does not protrude on the obstacle. Moreover, even if it is just a shelf, if its height is different, a difference occurs in whether or not it can protrude from the shelf. If the area of the obstacle is defined as an area where no person can exist without considering the protrusion of the person to the area where the obstacle is placed, the error will occur due to a minute error in human position estimation. It may happen that the position of the person is not excluded around the area. In addition, when a person position is determined probabilistically from a large number of set person position hypotheses as in a particle filter, if a hypothesis of an area defined as a person cannot exist is excluded, Regardless of the amount of protrusion, the person position is determined only outside the area where the obstacle is placed.

  FIG. 8 is a diagram for explaining the above-described problem, and is a schematic diagram showing a person 90 leaning on an obstacle 91 (for example, a shelf). FIG. 8A is a diagram seen from the side, FIG. 8B is a diagram seen from above, and the original center of gravity position of the person is indicated by “x” in FIG. 8B. If the area of the obstacle 91 is defined as an area where the person 90 cannot exist as in the prior art, the barycentric position detected in the area of the obstacle 91 as shown in FIG. 8 is excluded as a false detection. Further, if the hypothesis set on the obstacle 91 is excluded by the particle filter according to the same definition, the person position is determined as a position away from the obstacle 91, and may be largely deviated from the center of gravity position suitable as the person position.

  As described above, according to the prior art, there is a problem that the tracking result in the vicinity of the obstacle may become unstable.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a moving object tracking device capable of tracking a moving object with high accuracy even in the vicinity of an obstacle.

  The moving object tracking device according to the present invention tracks a moving object moving in the space using a time-series image obtained by photographing a predetermined space, and the moving object may exist at the time of interest. A position prediction unit that predicts a plurality of candidate positions, and calculates the existence accuracy of the moving object at the candidate position according to the feature amount of the moving object extracted from the portion corresponding to the candidate position in the image at the time of interest An object position determination unit that determines the position of the moving object using the existence accuracy for each of the plurality of candidate positions; an obstacle region where an obstacle that prevents the movement in the space is present; and the obstacle A storage unit that stores height in advance, and a penalty setting unit that sets a value corresponding to the height of the obstacle in the obstacle area of the obstacle as a penalty value for the existence accuracy; It has the object position determination unit, when the penalty value is set to the candidate position, makes the determination by lowering the said presence probability of the candidate positions according to the penalty value.

  In the moving object tracking device according to another aspect of the invention, the penalty setting unit increases with the height of the obstacle and becomes maximum when the height of the obstacle is equal to or higher than the height of the moving object. Set the penalty value. In the moving object tracking device, the penalty setting unit may set the penalty value that increases stepwise at a predetermined joint height of the moving object.

  In still another moving object tracking device, the penalty setting unit decreases the penalty value by decreasing the closer to the periphery of the obstacle region in an inner edge region of a predetermined width along the periphery of the obstacle region in the obstacle region. Set. In the moving object tracking device, the penalty setting unit may set the width of the inner edge region according to the remaining height obtained by subtracting the height of the obstacle from the height of the moving object. .

  In the moving object tracking device according to another aspect of the invention, the penalty setting unit may include an inside of the obstacle region in an outer edge region set with a width according to the width of the moving object on the outside along the circumference of the obstacle region. The penalty value of a smaller value is set.

  According to the present invention, when determining the position of the moving object, the influence of the moving object protruding on the obstacle according to the height of the obstacle is included in the evaluation. Therefore, the determination accuracy of the position of the moving object in the vicinity of the obstacle is included. Can be improved.

It is a block block diagram of the moving object tracking apparatus which concerns on embodiment of this invention. It is a perspective view which shows an example of a three-dimensional model typically. The example of the penalty map of the monitoring space simulated by the three-dimensional model of FIG. 2 is shown. It is a schematic diagram explaining the penalty map in a failure area | region and an outer peripheral area | region. It is a typical graph which shows the aspect of the change of the maximum value of a penalty value according to the height of an obstruction. It is a general | schematic flowchart of the tracking process of the moving object tracking apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows notionally the example of a process by a position estimation part and an object position determination part. It is a schematic diagram for demonstrating the problem of a prior art.

  Hereinafter, a moving object tracking device 1 according to an embodiment of the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. The moving object tracking device 1 sets a room in which furniture is placed as a monitoring target space, and sets a person moving in the monitoring space as a tracking target (hereinafter referred to as a moving object). The moving object tracking device 1 processes a monitoring image obtained by imaging a monitoring space to track a moving object. Here, the area where the fixture is installed in the monitoring space is an obstacle area that hinders the movement of the moving object. Other examples of obstacle areas include walls, pillars and water heaters. The position of the obstacle area is known in advance without moving. The monitoring space is not limited to indoors but may be outdoor. In the case of outdoors, ponds, moats, etc., as well as houses, fences and various exteriors are obstacle areas. The moving object tracking device 1 is configured to perform tracking processing in consideration of the presence of an obstacle region, and improves tracking accuracy.

[Configuration of moving object tracking device]
FIG. 1 is a block diagram of a moving object tracking device 1 according to the embodiment. The moving object tracking device 1 includes an imaging unit 2, a setting input unit 3, a storage unit 4, a control unit 5, and an output unit 6. The imaging unit 2, the setting input unit 3, the storage unit 4, and the output unit 6 are connected to the control unit 5.

  The imaging unit 2 is a monitoring camera, is installed so as to face the monitoring space, and images the monitoring space at a predetermined time interval. The captured monitoring images of the monitoring space are sequentially output to the control unit 5. In order to grasp the position and movement of a person who moves along a reference plane such as the floor surface or the ground surface, the imaging unit 2 is basically installed at a height that allows a bird's-eye photography, for example, in this embodiment, The moving object tracking device 1 is used for indoor monitoring, and the imaging unit 2 is installed on the ceiling. The time interval at which the monitoring image is captured is 1/5 second, for example. Hereinafter, the unit of time recorded at the time interval of imaging is referred to as time.

  The setting input unit 3 is an input means for the administrator to perform various settings for the control unit 5, and is a user interface device such as a touch panel display, for example.

  The storage unit 4 is a storage device such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage unit 4 stores various programs and various data, and inputs / outputs such information to / from the control unit 5. Various data includes a three-dimensional model 40, a penalty map 41, a camera parameter 42, a predicted position 43, an object position 44, and a background image.

  The three-dimensional model 40 describes a state in which a moving object model that approximates a three-dimensional shape of a moving object and / or an obstacle model that approximates a three-dimensional shape of an obstacle is arranged in a virtual space that imitates a monitoring space (real space). It is data. An obstacle is an object or terrain that forms an obstacle area. In the present embodiment, the monitoring space and the virtual space are represented by a right-handed orthogonal coordinate system including the X, Y, and Z axes, and the vertically upward direction is set to the positive direction of the Z axis. A reference surface such as a floor surface is horizontal and is defined by an XY plane represented by Z = 0. The arrangement of the obstacle model in the virtual space is matched with the actual arrangement of the obstacle in the monitoring space. On the other hand, the arrangement of the moving object model can be set at an arbitrary position.

  The obstacle model is set for each obstacle in advance prior to the detection / tracking process of the moving object. The obstacle model is data on the shape and dimensions of obstacles actually existing in the monitoring space, and data on the arrangement of obstacles in the monitoring space. The obstacle area occupied by the obstacle relative to the reference plane and the obstacle And height information. The shape and dimensions of the obstacle can be obtained by actually measuring the obstacle placed in the monitoring space, and for example, three-dimensional data of a product provided by a fixture manufacturer or the like can be used. The arrangement of the obstacle can be obtained by actual measurement. For example, when designing the furniture layout of the room on the computer, the design data can be used. Data on the obstacle thus obtained is input to the control unit 5 from the setting input unit 3 or a connection interface (not shown) with an external device, and the control unit 5 stores the storage unit 4 based on the input data. The obstacle model is stored in.

  The moving object model is, for example, data describing a partial model representing a three-dimensional shape for each of a plurality of constituent parts constituting the moving object, and an arrangement relationship between the partial models. The moving object to be monitored by the moving object tracking device 1 is a standing person. In this embodiment, for example, a spheroid approximating the three-dimensional shape of the three parts of the person's head, torso, and leg is used. A moving object model stacked in the Z-axis direction is set. The height from the reference plane to the top of the head is represented by H, and the maximum width of the trunk (the trunk minor axis diameter) is represented by W. In this embodiment, in order to simplify the description, the height H and the width W are standard human sizes and are common to any moving object. The center of the head is set as the representative position of the moving object. Note that the moving object model may be further simplified and approximated by one spheroid. Data relating to the three-dimensional shape of the moving object model is stored in the storage unit 4 prior to the tracking process.

  The data stored as a moving object model in the storage unit 4 is the feature information of each moving object extracted from the monitoring image (for example, image features such as a color histogram) as reference information for individually characterizing the moving object to be tracked. including. The feature quantity of the moving object is extracted from the image area corresponding to the detected position and stored in the storage unit 4 when a moving object to be tracked is newly detected in the input image. Note that the data related to the shape of the moving object model is also used as reference information when extracting the image features of the moving object and is compared with the monitoring image.

  FIG. 2 is a perspective view schematically showing an example of the three-dimensional model 40. The reference plane 100 of the virtual space, the camera position 101, and the obstacle # 1 are obtained by discretizing the monitoring space into N × M × K position coordinates. , # 2 obstacle models 102 and 103 and a moving object model 104 are shown. Obstacle # 1 is lower than the person's height H and is approximately the person's waist height. Obstacle # 2 is higher than the height H of the person.

  The penalty map 41 is data in which a penalty value indicating a degree that a moving object cannot exist at each position in the monitoring space is associated with the position. The factor that obstructs the presence of the moving object is an obstacle, and a positive value is set as a penalty value at least in the obstacle area, and preferably a positive value is also set around the obstacle area. A larger penalty value is set in the obstacle area as the corresponding obstacle is higher.

  FIG. 3 shows an example of a penalty map 41 of the monitoring space simulated by the three-dimensional model 40 of FIG. The penalty map 41 has an X-axis and a Y-axis that represent a position corresponding to the reference plane, and a P-axis that represents a penalty value orthogonal to the X-axis and the Y-axis. A penalty value 200 due to obstacle # 1 is set in and around the obstacle area where obstacle # 1 exists, and a penalty value 201 due to obstacle # 2 is set around and in the obstacle area where obstacle # 2 exists. Is set. Since the obstacle # 2 is higher than the obstacle # 1, the penalty value 201 is set larger than the penalty value 200. The penalty value at a position away from the obstacle is zero. A one-dot chain line represents a position where the penalty value becomes zero.

  The camera parameter 42 includes information regarding projection conditions when the imaging unit 2 captures a monitoring image in which the monitoring space is projected. For example, external parameters such as the installation position and imaging direction of the imaging unit 2 in the actual monitoring space, focal length, angle of view, lens distortion, and other lens characteristics of the imaging unit 2, and internal parameters such as the number of pixels of the imaging element are included. These parameters obtained by actual measurement or the like are input in advance from the setting input unit 3 and stored in the storage unit 4. The three-dimensional model 40 can be projected onto the coordinate system of the monitoring image (imaging plane of the imaging unit 2; xy coordinate system) by a coordinate transformation formula in which the camera parameter 42 is applied to a known camera model such as a pinhole camera model.

  The predicted position 43 (hypothesis) is information regarding the predicted value (predicted position) of the position of the moving object at each time. In order to determine the position (object position) of a moving object stochastically, a large number of predicted positions are set for each moving object (the number is represented by α, for example, 200 per moving object). Specifically, the predicted position 43 is time-series data in which an identifier of a moving object is associated with an index (0 to α-1) of a predicted position at each time and its position coordinates (XYZ coordinate system).

  The object position 44 is information regarding the position of the moving object at each time, and specifically is time-series data in which the identifier of the moving object is associated with the position coordinates (XYZ coordinate system).

  Note that the processing can be speeded up as a configuration in which the predicted position 43 and the object position 44 are specified by the horizontal coordinate (XY coordinate system) of the monitoring space. In the present embodiment, the configuration will be described as an example for easy understanding.

  The control unit 5 is configured using an arithmetic device such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit), and reads and executes a program from the storage unit 4, and a penalty setting unit 50. , Function as a change pixel extraction unit 51, position prediction unit 52, object position determination unit 53, and abnormality determination unit 54.

  The penalty setting unit 50 creates a penalty map 41 with reference to the obstacle model and the moving object model, and stores the created penalty map 41 in the storage unit 4. Specifically, the penalty setting unit 50 sets a penalty value for an evaluation value calculated corresponding to a predicted position of a moving object, which will be described later, at each position on the reference plane. The evaluation value is calculated based on the image feature of the monitoring image, and represents the degree of likelihood that there is a moving object at each position in the monitoring space, whereas the penalty value is calculated based on the presence of an obstacle. This represents the degree to which no moving object can exist at each position in the monitoring space.

  The penalty setting unit 50 sets a penalty value at least in an obstacle area where an obstacle exists. This penalty value corresponds to the ease with which a moving object protrudes onto each obstacle, and a larger value is set for a higher obstacle. The maximum penalty value for each obstacle is referred to as a unique penalty value. The inherent penalty value is set so as to increase according to the height of the obstacle. This unique penalty value can be defined simply according to the height of the obstacle, and also defined in consideration of the relationship between the height of the obstacle and the height of the moving object, as in this embodiment. You can also.

  Specifically, the penalty setting unit 50 sets the maximum penalty value (inherent penalty value) Ph in the obstacle area of each obstacle to a lower limit value Pmin (likelihood described later) when the obstacle height h is zero. = 0.0), and when the height h of the obstacle is equal to or higher than the height H of the moving object (h ≧ H), a likelihood upper limit value Pmax (for example, 1.0) described later is set, When the height h of the obstacle is in the range of 0 ≦ h <H, Ph is increased according to h. For example, when 0 ≦ h <H, Ph can be set to a proportional function of h (FIG. 5A). By setting in this way, for example, when the obstacle is a hip-height table rather than a person's shoulder-height shelf, the object position is more easily determined within the obstacle region, The object position can be determined in accordance with the actual situation. Further, for example, an upper limit value Pmax is set for an obstacle region such as a high shelf, wall, or pillar that exceeds the height of a person, and the object position that protrudes from the obstacle region is not determined, so that the correct object position can be determined.

  Further, the penalty setting unit 50 sets the penalty value of the edge part smaller than the center of the obstacle in the obstacle region, corresponding to the fact that the protrusion of the moving object is likely to occur at the edge part of the obstacle. Specifically, the penalty setting unit 50 determines the uniqueness of the obstacle corresponding to the obstacle area as a penalty value in the inner area excluding the edge (inner edge area) set along the circumference of the obstacle area. A penalty value Ph is set, and in the inner edge region, a penalty value that decreases from the inherent penalty value as the circumference of the failure region is approached is set.

  The penalty setting unit 50 sets the width u of the edge according to the remaining height obtained by subtracting the height of the obstacle from the height of the moving object. That is, the width u of the edge of each obstacle area is set wider as the height of the obstacle corresponding to the obstacle area is lower than the height of the moving object. The width u of the edge set in this way corresponds to the size of the portion where the human body can protrude over the obstacle. For example, when the obstacle is a shelf at the height of a person's shoulder, the upper part can protrude from the shoulder, but when the obstacle is a table at the waist level, the entire upper body can protrude. This setting makes it possible to determine the actual object position.

  Further, the penalty setting unit 50 can be configured to set a positive penalty value also in the outer peripheral area (outer edge area) of an obstacle having a width corresponding to the width W of the moving object. This configuration is based on the fact that moving objects such as humans and animals generally choose less movement that hits an obstacle. The penalty value of the outer peripheral area is basically set lower than that in the obstacle area in order to achieve consistency with allowing the moving object to protrude into the obstacle area.

  For example, the penalty setting unit 50 sets the width of the outer peripheral area to 1.5 W, sets the penalty value p at a distance of 1.5 W outward from the boundary of the obstacle area to 0, and moves from the position to the inner side of the outer peripheral area. Increase the penalty value toward. In the present embodiment, the penalty setting unit 50 sets an edge on the inner side of the failure area, and, for example, a penalty value is set at a distance of 1.5 W from the failure area to the outside according to the positions in the outer peripheral area and the edge. The value is continuously changed from the value 0 at 0 to the value Ph at the position of the distance u inward. If the edge width u set based on the difference between the height of the moving object and the height of the obstacle is larger than the width of the obstacle area, the penalty setting unit 50 has the maximum penalty value at the center of the obstacle area. Correction is made to the change function of the penalty value at the edge or the edge and the outer peripheral region so as to be the value Ph.

  By using the penalty value as described above, it is possible to perform processing that allows the position of the moving object to exist within the obstacle area in a limited manner, instead of the process that uniformly eliminates the existence of the moving object in the obstacle area. Further, the penalty value in the outer peripheral region has an effect of making it difficult for the position of the moving object to approach the obstacle region.

  FIG. 4 is a schematic diagram for explaining a penalty map 41 in the failure area and the outer peripheral area. In the figure, the change in penalty value along the line perpendicular to the boundary of the obstacle region is indicated by a bold line, with the horizontal axis indicating the distance d with the boundary of the obstacle region as the origin and the vertical axis indicating the penalty value p. Show. On the d-axis, the left side from the origin is the outer peripheral area and the right side is the obstacle area. FIG. 4 shows four cases where the heights h of the obstacles are different, and the cross section of the obstacle is indicated by oblique lines in the figure. In addition, the shape of a person who is a moving object is indicated by a dotted line. The z-axis on the right side of the figure is a coordinate axis related to the heights of these obstacles and moving objects. As described above, the maximum value Pmax of the penalty value is set corresponding to the height H of the moving object, and the inherent penalty value Ph of each obstacle is set corresponding to the height h of the obstacle. Pmax and Ph on the axis and H and h on the z-axis are shown aligned at the same vertical position.

  In FIG. 4, the height of the obstacle increases in the order from (a) to (d), and the obstacle in (d) is higher than H. Correspondingly, the maximum value Ph (flat portion) of the penalty value increases in the order from (a) to (d) and becomes the upper limit value Pmax in (d). In addition, the change of Ph according to the height of the obstacle in FIG. 4 is according to the example shown in FIG. The width u of the edge is narrowed in the order from (a) to (d) according to the height at which the moving object exceeds the obstacle, and is set to 0 at (d). Comparing (a) to (d), the penalty values at the positions corresponding to each other increase in the order of (a), (b), (c), and (d) in both the inner region and the edge.

  The penalty value is set based on an attenuation curve that monotonously decreases outward from the position of the distance u to the inside of the obstacle area, and becomes 0 at a distance of 1.5 W in a direction away from the boundary of the obstacle area. In order to make it difficult to determine the position where the moving object model contacts or interferes with the obstacle model as the object position, the width of the outer peripheral region needs to be at least W / 2. On the other hand, the width of the outer peripheral region should be about 2 W as an upper limit so that the object position away from the obstacle is not easily determined. In order to reflect the fact that the higher the obstacle is, the more the moving object protrudes onto the obstacle and the more easily the object position is moved outside the obstacle area, the penalty values are attenuated at the edge and outer peripheral area. It is preferable to set the attenuation rate of the curve higher as the width u becomes smaller. In the example shown in FIG. 4, the attenuation curve is set by an exponential function.

  FIG. 5 is a schematic graph showing an aspect of a change in the specific penalty value Ph according to the height h of the obstacle. In FIG. 5, the horizontal axis is the height h of the obstacle, and the vertical axis is the inherent penalty value Ph. FIG. 5A shows a configuration in which 0 ≦ h <H and Ph is represented by a proportional function of h, and the above-described configuration has been described as an example using this function.

  In another configuration example, the function representing Ph when 0 ≦ h <H is set to increase in a stepwise manner at the height of a predetermined joint portion included in the moving object. The step function of FIG. 5B shows this configuration. Specifically, knees, hip joints (inseam), hips, armpits (shoulders), necks, etc. are defined as main joint parts in the human body, and the maximum penalty value Ph is set step by step for each joint part height. Set. The height of the joint can be preset according to the standard physique of the human body. For example, the height of the joint part is stored in the storage unit 4 as a part of the moving object model, and the penalty setting unit 50 compares the height h of the obstacle with the height of the joint part read from the storage unit 4, Ph for the obstacle is set. The correspondence between the height of the joint and Ph is also stored in the storage unit 4 in advance, and the penalty setting unit 50 reads this and sets Ph corresponding to the height of the obstacle. Alternatively, a Ph function that changes stepwise according to the joint height may be set in advance, and the penalty setting unit 50 may calculate the function and set Ph. The flexion of the body at the joint is the main cause of the human body protruding above the obstacle, and which joint portion is above the obstacle affects the ease with which the human body protrudes over the obstacle and the amount of protrusion. Therefore, the Ph setting in consideration of the joint in this way is suitable for improving the determination accuracy of the position of the moving object. At that time, it is ideal to consider the range of motion for various joints, but the model becomes complicated. In this respect, the above-described configuration taking into account the heights of the main joints can easily express the influence of bending of the body with a small number of parameters and reflect it in the determination of the object position.

  Yet another configuration example is a function in which the inclination is continuously changed according to the height of the main joint. FIG. 5C is an example thereof, which is a function obtained by curve approximation of the step function of FIG.

  The change pixel extraction unit 51 extracts a change pixel from the monitoring image newly input from the imaging unit 2, and outputs information about the extracted change pixel to the object position determination unit 53. Therefore, the change pixel extraction unit 51 performs a difference process between the newly input monitoring image and the background image, and extracts a pixel whose difference is equal to or greater than a preset difference threshold as a change pixel. The change pixel is an image feature of the moving object extracted by comparing with the monitoring image using the background image as reference information. Note that the changed pixels may be extracted by a correlation process between the newly input monitoring image and the background image instead of the difference process, or the background model is learned instead of the background image and the difference process with the background model is performed. The change pixel may be extracted by.

  The position prediction unit 52 (hypothesis setting unit) predicts the movement of the moving object using the past position information of the moving object (the object position 44 or the predicted position 43), and the candidate position of the movement destination at the current time of the moving object A plurality of predicted positions are obtained.

  First, the position prediction unit 52 performs motion prediction from the object position of each moving object determined in the past or the predicted position of each moving object set in the past, and the moving object at the time when a new monitoring image is input. Is predicted, and the predicted position (predicted position) is output to the object position determination unit 53. As described above, α predicted positions are set for each moving object. A method of determining a position (object position) of a moving object in a probabilistic manner by sequentially setting a large number of predicted positions in this manner is called a particle filter or the like, and the set predicted position is called a hypothesis or the like. The predicted position can be set in the xy coordinate system of the monitoring image, but in the present embodiment, it is set in the XYZ coordinate system of the monitoring space. Motion prediction is performed by applying a predetermined motion model or applying a predetermined prediction filter to past position data.

  The object position determination unit 53 receives a monitoring image and a candidate position where the target object can exist in the monitoring space, and corresponds to the feature amount of the moving object extracted from the monitoring image at the current time corresponding to each candidate position. An evaluation value is calculated. Then, the position of the moving object is determined based on the evaluation value of each candidate position, and the determination result is stored in the storage unit 4. The candidate position is a predicted position input from the position prediction unit 52.

  The object position determination unit 53 calculates a probability that the moving object exists at each predicted position of the moving object by taking a penalty value into account, and the moving object from the evaluation of each predicted position. An object position calculation unit 531 for determining one object position for each is configured. Hereinafter, the existence accuracy calculation unit 530 and the object position calculation unit 531 will be described in detail.

  The existence accuracy calculation unit 530 extracts the feature amount of the moving object from the region corresponding to the predicted position of each moving object in the monitoring image, and the likelihood as the object position of the predicted position according to the degree of feature amount extraction. As the existence accuracy and output to the object position calculation unit 531. At this time, the existence accuracy calculation unit 530 refers to the penalty map 41 and performs correction to reduce the likelihood of the predicted position by the penalty value when a (positive) penalty value is set at the predicted position.

If the XY coordinate of the predicted position is (X, Y), the likelihood before correction for the predicted position is L0 (X, Y), and the penalty value of the position is p (X, Y), the corrected position for the predicted position is corrected. The likelihood L (X, Y) of is calculated by subtracting the penalty value from the likelihood before correction as in the following equation (1).
L (X, Y) = L0 (X, Y) -p (X, Y) (1)

  Incidentally, the above-described penalty value range [0, 1] corresponds to the likelihood range [0, 1]. When the range of penalty values is not normalized to [0, 1], the subtraction result on the right side of equation (1) can be less than 0. In this case, L (X, Y) is defined as 0.

Further, the likelihood L (X, Y) after correction is obtained by subtracting the penalty value from the upper limit value Pmax (here, normalized to “1”) of the penalty value as shown in the following equation (2). The calculated value may be calculated as a correction coefficient by multiplying the likelihood before correction.
L (X, Y) = L0 (X, Y) × {1-p (X, Y)} (2)

  Here, the penalty value is defined in the range of [0, Pmax] as shown in FIG. On the other hand, the penalty value itself is allowed to be set to exceed Pmax, and L (X, Y) is defined as 0 when p (X, Y)> Pmax at the correction stage shown by the equation (2). You may decide.

The likelihood L0 before correction is calculated as follows. The existence accuracy calculation unit 530 generates the three-dimensional model 40 in which the obstacle model is arranged at a predetermined position and the moving object model is virtually arranged at the predicted position. Then, the camera parameter 42 is used to project the three-dimensional model 40 onto the imaging surface of the imaging unit 2 to obtain a non-hidden region of the moving object model, and the likelihood is calculated by any of the following methods (1) to (4). calculate. Here, the non-hidden region is a region in which the moving object model of interest appears as an image on the monitoring image without being hidden by an obstacle or another moving object.
(1) A non-obscured region is superimposed on the changed pixel extracted by the changed pixel extraction unit 51, and a ratio (inclusion degree) in which the changed pixel is included in the non-hidden region is obtained. Inclusion degree increases as the predicted position is closer to the position where the moving object actually exists, and tends to decrease as the distance increases. Therefore, the likelihood corresponding to the high degree of inclusion is calculated.
(2) Extract an edge from a portion corresponding to the contour of the non-hidden region in the monitoring image. The closer the predicted position is to the position where the moving object actually exists, the more the edge extraction degree (for example, the sum of the extracted edge strengths) becomes higher because the contour of the non-hidden region matches the edge position. The degree tends to be low. Therefore, the likelihood corresponding to the degree of edge extraction is calculated.
(3) The feature amount extracted from the non-hidden region at the past object position of each moving object is stored in the storage unit 4 as reference information of the moving object. As the predicted position is closer to the position where the moving object actually exists, the feature amount of the background and other moving objects will not be mixed, so the similarity between the feature amount extracted from the non-hidden region and the reference information becomes high, The similarity tends to decrease as the distance increases. Therefore, the feature amount in the non-hidden region is extracted from the monitoring image, and the similarity between the extracted feature amount and the reference information is calculated as the likelihood. As the feature amount here, for example, various image feature amounts such as an edge distribution, a color histogram, or both of them can be used.
(4) The likelihood may be calculated according to a weighted addition value of a plurality of degrees of the above-described inclusion degree, edge extraction degree, and similarity degree.

  Instead of using the likelihood as an evaluation value, the above-described inclusion degree, edge extraction degree, similarity degree, or a combination of two or more of these can be used as the evaluation value.

The object position calculation unit 531 determines the position (object position) of the moving object from each predicted position of the moving object and the likelihood calculated for each predicted position, and stores the determination result in the storage unit 4 for each moving object. Accumulate in time series. When all the likelihoods are less than a predetermined lower limit (likelihood lower limit), it is determined that there is no object position, that is, disappeared. The following (1) to (3) are examples of the object position calculation method.
(1) For each moving object, a weighted average value of predicted positions weighted by likelihood is calculated, and this is set as the object position of the moving object.
(2) For each moving object, a predicted position where the maximum likelihood is calculated is obtained and set as the object position.
(3) For each moving object, an average value of predicted positions at which likelihoods equal to or higher than a preset likelihood threshold value are calculated is set as the object position. Here, the likelihood threshold> the likelihood lower limit value.

  The abnormality determination unit 54 refers to the time-series object positions accumulated in the storage unit 4, determines that a suspicious movement that stays for a long time or a suspicious movement that deviates from the normal flow line is abnormal, and determines the abnormality. And output an abnormal signal to the output unit 6.

  The output unit 6 includes a sound output means for outputting a warning sound, a display means for displaying a monitoring image determined to be abnormal, or a communication means for transmitting to a security company center device via a communication line. When an abnormality signal is input from the unit 54, the fact that an abnormality has occurred is output to the outside.

[Operation of moving object tracking device]
Next, the tracking operation of the moving object tracking device 1 will be described. FIG. 6 is a schematic flowchart of the tracking process of the moving object tracking device 1.

  When the tracking process is started, the control unit 5 confirms whether the penalty map 41 is created (S1), and if not created, the penalty setting unit 50 creates the penalty map 41 and stores it in the storage unit 4 ( S2).

  The moving object tracking device 1 starts a substantial tracking process (steps S3 to S16) for determining the object position of the moving object for each monitoring image in a state where the penalty map 41 is stored in the storage unit 4.

  The control unit 5 receives a monitoring image every time the imaging unit 2 images the monitoring space (S3). Hereinafter, the time when the latest monitoring image is input is called the current time, and the latest monitoring image is called the current image.

  The current image is compared with the background image by the change pixel extraction unit 51, and the change pixel extraction unit 51 extracts the change pixel (S4). Here, the isolated change pixel is excluded from the extraction result as being caused by noise. Immediately after the start of the operation without a background image, the current image is stored in the storage unit 4 as a background image, and there is no change pixel for convenience.

  Further, the position prediction unit 52 sets α predicted positions based on the motion prediction for each moving object being tracked (S5). Note that the predicted positions of the moving object determined to be a new appearance in step S16, which will be described later, cannot be predicted, and therefore α predicted positions are set in a wider range centering on the appearance position. Further, the predicted position of the moving object determined to be lost in step S16 described later is deleted.

  If the changed pixel is not extracted in step S4 and the predicted position is not set in step S5 (there is no moving object being tracked) (if “YES” in S6), the control unit 5 performs the step. Returning to S3, the input of the next monitoring image is awaited.

  On the other hand, if “NO” in the step S6, the processes of the steps S7 to S16 are performed. The control unit 5 sequentially sets each moving object being tracked as a target object (S7). Subsequently, the control unit 5 sequentially sets each predicted position of the target object as the target position (S8). However, the predicted position outside the field of view of the monitoring image is excluded from the target position setting targets, the object region at the predicted position is not estimated, and the likelihood is set to zero.

  The existence accuracy calculation unit 530 arranges the moving object model at the position of interest in the virtual space, and generates the three-dimensional model 40 in which the moving object model and the obstacle model are arranged. Then, the camera parameter 42 is used to project the three-dimensional model 40 onto the imaging surface of the imaging unit 2 to obtain a non-hiding area of the moving object model, which is estimated as a non-hiding area of the moving object in real space (S9). . Then, a pre-correction likelihood L0 corresponding to the target position is calculated based on the estimated non-hidden region (S10).

  The existence accuracy calculation unit 530 refers to the penalty map 41 and acquires the penalty value P at the position of interest. When the penalty value P is set at the position of interest (when “YES” in S11), the existence accuracy calculation unit 530 performs likelihood correction using the above-described equation (1) and the like and the corrected likelihood L is calculated (S12). When the penalty value P is not set at the attention position (P = 0) (“NO” in S11), the likelihood correction is not performed, and the attention position L0 is used as it is. The likelihood L is determined.

  The control part 5 repeats step S8-S12, when the prediction position in which the likelihood is not calculated remains (in the case of "NO" in S13). When the likelihood is calculated for all α predicted positions (in the case of “YES” in S13), the object position calculation unit 531 calculates each predicted position (X, Y) of the target object and each of the predicted positions. The object position of the object of interest is calculated using the obtained likelihood L (X, Y) (S14). The object position calculated for the current time is added in association with the object position of the object of interest stored in the storage unit 4 one hour before. In the case of a newly appearing moving object, a new identifier is assigned and registered. If the likelihood at all predicted positions is less than the lower limit of likelihood, it is determined that there is no object position.

  When an unprocessed moving object remains ("NO" in S15), the control unit 5 repeats the process of determining the object position for the moving object (steps S7 to S14). On the other hand, when the determination of the object positions for all the moving objects is completed (in the case of “YES” in S15), the appearance and disappearance of the objects are determined (S16). Specifically, the object position calculation unit 531 combines the object regions estimated for each object position, detects a change pixel outside the combination region among the change pixels extracted by the change pixel extraction unit 51, Among the detected change pixels, adjacent change pixels are labeled. If the label is of a size that can be regarded as a moving object, a new appearance is recorded in the storage unit 4 together with the label position (appearance position). If there is a moving object without an object position, the fact that the moving object has disappeared is recorded in the storage unit 4. When the above processing is completed, the processing returns to step S3 in order to perform processing on the monitoring image at the next time.

  FIG. 7 is a schematic diagram conceptually showing a processing example by the position predicting unit 52 and the object position determining unit 53, and a moving object 71 (person) leaning on an obstacle 70 (shelf) as in FIG. It is about. FIG. 7A shows the distribution of the positions of the obstacle 70 (obstacle region 72) and α predicted positions 73 with respect to the moving object 71 on the horizontal plane (XY plane). Due to the moving object 71 leaning on the obstacle 70, the predicted position 73 is set not only outside the obstacle area 72 but also inside the obstacle area 72.

  FIG. 7B shows the shapes of the obstacle 70 and the moving object 71 on the vertical plane along the straight line 74 shown by the alternate long and short dash line in FIG. Furthermore, in the same figure, the distribution of the likelihood L0 of the predicted position 73 on the straight line 74, the penalty value P corresponding to the obstacle 70, and the distribution of the likelihood L after correction are shown in schematic graphs 75 to 77, respectively. Show. Note that the reference lines (levels with a value of “0”) in the L0 graph 75, the P graph 76, and the L graph 77 are matched with straight lines indicating the reference plane Z = 0. Further, the positive direction of L (and L0) is upward and the positive direction of P is downward, corresponding to the likelihood L after correction being subtracted from the penalty value P from the likelihood L0 before correction. It is set.

  As described above, the predicted position 73 is also set in the failure area 72, and thus the likelihood distribution graph 75 before correction has a peak in the failure area 72. The height of the obstacle 70 is such that the upper body of the moving object 71 can get on, and an inherent penalty value Ph smaller than 1 is set according to the height. In addition, an edge having a width corresponding to a difference obtained by subtracting the height of the obstacle 70 from the height of the moving object 71, that is, an amount by which the moving object 71 can get on the obstacle 70 is set in the obstacle region 72. The value P is attenuated by the inherent penalty value Ph set in the inner region of the obstacle region 72 at the edge. The peak of the likelihood distribution graph 77 corrected using the penalty value P moves to the boundary side of the failure region 72 from the peak before correction, but can remain in the failure region 72. That is, the moving object tracking device 1 according to the embodiment of the present invention can determine the object position in the obstacle area. In other words, in the conventional processing in which no moving object can exist in the obstacle area 72, the object position is rejected outside the obstacle area as described with reference to FIG. Even when this occurs, the moving object tracking device 1 can determine the object position with high accuracy.

  DESCRIPTION OF SYMBOLS 1 Moving object tracking device, 2 Imaging part, 3 Setting input part, 4 Storage part, 5 Control part, 6 Output part, 40 3D model, 41 Penalty map, 42 Camera parameter, 43, 73 Predicted position, 44 Object position, 50 penalty setting unit, 51 changed pixel extraction unit, 52 position prediction unit, 53 object position determination unit, 54 abnormality determination unit, 70 obstacle, 71 moving object, 72 obstacle region, 100 reference plane, 101 camera position, 102, 103 Obstacle model, 104 moving object model, 200, 201 penalty value, 530 existence accuracy calculation unit, 531 object position calculation unit.

Claims (6)

A moving object tracking device for tracking a moving object moving in the space using a time-series image of a predetermined space,
A position prediction unit that predicts a plurality of candidate positions where the moving object may exist at the time of interest;
The presence probability of the moving object at the candidate position is calculated according to the feature quantity of the moving object extracted from the portion corresponding to the candidate position in the image at the time of interest, and the existence for each of the plurality of candidate positions An object position determination unit that determines the position of the moving object using accuracy;
A storage unit for preliminarily storing an obstacle area where an obstacle that prevents the movement in the space exists and a height of the obstacle;
As a penalty value for the existence accuracy, a penalty setting unit that sets a value corresponding to the height of the obstacle in the obstacle area of the obstacle;
Have
The object position determination unit, when the penalty value is set in the candidate position, to perform the determination by reducing the existence accuracy of the candidate position according to the penalty value;
A moving object tracking device. The moving object tracking device according to claim 1,
The penalty setting unit sets the penalty value that increases with the height of the obstacle and is maximized when the height of the obstacle is equal to or higher than the height of the moving object. Object tracking device. The moving object tracking device according to claim 2,
The moving object tracking device, wherein the penalty setting unit sets the penalty value that increases stepwise at a predetermined joint height of the moving object. In the moving object tracking device according to any one of claims 1 to 3,
The penalty setting unit is configured to set the penalty value by decreasing the closer to the circumference of the obstacle area in the inner edge area having a predetermined width along the circumference of the obstacle area in the obstacle area. Object tracking device. The moving object tracking device according to claim 4,
The penalty setting unit sets the width of the inner edge region according to the remaining height obtained by subtracting the height of the obstacle from the height of the moving object. In the moving object tracking device according to any one of claims 1 to 5,
The penalty setting unit is configured to set the penalty value of a value smaller than that in the obstacle area in an outer edge area set with a width according to the width of the moving object on the outside along the circumference of the obstacle area; A moving object tracking device.

Images