JP6548683B2 - Object image estimation device and object image determination device - Google Patents

Description

  The present invention relates to an object image estimation device and an object image determination device for estimating an object visible region in which an image of an object in an image appears, and in particular to an object that can be concealed on the image by an installation object such as a fixture. The present invention relates to an apparatus for estimating a visible region.

  Extracting an image of a monitoring target such as a person from a monitoring image obtained by imaging the monitoring space, or determining or tracking the presence of the monitoring target based on the image feature of the image is performed. In the surveillance image, the entire image of the surveillance object can not always be observed, and a part of the surveillance image is often concealed by fixtures or columns in the surveillance space, or by another person or the like. This concealment changes image features such as observed size and color components of the monitored object, which causes detection failure and tracking failure.

  Therefore, by simulating concealment using a three-dimensional model, the area of the monitored object (object visible area) not concealed in the surveillance image is estimated, and the image of the object is derived from the image features of the object visible area. It is performed to prevent detection failure and tracking failure due to fluctuation of image features by determining the presence of

  In addition, since the processing load of the three-dimensional model in the simulation is large and the detection and tracking in real time are difficult, the processing load is reduced by the projection processing in advance.

  For example, in the object image determination apparatus described in Patent Documents 1 and 2, a three-dimensional model is projected in advance for each of an object such as a person and an installation object in the monitoring space, and an object model image and an installation object model image A correspondence between each and the floor coordinates, and a map of the floor position where concealment occurs due to the installation is generated and stored in the storage unit, and if the person position is the concealed position, an object corresponding to the person position The object visible region is determined by excluding the overlapping portion with the installation object model image from the object model image.

JP 2012-108574 A JP 2012-155595 A

  However, in the prior art, an object whose depth width can be ignored such as a standing person is a monitoring target object, and the concealing situation may be different between the front and the back like a fallen person (person in recumbent position) The estimation accuracy for the monitored object was reduced.

  Furthermore, in the prior art, since the model projection image for each installation and the correspondence for each floor coordinate are stored, it is necessary to increase the storage capacity according to the complexity and the size of the monitoring space. There has been a problem that the time required for pre-processing is increased when changing the layout of the image.

  That is, in the prior art, since the concealed position on the floor surface is associated with the two-dimensional installation object model image standing at the position, the concealed position for a complex installation having unevenness in the depth direction when viewed from the camera The model image of a different installation must be stored for each concealed position by setting a plurality of.

  Further, in the prior art, since the installation object model image is stored for each installation object, when the number of installation objects increases, it is necessary to increase the capacity for storing the installation object model image.

  Further, in the prior art, since the presence / absence of concealment and the correspondence relationship with the installation object model image are stored for each position in the surveillance space, the presence / absence of the concealment and the installation object model image We had to increase the capacity to store correspondences.

  The present invention has been made in view of the above problems, and is an object that can accurately estimate a non-hidden region of a monitored object that can take a plurality of postures, including a posture with a long depth such as a recumbent person An object of the present invention is to provide an image estimation device and an object image determination device capable of determining the presence of an image of a monitored object with high accuracy. Another object of the present invention is to provide an object image estimation device and an object image determination device in which the storage capacity required for the installation does not change even if the complexity and the size of the monitoring space increase.

  (1) The object image estimation apparatus according to the present invention is an object visual estimation area in which an image of an object appears in an image obtained by capturing a predetermined three-dimensional space by an imaging unit. An apparatus for estimating using a three-dimensional model, wherein horizontal distance and vertical distance, which is the length of an image obtained by orthogonally projecting the eye vector onto the horizontal plane and the vertical axis, as two types of projection distance of the eye vector of the photographing means Storage of the installation object distance stored as the installation object distance, in correspondence with each pixel of the image, the projection distance of the eye vector corresponding to the pixel to the installation object in the three-dimensional space Means, candidate postures of the object, and candidate setting means for setting candidate positions in the three-dimensional space in which the object may exist; Model image outputting means for outputting a model image obtained by projecting the image data onto the coordinate system of the image; and at least one designated type predetermined for the candidate posture among The model distance calculating means for calculating the projection distance of the line-of-sight vector as a model distance, and of the pixels constituting the model image of the solid model, the installation object distance of the designated type of the pixel is the specification of the solid model And a visible area estimation unit configured to estimate a pixel that is equal to or more than the model distance of a type as an object visible pixel.

  (2) In the object image estimation device according to (1), the object is a person, and the horizontal direction is the projection distance of the designated type when the candidate posture is set to the standing position. A distance can be set.

  (3) In the object image estimation device according to the above (1) and (2), the projection of the designated type with respect to the case where the object is a human and the candidate posture is set in a recumbent position The vertical distance may be set as the distance.

  (4) The object image determination apparatus according to the present invention is the object image estimation apparatus according to any one of the above (1) to (3), and the image features of the object visible pixels in the image. Object determination means for determining the presence of an image of an object.

  According to the present invention, it is possible to determine with high accuracy the presence of an image of a monitored object capable of taking a plurality of postures. Further, according to the present invention, the presence of the image of the object to be monitored can be determined at high speed and with high accuracy without increasing the storage capacity required for the installation even if the complexity and the size of the monitoring space increase.

FIG. 1 is a block diagram of a moving object tracking device according to an embodiment of the present invention. 1 is a schematic functional block diagram of a moving object tracking device according to an embodiment of the present invention. It is a functional block diagram of a model image output means. It is a schematic diagram of the monitoring space for demonstrating the concrete definition of the model distance according to a candidate attitude | position. It is a typical sectional view of surveillance space for explaining the significance of expressing the position of a solid model at the time of judging concealment by an installation thing with the distance along the plane perpendicular to the main axis of a solid model. It is a schematic diagram for demonstrating the specific example of installation object distance. It is a schematic diagram explaining the process example of a visible area estimation means in case a candidate attitude | position is standing position. It is a schematic diagram explaining the process example of a visible area estimation means in case a candidate attitude | position is recumbent. It is a schematic flowchart of a tracking process of a moving object tracking device according to an embodiment of the present invention. It is a schematic flowchart of a tracking process of a moving object tracking device according to an embodiment of the present invention. It is a schematic flowchart of an installation thing concealment estimation process.

  Hereinafter, a moving object tracking device 1 according to an embodiment of the present invention (hereinafter referred to as an embodiment) will be described based on the drawings. The moving object tracking device 1 can set the indoor and outdoor three-dimensional space in which the installation object is present, such as a room in which the fixtures are arranged, as the monitoring target space, and the person moving in the monitoring space is the tracking target It is an object (hereinafter referred to as an object). The moving object tracking device 1 processes a surveillance image obtained by imaging a surveillance space to detect and track an object. The installation object in the monitoring space such as furniture does not move like an object, and its installation position is known in advance. Other examples of the installation include pillars and water heaters. The installation is a shield that can hide the image of the object from the viewpoint of image processing. In addition, objects other than the object of interest may also be shielding objects.

[Configuration of Moving Object Tracking Device 1]
FIG. 1 is a block diagram of a moving object tracking device 1 according to the embodiment. The moving object tracking device 1 is configured to include an imaging unit 2, a storage unit 3, an image processing unit 4, and an output unit 5. The photographing unit 2, the storage unit 3 and the output unit 5 are connected to the image processing unit 4.

  The photographing unit 2 is a surveillance camera, which is installed to face the surveillance space, and photographs the surveillance space at predetermined time intervals. The captured surveillance images of the surveillance space are sequentially output to the image processing unit 4. In order to grasp the position and movement of a person moving along a reference surface such as a floor surface or ground surface, the imaging unit 2 is basically installed at a height at which a person can be photographed. For example, in the present embodiment The moving object tracking device 1 is used for indoor monitoring, and the imaging unit 2 is installed on the ceiling. The time interval at which the monitoring image is captured is, for example, 1⁄5 second. Hereinafter, the unit of time carved at the time interval of this imaging is referred to as time.

  The storage unit 3 is a storage device such as a read only memory (ROM) or a random access memory (RAM). The storage unit 3 stores various programs and various data, and inputs and outputs such information from and to the image processing unit 4. Various data include data on models of objects and installations, and camera parameters.

  The image processing unit 4 is configured using an arithmetic device such as a central processing unit (CPU), a digital signal processor (DSP), a micro control unit (MCU) or the like, and is connected to the imaging unit 2, the storage unit 3 and the output unit 5. . The image processing unit 4 functions as each unit described later by reading and executing a program from the storage unit 3.

  The output unit 5 includes an acoustic output means such as a speaker or a buzzer for outputting a warning sound, a liquid crystal display for displaying a monitoring image determined to be abnormal, a display means such as a CRT, etc. When input, it outputs the effect of abnormality occurrence to the outside. In addition, the output unit 5 may include communication means for transmitting an alarm signal to a center apparatus installed at a monitoring center of a security company via a communication line.

  FIG. 2 is a schematic functional block diagram of the moving object tracking device 1. In the configuration shown in FIG. 2, the storage unit 3 functions as an installed object distance storage unit 30. Further, the image processing unit 4 functions as a hypothesis setting unit 40, a model image output unit 41, a model distance calculation unit 42, a visible area estimation unit 43, an object determination unit 44 and an abnormality determination unit 45.

  The hypothesis setting means 40 (candidate setting means) performs motion prediction from the object position of each object determined in the past or the candidate position of each object set in the past, and the object is selected in the newly input monitoring image The position where the object exists is predicted, and the predicted position (candidate position) is output to the model image output unit 41 and the model distance calculation unit 42.

  For example, position prediction can be performed using a method called a particle filter or the like. In this method, a large number of (the number is represented by P, for example, 200 per object) is sequentially set for each object to determine the position (object position) of the object stochastically. The candidate positions to be set are called hypotheses and the like. The candidate position may be set in the xy coordinate system of the monitoring image, but in the present embodiment, it is set in the rectangular coordinate system (XYZ coordinate system) of the monitoring space. The motion prediction is performed by applying a predetermined motion model to a past object position ((1) below) or applying a predetermined motion model to a past candidate position ((2) below).

(1) Prediction from Object Position An average velocity vector is calculated from the object position at T time (for example, T = 5) before the target time. The average velocity vector is added to the object position one time before to predict the object position at the target time. P candidate positions are randomly set in a predetermined range centered on the predicted object position. In this method, the object position for the past T time is cyclically stored in the storage unit 3.

(2) Prediction from candidate position An average velocity vector is calculated from candidate positions at T time (for example, T = 5) before the target time. The average velocity vector is added to the candidate position one time before to predict a new candidate position at the target time. The prediction is performed for each of the P candidate positions, and the new identifier position and the original candidate position in the past are assigned the same identifier and circularly stored. Among candidate positions one time earlier, those whose likelihood is lower than a likelihood threshold set in advance are deleted. On the other hand, in order to compensate for this deletion, new candidate positions one time earlier as many as the deleted number are randomly set within a predetermined range centered on the predicted object position one time ago, and these candidate positions And the corresponding candidate positions before 2 hours are extrapolated according to the motion of the object position in the past. Therefore, in addition to the past candidate positions, the object position for the past T time is also circularly stored in the storage unit 3.

  When a new object is detected, the hypothesis setting unit 40 randomly sets P candidate positions in a predetermined range centered on the detected position.

  In addition, the hypothesis setting means 40 includes candidate postures in the hypothesis. For example, a standing posture (standing position) is set to the candidate postures for half of the hypotheses of each object, and a falling posture (falling posture) is set to the candidate postures for the other half.

  Alternatively, for an object whose current posture can be predicted based on the shape of a group of change pixels (change region), the size of movement speed, etc., the hypothesis setting means 40 predicts the predicted posture hypothesis from the hypothesis of other postures. You can also set many. For example, for the person being tracked, a standing posture is set to the candidate posture of 90% of the hypotheses, and the remaining 10% of the candidates are the persons whose change region having a long shape in the vertical direction is extracted around the candidate position. Set posture to posture.

  The model image output unit 41 outputs, to the visible region estimation unit 43, a model image obtained by projecting the three-dimensional model of the object at the candidate position on the coordinate system of the monitoring image. FIG. 3 is a functional block diagram of the model image output means 41, and shows two configuration examples of the model image output means 41. As shown in FIG. In the configuration shown in FIG. 3, the three-dimensional model storage unit 300, the camera parameter storage unit 301 and the model image storage unit 302 are realized by the storage unit 3, and the model image generation unit 420 and the model image readout unit 421 are realized by the image processing unit 4. Be done.

  The model image output means 41 shown in FIG. 3A is a form of projecting a three-dimensional model to generate a model image each time estimation processing of a visible region of a target object is performed. Three-dimensional model storage means 300, camera parameter storage means 301 and model image generation means 420. The three-dimensional model storage means 300 stores a three-dimensional model of the object, and the camera parameter storage means 301 stores camera parameters of the photographing means. The model image generation means 420 reads the three-dimensional model and the camera parameters from the three-dimensional model storage means 300 and the camera parameter storage means 301. Then, the model image generation unit 420 projects a three-dimensional model of the object at the candidate position on the coordinate system of the monitoring image using the camera parameters to generate and output a model image.

  The model image output means 41 shown in FIG. 3B has a form for reading out a model image obtained by projecting a three-dimensional model in advance, and comprises a model image storage means 302 and a model image readout means 421. The model image storage means 302 stores a model image at each position in the three-dimensional space, and the model image readout means 421 reads out from the model image storage means 302 the model image stored corresponding to the candidate position. Output.

  Alternatively, a model image at a representative position in a three-dimensional space is stored as an intermediate form of these, and the model image at the representative position is modified such as enlargement / reduction according to the relationship with the representative position of the candidate position. It is also possible to generate and output a model image of a candidate position by performing processing.

  The model distance calculation means 42 receives candidate positions and candidate postures as hypotheses from the hypothesis setting means 40, calculates a projection distance from the photographing means to the candidate position, and outputs the projection distance to the visible area estimation means 43. Here, assuming that a linear subspace in which a major axis vector in the longitudinal direction of a solid model is defined in a three-dimensional space which is a monitoring space is referred to as a major axis vector space for each candidate posture, the projection distance is It is defined as the size of the component orthogonal to the principal axis vector space. The projection distance calculated by the model distance calculation means 42 is a distance with respect to a sight line vector having the viewpoint of the camera as the photographing means as a start point and the candidate position as an end point. The projection distance from the photographing means to the candidate position is referred to as a model distance in order to distinguish it from the projection distance for the sight line vector from the photographing means described later to the installation.

  Therefore, the model distance calculation means 42 switches the type of model distance according to the candidate postures of the object input from the hypothesis setting means 40. FIG. 4 is a schematic view of the monitoring space for explaining the specific definition of the model distance according to the candidate posture, FIG. 4 (A) is a perspective view, and FIG. 4 (B) is along the YZ plane. FIG. Hereinafter, the case where the candidate posture is standing and the case where the candidate is recumbent will be sequentially described.

(1) When the candidate posture is standing position The main axis 601 of the three-dimensional model 600 of a standing person is a direction parallel to the Z axis, ie, a vertical direction, so the main axis vector is bound to a one-dimensional space which is a straight line in the vertical direction. , This is the principal axis vector space. In this case, the XY plane which is a plane perpendicular to the main axis 601 is the orthogonal complement space of the main axis vector space, and the size of the sight line vector on this XY plane is the projection distance. Here, the projection distance is defined by the size of an image obtained by orthogonally projecting the gaze vector onto the horizontal surface 610 (in the example of FIG. 4, the floor of a room which is a monitoring space). Therefore, the model distance calculating means 42 calculates the horizontal distance d _HM1 from the photographing means to the candidate position P ₁ as a model distance for the three-dimensional model 600 having the vertical main axis 601.

Coordinates of position C of the photographing means in three-dimensional space expressed in the Cartesian coordinate system _{XYZ (X C, Y C,} Z C), standing the human a coordinate candidate position _{P 1 (X 1, Y 1} , Assuming that Z ₁ ), the horizontal distance d _HM1 which is a model distance is given by the following equation.
d _HM1 = {(X _1- X _C ) ² + (Y _1- Y _C ) ² } ^1/2 ... (1)

Incidentally, the model image reading means 421, taking into account the thickness 2W ₁ of the three-dimensional model 600, the candidate positions _{P 1} is defined as a point 602 close as _{W 1} to the imaging means than the main shaft 601 of the three-dimensional model 600 models the distance It may be calculated. The definition is effective, for example, when the installation includes a wall. Including this example, the positional relationship between the candidate position P ₁ and the three-dimensional model 600 can be defined by the front and rear from the main axis 601 in a range of thickness of the three-dimensional model 600. This is because the magnitude relationship between the model distance and the installed object distance described later is maintained within that range, and the visible region estimation means 43 described later can correctly determine the presence or absence of the concealment.

(2) When the candidate posture is recumbent The main axis 621 of the three-dimensional model 620 of the recumbent human being is in a direction parallel to the XY plane, that is, the orientation varies in the horizontal direction. Bound, this is the principal vector space. In this case, the vertical axis is an orthogonal complement of the main axis vector space, and the projection distance is defined by the vertical distance which is the size of an image obtained by orthogonal projection of the sight line vector on the vertical axis.

Thus, for the three-dimensional model 620 having a horizontal main shaft 621, the model distance calculating unit 42 calculates the vertical distance d _VM2 of line-of-sight vector from the imaging means (point C) to the candidate position P ₂ as a model distance. Assuming that the coordinates of the candidate position P ₂ are (X ₂ , Y ₂ , Z ₂ ), the vertical distance d _VM2 is given by the following equation.
d _VM2 = | Z ₂ −Z _C | ......... (2)

Incidentally, the model distance calculating unit 42, taking into account the thickness 2W ₂ of the three-dimensional model 620, the candidate position _{P 2} is defined as _{W 2} as close point 622 to the photographing unit than the main shaft 621 of the three-dimensional model 620 models the distance It may be calculated. The definition is valid if the installation includes a floor. Including this example, it is possible for the same reason as in the case of horizontal distance, positional relationship between the candidate position P ₂ and the three-dimensional model 620 to define by longitudinal (vertical) from the spindle 621 in a range of thickness of the three-dimensional model 620 .

  Assuming that the horizontal distance and the vertical distance, which are the lengths of the image obtained by orthogonally projecting the line-of-sight vector of the photographing means onto the horizontal plane and the vertical axis, respectively, are the two types of projection distances of the line-of-sight vector. The projection distance of the gaze vector from the photographing means to the candidate position is calculated as a model distance for at least one designated type predetermined for the candidate posture among the types of the projection distance. Specifically, the projection distance of the designated type in the present embodiment is a horizontal distance when the candidate posture is standing, and is a vertical distance when the candidate posture is recumbent.

  The method of calculating the model distance in accordance with the candidate posture has been described above. In this method, both the horizontal distance and the vertical distance are distances along a plane perpendicular to the major axis of the solid model. That is, in any of the candidate postures, the position of the three-dimensional model at the time of judging the concealment by the installation is captured at a distance along a plane perpendicular to the main axis of the three-dimensional model. The significance of this method will be described using the schematic diagram of FIG.

FIG. 5 is a schematic cross-sectional view along the YZ plane of the monitoring space in the arrangement example of the three-dimensional models 600 and 620 shown in FIG. 4 (A). The candidate positions P ₁ and P ₂ corresponding to the three-dimensional models 600 and 620 are located at the centers of the three-dimensional models 600 and 620 on the horizontal surface 610. The standing of the three-dimensional model 600 position along the major axis 601 as a point for comparison with the candidate position P ₁ (i.e. Z-axis coordinate) is different from P _1a, illustrates the P _1b. To candidate positions P ₁ that is a ground point between the floor of the three-dimensional model 600, the point P _1a is the intersection of the height center toward the eye vector and the three-dimensional model 600 of the surface of the three-dimensional model 600, the point P _{1 b} is located at the head of the three-dimensional model 600. The position along the major axis 621 as a point for comparison with the candidate position P ₂ in the three-dimensional model 620 decubitus (i.e. Y-axis coordinate) is different from P _2a, illustrates a P _2b. To candidate position P ₂ that is a ground point of the floor in the longitudinal direction of the center of the three-dimensional model 620, the point P _2a is towards the head of the three-dimensional model 620 is a portion close to the photographing means from the candidate position P ₂ sight The point P _{2 b} is an intersection point of the line-of-sight vector directed to the leg of the three-dimensional model 620 which is a portion farther from the photographing means than the candidate position P ₂ and the three-dimensional model 620.

  As can be seen in FIG. 5, the distance from the camera to the object varies with the orientation of the gaze vector. In other words, even if the distance is the same object, it may be different for each pixel in the monitoring image. Here, while the processing load becomes excessive if the distance is calculated for each pixel for the same object, the processing load can be reduced if the distance relating to the candidate position is commonly used for each pixel of the same object. On the other hand, when a distance which can be originally different for each pixel is replaced with a common distance, it is naturally possible that an error occurs and an erroneous concealment state is estimated.

  In this regard, in the present invention, the magnitude of the line-of-sight vector from the camera to the candidate position is represented by the distance in the direction orthogonal to the main axis of the three-dimensional model, and the distance is commonly used for each pixel of the same object. I am trying to reduce the error.

That is, the difference in the length of the line-of-sight vector to each point of the solid model of the same object is the contribution from the difference in the vertical component of the line-of-sight vector (referred to as the vertical difference contribution) When divided into contributions from differences (referred to as horizontal difference contributions), in the standing three-dimensional model 600, the size in the horizontal direction (the width of the person) is smaller than the size in the vertical direction (the height of the person who is the object). Therefore, the horizontal difference contribution is smaller than the vertical difference contribution. For example, with regard to three line-of-sight vectors CP ₁ , CP _1a , and CP _{1b with} respect to the solid model 600, the difference in the component in the direction orthogonal to the main axis 601 (Y axis direction in FIG. It becomes smaller than the difference of the height grade of solid model 600 which arises in the component of a parallel direction (in FIG. 5, Z-axis direction). The distance in the direction orthogonal to the main axis of the solid model, such as the horizontal distance, eliminates the component in the Z-axis direction, so the length of the line-of-sight vector itself or the length of the image obtained by orthogonal projection of the line-of-sight vector onto the Z-axis, Z The difference across the stereo model 600 is small compared to the distance defined including the axial components. Thus, the candidate posture can be reduced in the above-described error by using in common to each pixel of the example (1) the same object the horizontal distance of the candidate positions P ₁ given by the formula in the case of standing.

On the other hand, in the recumbent three-dimensional model 620, since the size in the vertical direction is smaller than the size in the horizontal direction, the vertical difference contribution is smaller than the horizontal difference contribution. For example, with regard to three line-of-sight vectors CP ₂ , CP _2a , and CP _{2b with} respect to the solid model 620, the difference in components in the direction orthogonal to the main axis 621 (Z axis direction in FIG. 5) is the height of the solid model 620 (thickness of person) The difference is smaller than the difference in the length (the height of the person) of the solid model 620 generated in the component in the direction parallel to the main axis 621 (the Y-axis direction in FIG. 5). In the distance in the direction orthogonal to the main axis of the three-dimensional model, such as the vertical distance, the component in the Y-axis direction is removed. The difference across the stereo model 620 is small compared to the distance defined including the axial components. Thus, if the candidate position is supine example (2) by using in common to the pixels of the same object in the vertical distance of the candidate position P ₂ given by equation can be reduced in the above-mentioned error.

  The installation object distance storage unit 30 stores the projection distance to the installation object in the three-dimensional space of the gaze vector corresponding to the pixel in association with each pixel of the monitoring image. In this configuration, the storage capacity required for the installation depends on the number of pixels of the surveillance image, and is not affected even if the number of installations existing in the surveillance space and the complexity and size of the surveillance space increase.

  In the present embodiment, different projection distances are used according to the candidate postures. In this case, the installed object distance storage means 30 uses the photographing means and the candidate positions for use when the candidate postures are standing. The horizontal distance is stored, and the vertical distance is stored for use in the case where the candidate posture is recumbent. That is, the installed object distance storage unit 30 stores two types of projection distances to the installed object in the three-dimensional space of the gaze vector corresponding to the pixel as the installed object distance in association with each pixel of the monitoring image.

  A model in which the horizontal distance or the vertical distance corresponding to the gaze vector from the photographing unit to the installation object stored in the installation object distance storage unit 30 is the horizontal distance or the vertical distance about the gaze vector from the photographing unit to the candidate position described above In order to distinguish it from distance, it is called installed object distance.

  The installed object distance stored in the installed object distance storage means 30 may be calculated in advance by simulation using a three-dimensional model of the installed object, or based on an actual measurement value obtained by a three-dimensional measuring instrument or the like. It may be calculated in advance.

FIG. 6 is a schematic view for explaining a specific example of the installed object distance. FIG. 6A is a perspective view of the monitoring space in which the installation object 700 exists, and the imaging surface 710 on which points P _{3 to} P _{5 on} the surface of the installation object 700 are projected is shown. The horizontal distances stored in the installed object distance storage means 30 constitute a horizontal distance group 720 equal in number to the number of pixels of the monitoring image, and FIG. 6B shows the horizontal distance group 720 as an image in the same xy coordinate system as the monitoring image. Is schematically represented in the form of Similarly, the vertical distances stored in the installed object distance storage means 30 constitute a vertical distance group 730 equal in number to the number of pixels of the monitoring image, and FIG. 6C shows the vertical distance group 730 in the same xy coordinate system as the monitoring image. It is schematically represented in the form of the image in.

As shown in FIG. 6A, the projection points on the imaging plane 710 corresponding to the points P _{3 to} P ₅ on the surface of the installation 700 are Q _{3 to} Q ₅ , and the projection points are the horizontal distance group 720 and the vertical at a distance group 730 are shown as a pixel _Q 3 to Q _5. The pixel _Q 3 to Q ₅ each at a horizontal distance group 720 shown in FIG. 6 (B) horizontal distance _d HF3 _{to d HF 5} shown in FIG. 6 (A) is stored. Also, it represents the horizontal distance of the pixel _{Q 6} not reflected is in installed objects FIG 6 (B) and _{d HF 6.} Vertical distance _d VF3 _{to d VF5} shown in FIG. 6 (A) is stored in the pixel _Q 3 to Q ₅ each in a vertical distance group 730 shown in FIG. 6 (C). Also, the vertical distance of the pixel _{Q 7} not reflected is in installed objects FIG 6 (C) represents the _{d VF7.}

Specifically, the coordinates in three-dimensional space of points _{P 3} of the surface of the installed objects 700 and _{(X 3, Y 3, Z} 3), also the point _{P 3} on the surface of the installed objects 700 and the position C of the photographing means intersection of the imaging plane 710 of the linearly with imaging means for connecting, i.e. when the coordinates (i.e., pixel positions) in the imaging plane 710 of the projection point _{Q 3} of the xy plane of the point _{P 3} and _(x 3, _{y 3),} installed horizontal distance _{d HF3} in the object distance storage means 30 corresponding to the coordinates of the pixel _{Q 3 (x 3, y 3} ), as the vertical distance _{d VF3,}
d _{HF 3} = {(X ₃ −X _C ) ² + (Y ₃ −Y _C ) ² } ^1/2
d _{VF 3} = | Z ₃ −Z _C |
Is stored.

The same applies to the other two points P ₄ and P ₅ on the surface of the installation 700, and the horizontal distance d _{HF 4} , to the installation distance storage means 30 corresponding to the coordinates (x ₄ , y ₄ ) of the pixel Q ₄ . As the vertical distance d _VF4 ,
d _{HF 4} = {(X ₄ −X _C ) ² + (Y ₄ −Y _C ) ² } ^1/2
d _{VF 4} = | Z ₄ −Z _C |
_Are stored in correspondence with the coordinates (x ₅ , y ₅ ) of the pixel Q ₅ in the installed object distance storage means 30 as the horizontal distance d _{HF 5} and the vertical distance d _{VF 5} ,
d _{HF 5} = {(X ₅ −X _C ) ² + (Y ₅ −Y _C ) ² } ^1/2
d _VF5 = | Z _5- Z _C |
Is stored.

Incidentally, exemplified as a horizontal distance in pixels Q ₆ in FIG. 6 (B), the the corresponding horizontal distance of no pixel of installed objects in the viewing direction d _{HF 6,} sets always larger than model distance. For example, the value of _dHF6 can be a line segment that can enter the monitoring space (shooting range) or a value greater than the maximum length of the gaze vector that can be set in the monitoring space. Alternatively, if the monitoring space is a room, the wall can be treated as being equivalent to the installation, in which case _{dHF 6} can be the horizontal distance from the photographing means to the wall.

Similarly, for the vertical distance d _{VF 7 of the} pixel having no installation in the corresponding viewing direction, which is exemplified as the horizontal distance at the pixel Q ₇ in FIG. _6C , a value that is necessarily larger than the model distance is set. . For example, the value of d _{VF 7} can be a line segment that can enter the monitoring space (photographing range) or a value greater than the maximum length of the gaze vector that can be set in the monitoring space. Alternatively, the floor or ground can be treated as conforming to the installation, in which case d _{VF 7} can be the vertical distance from the imaging means to the floor.

  Among the pixels constituting the model image of the three-dimensional model, the visible region estimation means 43 corresponds to the pixels and the pixels whose installation object distance stored in the installation object distance storage means 30 is equal to or more than the model distance of the three-dimensional model It is estimated that the object is a visible pixel. Specifically, the visible region estimation means 43 receives the model image corresponding to the hypothesis (ie, the candidate position and the candidate posture) from the model image output means 41, and the model distance corresponding to the hypothesis from the model distance calculation means 42. And estimate the object visible region not concealed by the installation object of the model image. In the estimation process, among pixels constituting a model image, a pixel whose installation object distance corresponding to the pixel is equal to or more than the model distance is regarded as an object visible pixel, and a pixel whose installation object distance is less than the model distance is an installation object Is a concealed pixel concealed by the The visible region estimation means 43 outputs the object visible region consisting of the estimated object visible pixels to the object determination means 44.

  As described above, the type (designated type) of the model distance is switched according to the candidate posture. The visible region estimation means 43 reads out the installed object distance having the designated type in common with the model distance from the installed object distance storage means 30 and uses it. When the candidate posture is a standing position, that is, when the model distance is a horizontal distance, the visible region estimation means 43 determines, for each pixel constituting the model image, the horizontal distance among the installation object distances of the pixels that is equal to or greater than the model distance. The pixel which is is estimated as the pixel of the object visible region, and the pixel whose installation object distance is less than the model distance is estimated as the hidden pixel.

FIG. 7 is a schematic diagram for explaining a processing example of the visible region estimation means 43 when the candidate posture is standing. For example, the candidate positions and the candidate orientation model image output unit 41 in response to output from the hypothesis setting means 40, the projection of the standing of the three-dimensional model 600 disposed in the candidate position P ₁ shown in FIG. 4 to the imaging surface xy model The image 800 is input to the visible area estimation means 43 (upper part of FIG. 7). The pixels constituting the model image 800 contains the line of sight direction of the pixel Q ₆ to a point no point line-of-sight direction of the pixel Q ₃ and installation of the P ₃ on the surface of the installed object 700 shown in FIG. 6 There is. Further, the model distance d _HM1 is input from the model distance calculation means 42 to the visible region estimation means 43 corresponding to the candidate position and the candidate posture outputted by the hypothesis setting means 40.

The visible region estimation means 43 corresponds to the candidate posture being a standing position, and constitutes pixels of the model image 800 from the horizontal distance group 720 of the installation object distances stored in the installation object distance storage means 30. And the corresponding horizontal distances (..., D _{HF 6} ,..., D _{HF 3} ,...) _Are read (the middle part of FIG. 7).

The visible region estimation means 43 compares the model distance with the installed object distance of each pixel constituting the model image 800, and estimates an object visible region 801 consisting of pixels whose model distance is equal to or less than the installed object distance Figure 7 lower). Specifically, the pixel _{Q 6,} since the model distance _{d HM1} is less than the installed object distance _{d HF 6,} it is determined that the visible pixels constituting the object visible region 801. On the other hand, the pixel _{Q 3} are model distance _{d HM1} is not determined visible pixels constituting the installed object distance _{d HF3} greater for objects visible region 801.

  In addition, when the candidate posture is recumbent, that is, when the model distance is the vertical distance, the visible region estimation means 43 calculates, for each pixel constituting the model image, the vertical distance among the installation object distances of the pixel. A pixel which is equal to or greater than the distance is estimated as a pixel of the object visible region, and a pixel whose installed object distance is less than the model distance is estimated as a hidden pixel.

FIG. 8 is a schematic view for explaining a processing example of the visible region estimation means 43 when the candidate posture is the decubitus position. For example, from the model image output means 41 corresponding to the candidate location and the candidate orientation output from the hypothesis setting means 40, the projection of the recumbent three-dimensional model 620 disposed in the candidate position P ₂ shown in FIG. 4 to the imaging surface xy model The image 810 is input to the visible region estimation means 43 (upper part of FIG. 8). The pixels constituting the model image 810 contains the line of sight direction of the pixel Q ₇ to a point no point line-of-sight direction of the pixel Q ₅ and installation of the P ₅ on the surface of the installed object 700 shown in FIG. 6 There is. Further, the model distance d _VM2 is input from the model distance calculation means 42 to the visible region estimation means 43 corresponding to the candidate position and the candidate posture outputted by the hypothesis setting means 40.

The visible region estimation means 43 corresponds to the candidate posture being recumbent, and the pixels constituting the model image 810 from the vertical distance group 730 of the installation object distances stored in the installation object distance storage means 30. the corresponding horizontal distance _{(..., d VF5, ...,} d VF7, ...) read out (FIG. 8 middle).

The visible region estimation means 43 compares the model distance with the installed object distance of each pixel constituting the model image 810, and estimates an object visible region 811 consisting of pixels whose model distance is equal to or less than the installed object distance Figure 8 bottom). Specifically, the pixel _{Q 7,} since the model distance _{d VM2} is less than the installed object distance _{d VF7,} it is determined that the visible pixels constituting the object visible region 811. On the other hand, the pixel _{Q 5,} the model distance _{d VM2} is determined not to visible pixels constituting the installed object distance _{d VF5} greater for objects visible region 811.

  The object determining means 44 determines the presence of the image of the object from the image features of the object visible pixels in the monitoring image. Specifically, the object judging means 44 receives the information of the object visible area from the visible area estimating means 43, judges the presence of at least the image of the object from the image features of the object visible area in the monitoring image, The result is output to the abnormality determination means 45. In the present embodiment, the abnormality determination means 45 further determines the position of the object which is the state of the object and the posture of the object. These processes are performed by the change pixel extraction unit 440, the likelihood calculation unit 441, and the position / posture determination unit 442 included in the object determination unit 44.

  The change pixel extraction unit 440 extracts a change pixel from the monitoring image newly input from the imaging unit 2, and outputs information of the extracted change pixel to the likelihood calculation unit 441. Information on the change pixel is also output to the hypothesis setting means 40 as needed. Extraction of the change pixel is performed by known background difference processing. That is, the change pixel extraction unit 440 performs difference processing between the newly input monitoring image and the background image, and extracts a pixel whose difference is equal to or more than a preset difference threshold as a change pixel. The change pixels may be extracted corresponding to the area where the object exists. The change pixel extraction unit 440 stores, in the storage unit 3, a monitoring image in a state in which no object exists in the monitoring area as a background image. For example, since the administrator basically activates the moving object tracking device 1 with no object in the monitoring area, it is possible to generate a background image from the monitoring image immediately after activation. Note that instead of the difference processing, the change pixels may be extracted by correlation processing between the newly input monitoring image and the background image, or instead of the background image, the background model is learned and the difference with the background model The changed pixels may be extracted by processing.

  The likelihood calculating unit 441 extracts the feature amount of the object in the object visible region estimated for each hypothesis from the monitoring image, and corresponds to the object position of the candidate position in the hypothesis according to the extraction degree of the feature amount. Calculating the likelihood of and outputting to the position / posture determination means 442. The following (1) to (4) are examples of the likelihood calculation method.

  (1) The object visible region is superimposed on the change pixel extracted by the change pixel extraction means 440, and the ratio (inclusion degree) in which the change pixel is included in the object visible region is determined. The degree of inclusion tends to be higher as the candidate position is closer to the position where the object is actually present, and tends to be lower as it goes away. Therefore, the likelihood is calculated based on the degree of inclusion.

  (2) Extract an edge from a portion corresponding to the contour of the object visible region in the surveillance image. As the candidate position is closer to the position where the object actually exists, the edge extraction degree (for example, the sum of extracted edge strengths) increases because the contour of the object visible region matches the edge position, while the distance increases The degree of extraction tends to decrease. Therefore, the likelihood is calculated based on the degree of extraction of the edge.

  (3) The feature amount extracted from the monitoring image at the past object position of each object is stored in the storage unit 3 as reference information of the object. As the candidate position is closer to the position where the object actually exists, the feature amounts of the background and other objects are not mixed, and the similarity between the feature amount extracted from the object visible region and the reference information becomes high. The farther you go, the lower the degree of similarity. Therefore, the feature amount in the object visible region is extracted from the monitoring image, and the similarity between the extracted feature amount and the reference information is calculated as the likelihood. For example, various image feature quantities such as edge distribution, color histogram, or both of them can be used as feature quantities here.

  (4) Further, the likelihood may be calculated according to the weighted addition value of a plurality of the degree of inclusion, the degree of extraction of the edge, and the degree of similarity described above.

  By using the object visible region estimated by the visible region estimation means 43 in this manner, the likelihood adapted to the concealed state of each object can be calculated, and thus the tracking reliability is improved.

The position / posture determination unit 442 determines the position (object position) of the object from each hypothesis of the object and the likelihood calculated for each hypothesis, and the determination result is stored in the storage unit 3 for each object. Accumulate in series. If all the likelihoods are less than a predetermined lower limit (likelihood lower limit), it is determined that there is no object position, that is, it has disappeared. The following (1) to (3) are examples of the method of calculating the object position.
(1) For each object, calculate a weighted average value of candidate positions weighted by the likelihood, and use this as the object position of the object.
(2) The candidate position for which the maximum likelihood has been calculated is determined for each object, and this is used as the object position.
(3) For each object, calculate the average value of the candidate positions for which the likelihood higher than the preset likelihood threshold has been calculated, and use this as the object position. Here, the likelihood threshold> the likelihood lower limit.

  As described above, in the object determining means 44, the likelihood calculating means 441 and the position / posture determining means 442 have a function of determining the presence of the image of the object from the image features of the object visible region in the image. Further, the object determining means 44 determines the candidate posture set as the hypothesis of the highest likelihood when the disappearance is not determined as the posture of the object.

  The abnormality determining means 45 refers to the time-series object positions accumulated in the storage unit 3 and determines that a suspicious movement staying for a long time or a suspicious movement deviating from the normal flow line is abnormal, and the abnormality is determined. And an alarm signal to the output unit 5.

[Operation of Moving Object Tracking Device 1]
Next, the tracking operation of the moving object tracking device 1 will be described. 9 and 10 are schematic flowcharts of the tracking process of the moving object tracking device 1.

  The image processing unit 4 acquires a monitoring image from the imaging unit 2 each time the imaging unit 2 captures an image of the monitoring space (step S1). Hereinafter, the time when the latest monitoring image is input is referred to as the current time, and the latest monitoring image is referred to as the current image.

  The current image is compared with the background image by the change pixel extraction means 440, and the change pixel extraction means 440 extracts change pixels (step S2). Here, isolated change pixels are excluded from the extraction result as noise. In addition, immediately after the start of the operation in which there is no background image, the current image is stored in the storage unit 3 as a background image, and it is assumed that there is no change pixel for convenience.

  Also, the hypothesis setting means 40 sets P hypotheses (candidate posture and candidate position) for each object being tracked based on motion prediction (step S3). The candidate positions of the object determined to be new appearances in step S16 described later are not predictable in motion, and P hypotheses are set in a wide range centered on the appearance positions. In addition, the hypothesis of the object determined to be lost at step S16 described later is deleted.

  The image processing unit 4 does not extract changed pixels in step S2 and no hypothesis is set in step S3 (if there is no target object being tracked) (in the case of "YES" in step S4) The process returns to step S1 and waits for the input of the next monitoring image.

  On the other hand, if "NO" in the step S4, the processes of the steps S5 to S16 are performed. First, the hypothesis setting means 40 determines the anteroposterior relation of the object (step S5). Specifically, the hypothesis setting unit 40 calculates the distance between the center of gravity (average value) of the candidate position and the camera position, and creates a context list in which the identifiers of the objects are arranged in ascending order of distance.

  The image processing unit 4 sequentially sets each target object as a target object in order from the top of the contextual list (step S6). Subsequently, the image processing unit 4 sequentially sets each hypothesis of the target object as a target hypothesis (step S7). However, hypotheses having candidate positions outside the field of view of the surveillance image are excluded from setting targets of the target hypothesis, and the object visible region of the hypothesis is not estimated, and the likelihood is set to zero.

  The model image output unit 41 reads, from the camera parameter storage unit 301, a three-dimensional model read from the three-dimensional model storage unit 300, for example, an object model image corresponding to the candidate posture and the candidate position input from the hypothesis setting unit 40. The projection image is projected and generated on the coordinate system of the monitoring image, and output to the visible region estimation means 43 (step S8). As described above, the model image output unit 41 may be configured to read an object model image corresponding to the input candidate posture and candidate position from the model image storage unit 302 and to output it to the visible region estimation unit 43. it can.

  The visible area estimation means 43 estimates the concealment by the installation object in the object model image input from the model image output means 41 (step S9). FIG. 11 is a schematic flowchart of the installed object hiding and estimating process in step S9.

If the model distance calculation means 42 is a standing hypothesis where the hypothesis input from the hypothesis setting means 40 is a candidate posture (in the case of “YES” in step S100), the horizontal distance d for the candidate position of the hypothesis is _{The HM} is calculated and output to the visible region estimation means 43 as a model distance (step S101).

The visible area estimation means 43 sequentially sets the pixels in the object model image input from the model image output means 41 as the pixel of interest (step S102), and sets the monitor image as the installation object distance to the installation object distance storage means 30. Of the horizontal distance and the vertical distance stored in association with each pixel, the horizontal distance _dHF (x, y) corresponding to the xy coordinates of the pixel of interest is read (step S103).

If the visible region estimating means 43 installed object distance _d HF (x, y) compared with the model distances _{d HM} and installed object distance _d HF (x, y) is the model distance _{d HM} or more (step S104 " In the case of “YES”, the target pixel is estimated to be an object visible pixel, that is, a non-hidden pixel not hidden by the installation (step S105), and the process proceeds to step S106. On the other hand, if the installation object distance _dHF (x, y) is less than the model distance _dHM (in the case of "NO" in step S104), the target pixel is treated as a concealed pixel concealed by the installation object. Therefore, the pixel of interest is not estimated to be an object visible pixel, and the process proceeds to step S106.

  The visible region estimation means 43 repeats the processing of steps S102 to S105 until the processing is completed for all pixels of the object model image (in the case of “NO” in step S106), and when the processing is completed for all pixels (step In the case of “YES” in S106), the process proceeds to step S10 in FIG.

In addition, when the hypothesis input from the hypothesis setting means 40 is not a standing hypothesis in which the candidate posture is a standing posture, that is, when the model distance calculating means 42 is a lying hypothesis (in the case of “NO” in step S100), The vertical distance d _VM is calculated for the candidate position of the hypothesis, and this is output to the visible region estimation means 43 as a model distance (step S107).

The visible area estimation means 43 sequentially sets the pixels in the object model image input from the model image output means 41 as the pixel of interest (step S108), and sets the monitor image as the installation object distance in the installation object distance storage means 30. Of the horizontal distance and the vertical distance stored in association with each pixel, the vertical distance d _VF (x, y) corresponding to the xy coordinates of the pixel of interest is read (step S109).

If the visible region estimating means 43 installed object distance _d VF (x, y) compared to a model distance _{d VM,} installed objects distance _d VF (x, y) is the model distance _{d VM} above (step S110 " In the case of “YES”, the target pixel is estimated to be an object visible pixel, that is, a non-hidden pixel not hidden by the installation object (step S111), and the process proceeds to step S112. On the other hand, if the installation object distance d _VF (x, y) is less than the model distance d _VM (in the case of “NO” in step S110), the pixel of interest is treated as a concealed pixel concealed by the installation object. Therefore, the target pixel is not estimated to be the object visible pixel, and the process proceeds to step S112.

  The visible region estimation unit 43 repeats the processing of steps S108 to S111 until the processing is completed for all the pixels of the object model image (in the case of “NO” in step S112), and when the processing is completed for all the pixels (step In the case of “YES” in S112), the process proceeds to step S10 in FIG.

  If there is an object in front of the object of interest with reference to the context list (in the case of “YES” in step S10), the visible region estimation means 43 an object model image corresponding to the object position of the object in front The object visible area is updated by performing mask processing for excluding the area of the object from the object visible area (step S11). If there are multiple objects in front, mask processing may be attempted for all of them, and among them, the angle between the projection line from the camera position to the object position and the projection line to the object of interest is 2 W in width The exclusion process may be performed by narrowing down to only the objects less than the corresponding angle. On the other hand, if there is no target in front (in the case of "NO" in step S10), the mask processing is not performed.

  When the hidden area by the installation object and the object in front is removed and the object visible area at the candidate position of the attention hypothesis is obtained, the likelihood calculation means 441 calculates the likelihood for the attention hypothesis (step S12) ). The likelihood calculation means 441 receives the part corresponding to the change pixel extracted by the change pixel extraction means 440 in the photographed image, and from the part, the feature of the object in the object visible region estimated for the attention hypothesis The amount is extracted, the likelihood as the object position of the hypothesis is calculated according to the extraction degree of the feature amount, and is output to the position / posture determination unit 442.

  The image processing unit 4 repeats the processes of steps S7 to S12 when there is a hypothesis whose likelihood is not calculated (in the case of “NO” in step S13). When the likelihood is calculated for all the P hypotheses (in the case of "YES" in step S13), the position / posture determination unit 442 calculates the likelihood calculated for each hypothesis of the object of interest and each of the hypotheses. The object position of the target object is calculated using this, and the posture is determined (step S14). The object position calculated for the current time is additionally written in association with the object position of the target object stored in the storage unit 3 up to one time ago. In addition, in the case of the object which appeared newly, a new identifier is provided and registered. Also, when the likelihood at all predicted positions is less than the likelihood lower limit value, it is determined that there is no object position.

  When an unprocessed target remains (in the case of “NO” in step S15), the image processing unit 4 repeats the processing of steps S6 to S14 for determining the object position of the target. On the other hand, when object positions are determined for all objects (in the case of “YES” in step S15), new appearance and disappearance of the objects are determined (step S16). Specifically, the image processing unit 4 synthesizes the object visible area estimated for each object position, and detects a change pixel outside the combination area among the change pixels extracted by the change pixel extraction unit 440. And label adjacent change pixels among the detected change pixels. If the label has a size that can be regarded as an object, the effect of the new appearance is recorded in the storage unit 3 together with the position (appearing position) of the label. Further, if there is an object without an object position, it is recorded in the storage unit 3 that the object has disappeared. When the above processing is completed, the process returns to step S1 to perform processing on the monitoring image of the next time.

[Modification]
In the above embodiment, a horizontal distance, which is the length of a vector obtained by orthogonally projecting the sight line vector of the photographing means on the horizontal plane, is calculated as the projection distance when the candidate posture is standing, and the model distance defined by the horizontal distance An example is shown in which the presence or absence of the concealment is determined by comparing the size of the object with the installed object distance. On the other hand, the X component or Y component of the orthographic projection vector corresponding to the horizontal distance is defined as the projection distance, and the concealment can be determined by comparing the model distance and the installation object distance according to the definition.

The coordinates of the position C of the photographing means are (X _C , Y _C , Z _C ), and the point P ₃ (X of the surface of the installation object with respect to the candidate position P ₁ (X ₁ , Y ₁ , Z ₁ ) of the standing person _The present modification will be specifically described with reference to determination of the presence or absence of concealment of ( ₃ , Y ₃ , Z ₃ ). Visible region estimating unit 43, for example, if one of the following: (3a) of the formula ~ (3d) expression Naritate, pixels corresponding to P ₃ among the pixels of the model image of the three-dimensional model of the candidate position P ₁ is the installed object It is determined not to be concealed, that is, visible pixels.
When | (X ₁ −X _C ) |> || (Y ₁ −Y _C ) | and (X ₁ −X _C )> 0,
(X ₁ -X _C ) <(X ₃ -X _C ), that is, X ₁ <X ₃ ... (3a)
When | (X ₁ −X _C ) |> || (Y ₁ −Y _C ) | and (X ₁ −X _C ) <0,
(X ₁ -X _C )> (X ₃ -X _C ), that is, X ₁ > X ₃ ... (3b)
When | (X ₁ −X _C ) | <| (Y ₁ −Y _C ) | and (Y ₁ −Y _C )> 0,
(Y ₁ -Y _C ) <(Y ₃ -Y _C ), that is, Y ₁ <Y ₃ ... (3c)
When | (X ₁ −X _C ) | <| (Y ₁ −Y _C ) | and (Y ₁ −Y _C ) <0,
(Y ₁ -Y _C )> (Y ₃ -Y _C ), that is, Y ₁ > Y ₃ ... (3d)

In this case, for example, the model distance calculating unit 42 generates an X component (X ₁ −X _C ) and a Y component (X component) of an orthogonal projection vector corresponding to the horizontal distance from the imaging unit to the candidate position P ₁ of the standing solid model 600. Y ₁ -Y _C) were output in the visible region estimating unit 43, installed objects distance storage unit 30 X component _(X 3 point _{P 3} on the surface of the installed objects 700 -X _C) and Y component _(Y 3 -Y _C ) is stored and output to the visible area estimation means 43. In this configuration, the data amount of the installed object distance stored corresponding to the standing position in the installed object distance storage means 30 is doubled as compared with the above embodiment which stores the horizontal distance, but for calculating the model distance The calculation of the power and the square root included in the equation (1) is unnecessary, and the processing amount can be reduced.

  Incidentally, the vertical distance used when the candidate posture in the above embodiment is recumbent is the size of the Z component of a vector obtained by orthogonal projection of the sight line vector of the photographing means on the vertical plane orthogonal to the main axis 621 of the three-dimensional model 620, It has technical common points with this modification.

  Reference Signs List 1 moving object tracking device, 2 imaging unit, 3 storage unit, 4 image processing unit, 5 output unit, 30 installed object distance storage unit, 40 hypothesis setting unit, 41 model image output unit, 42 model distance calculation unit, 43 visible region Estimating means, 44 object determining means, 45 abnormality determining means, 300 three-dimensional model storing means, 301 camera parameter storing means, 302 model image storing means, 420 model image generating means, 421 model image reading means, 440 changing pixel extracting means, 441 likelihood calculation means, 442 position / posture determination means, 600, 620 three-dimensional model, 601, 621 main axis, 610 horizontal plane, 700 installation object, 710 photographing plane.

Claims (4)

An object image estimation device that estimates an object visible region in which an image of an object appears in an image obtained by capturing a predetermined three-dimensional space by an imaging unit using a three-dimensional model of the object in the three-dimensional space. ,
As two types of projection distances of the line-of-sight vector of the photographing means, horizontal distance and vertical distance which are the length of an image obtained by orthogonally projecting the line-of-sight vector onto the horizontal plane and vertical axis are defined.
An installed object distance storage unit which stores, as an installed object distance, two types of the projection distances of the gaze vector corresponding to the pixel to the installation object in the three-dimensional space in association with each pixel of the image;
Candidate setting means for setting a candidate posture of the object and a candidate position in the three-dimensional space in which the object may exist;
Model image output means for outputting a model image obtained by projecting the three-dimensional model in the candidate posture at the candidate position onto the coordinate system of the image;
Model distance calculating means for calculating, as a model distance, the projection distance of the sight line vector to the candidate position for at least one designated type predetermined for the candidate posture among the types of the projection distance;
Among the pixels constituting the model image of the three-dimensional model, a pixel for which the installation object distance of the specified type of the pixel is equal to or more than the model distance of the specified type of the three-dimensional model is visible Area estimation means,
An object image estimation apparatus characterized by having. The object is a person,
The horizontal distance is determined as the projection distance of the designated type when the candidate posture is set to the standing position.
The object image estimation device according to claim 1, characterized in that The object is a person,
The vertical distance is determined as the projection distance of the designated type with respect to the case where the candidate posture is set in the recumbent position,
The object image estimation device according to claim 1 or 2, characterized in that An object image estimation apparatus according to any one of claims 1 to 3;
Object determination means for determining the presence of an image of the object from image features of the object visible pixels in the image;
An object image judging device characterized by having.

Images