JP2018136832A - Object image estimation device and object image determination device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem in which estimation accuracy deteriorates when a front side and a back side have different concealment situations as a person in a decubitus position in an object image estimation device that estimates a visible area of an object using a three-dimensional model in an image imaging a three-dimensional space.SOLUTION: A horizontal distance and a vertical distance being lengths of an image, in which visual line vectors are orthogonally projected to a horizontal plane and a vertical axis respectively, are defined as two kinds of projective distances of a visual line vector of imaging means. An installation object distance storage means 30 stores the two kinds of projective distances to an installation object as an installation object distance by associating with each pixel of the image. A model distance calculation means 42 calculates the projective distance to a candidate position of an object, being a specified kind corresponding to a candidate posture as a model distance. A visible area estimation means 43 estimates a pixel, in which the specified kind of installation object distance of the pixel is more than or equal to the specified kind of model distance of a three-dimensional model, of the pixels constituting a model image of the three-dimensional model at the candidate position as an object visible pixel.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an object image estimation device and an object image determination device that estimate an object visible region in which an image of an object appears in an image, and particularly to an object that can be concealed on an image by an installation such as a fixture. The present invention relates to an apparatus for estimating a visible region.

  An image of a monitoring object such as a person is extracted from a monitoring image obtained by imaging the monitoring space, and the presence or absence of the monitoring object is determined or tracked based on image characteristics of the image. In the monitoring image, it is not always possible to observe the entire image of the monitoring object, and a part of the monitoring image is often concealed by fixtures and pillars in the monitoring space, and other persons. This concealment fluctuates image characteristics such as the observed size and color component of the monitored object, which causes detection failure and tracking failure.

  Therefore, by simulating concealment using a three-dimensional model, the region of the monitored object (object visible region) that is not concealed in the monitored image is estimated, and the image of the object is determined from the image features of the object visible region. By detecting the presence of the image, detection failure and tracking failure due to variation in image characteristics are prevented.

  Further, since the projection processing of the three-dimensional model in this simulation has a large processing load and makes it difficult to detect and track in real time, the processing load is reduced by prior projection processing.

  For example, in the object image determination apparatuses described in Patent Documents 1 and 2, a three-dimensional model is projected in advance on each of an object such as a person and an object in a monitoring space, and the object model image and the object model image are displayed. Correspondence between each and floor surface coordinates, and a floor surface map that is concealed by the installed object is generated and stored in the storage unit. If the person position is the concealment position, the object corresponding to the person position The object visible region is obtained by excluding the overlapping part with the installation object model image from the object model image.

JP 2012-108574 A JP 2012-155595 A

  However, in the conventional technology, an object whose depth width can be ignored, such as a standing person, is set as a monitoring target, and the concealment state may be different between the front and the back like a fallen person (person in a supine position). The estimation accuracy for the monitored object was reduced.

  Furthermore, in the conventional technology, the model projection image for each installation and the correspondence for each floor coordinate are stored, so the storage capacity must be increased according to the complexity and size of the monitoring space. There is a problem that the time and effort of the pre-processing is increased when the layout is changed.

  That is, in the prior art, the concealment position on the floor surface is associated with the two-dimensional object model image standing at the position. It was necessary to memorize a model image of a different installation object for each concealment position by setting multiple.

  Further, in the prior art, since the installation object model image is stored for each installation object, the capacity for storing the installation object model image has to be increased as the number of installation objects increases.

  Further, in the prior art, since the presence / absence of concealment and the correspondence relationship with the installed object model image are stored for each position in the monitoring space, the presence / absence of concealment and the installed object model image are determined according to the size of the monitored space. The capacity for memorizing the correspondence had to be increased.

  The present invention has been made in view of the above problems, and an object that can accurately estimate a non-hidden region of a monitoring object that can take a plurality of postures including a posture with a long depth such as a prone person. It is an object of the present invention to provide an image estimation apparatus and an object image determination apparatus that can determine the presence of an image of a monitoring object with high accuracy. Another object of the present invention is to provide an object image estimation device and an object image determination device that do not change the storage capacity required for an installed object even if the monitoring space is increased in complexity and size.

  (1) The object image estimation apparatus according to the present invention is configured to display an object visible region in which an image of an object appears in an image obtained by photographing a predetermined three-dimensional space by a photographing unit, and the object in the three-dimensional space. An apparatus for estimation using a three-dimensional model, wherein two types of projection distances of the line-of-sight vector of the photographing means are the horizontal distance and the vertical distance, which are the lengths of images obtained by orthogonal projection of the line-of-sight vector onto the horizontal plane and the vertical axis, respectively. An installation distance storage that stores two types of projection distances to the installation in the three-dimensional space of the line-of-sight vector corresponding to the pixel as the installation distance in association with each pixel of the image Means, candidate setting means for setting the candidate posture of the object, and a candidate position in the three-dimensional space where the target object may exist, and the stereoscopic model at the candidate posture at the candidate position. Model image output means for outputting a model image obtained by projecting the image on the coordinate system of the image, and at least one specified type predetermined for the candidate posture among the types of the projection distance, Model distance calculation means for calculating the projection distance of the line-of-sight vector as a model distance, and among the pixels constituting the model image of the stereo model, the specified object distance of the specified type of the pixel is the specification of the stereo model Visible region estimation means for estimating a pixel that is equal to or larger than the model distance as a target visible pixel.

  (2) In the object image estimation apparatus according to (1), the object is a person and the horizontal as the projection distance of the specified type when the candidate posture is set to a standing position. The distance may be determined.

  (3) In the object image estimation apparatus according to (1) and (2) above, when the object is a person and the candidate posture is set to the supine position, the projection of the specified type The vertical distance can be determined as the distance.

  (4) The object image determination apparatus according to the present invention is based on the object image estimation apparatus according to any one of (1) to (3) above, and the image feature of the object visible pixel in the image. Object determining means for determining presence of an image of the object.

  According to the present invention, it is possible to determine the existence of an image of a monitoring object that can take a plurality of postures with high accuracy. Further, according to the present invention, it is possible to determine the presence of the image of the monitoring object at high speed and with high accuracy without increasing the storage capacity required for the installed object even if the monitoring space is increased in complexity and size.

It is a block block diagram of the moving object tracking apparatus which concerns on embodiment of this invention. 1 is a schematic functional block diagram of a moving object tracking device according to an embodiment of the present invention. It is a functional block diagram of a model image output means. It is a schematic diagram of the monitoring space for demonstrating the specific definition of the model distance according to a candidate attitude | position. It is a typical sectional view of surveillance space for explaining the meaning of expressing the position of a solid model at the time of judging concealment by an installation object with the distance along the plane perpendicular to the principal axis of a solid model. It is a schematic diagram for demonstrating the specific example of the installation object distance. It is a schematic diagram explaining the example of a process of the visible region estimation means when a candidate attitude | position is standing. It is a schematic diagram explaining the example of a process of the visible region estimation means when a candidate attitude | position is a supine position. It is a general | schematic flowchart of the tracking process of the moving object tracking apparatus which concerns on embodiment of this invention. It is a general | schematic flowchart of the tracking process of the moving object tracking apparatus which concerns on embodiment of this invention. It is a general | schematic flowchart of an installation object concealment estimation process.

  Hereinafter, a moving object tracking device 1 according to an embodiment of the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. The moving object tracking device 1 can set an indoor / outdoor three-dimensional space where an installation is present, such as a room in which furniture is arranged, as a monitoring target space, and a person who moves in the monitoring space can be tracked An object (hereinafter referred to as an object). The moving object tracking device 1 detects and tracks a target object by processing a monitoring image obtained by imaging the monitoring space. The installation object in the monitoring space such as a fixture does not move like an object, and its installation position is known in advance. Other examples of installed objects include pillars and water heaters. The installed object is a shielding object that can conceal the image of the object from the viewpoint of image processing. Note that objects other than the object of interest can also be a shielding object.

[Configuration of Moving Object Tracking Device 1]
FIG. 1 is a block diagram of a moving object tracking device 1 according to the embodiment. The moving object tracking device 1 includes an imaging unit 2, a storage unit 3, an image processing unit 4, and an output unit 5. The photographing unit 2, the storage unit 3, and the output unit 5 are connected to the image processing unit 4.

  The imaging unit 2 is a monitoring camera, is installed so as to face the monitoring space, and images the monitoring space at a predetermined time interval. The captured monitoring images of the monitoring space are sequentially output to the image processing unit 4. In order to grasp the position and movement of a person who moves along a reference plane such as the floor surface or the ground surface, the photographing unit 2 is basically installed at a height at which a person can be seen from a bird's-eye view. The moving object tracking device 1 is used for indoor monitoring, and the photographing unit 2 is installed on the ceiling. The time interval at which the monitoring image is captured is 1/5 second, for example. Hereinafter, the unit of time recorded at the time interval of imaging is referred to as time.

  The storage unit 3 is a storage device such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage unit 3 stores various programs and various data, and inputs and outputs such information to and from the image processing unit 4. The various data includes data relating to the model of the object or the installation object, and camera parameters.

  The image processing unit 4 is configured using an arithmetic device such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit), and is connected to the photographing unit 2, the storage unit 3, and the output unit 5. . The image processing unit 4 functions as each unit to be described later by reading and executing a program from the storage unit 3.

  The output unit 5 includes a sound output unit such as a speaker or a buzzer that outputs a warning sound, a display unit such as a liquid crystal display or a CRT that displays a monitoring image determined to be abnormal, and an alarm signal is output from the image processing unit 4. When input, the fact that an error has occurred is output to the outside. Further, the output unit 5 may include a communication unit that transmits an alarm signal to a center device installed in a monitoring center of a security company via a communication line.

  FIG. 2 is a schematic functional block diagram of the moving object tracking device 1. In the configuration shown in FIG. 2, the storage unit 3 functions as the installation distance storage unit 30. The image processing unit 4 functions as a hypothesis setting unit 40, a model image output unit 41, a model distance calculation unit 42, a visible region estimation unit 43, an object determination unit 44, and an abnormality determination unit 45.

  The hypothesis setting means 40 (candidate setting means) performs motion prediction from the object position of each target object determined in the past or the candidate position of each target object set in the past, and in the newly input monitoring image The position where the object is present is predicted, and the predicted position (candidate position) is output to the model image output means 41 and the model distance calculation means 42.

  For example, position prediction can be performed using a method called a particle filter. In this method, a large number of candidate positions (the number of which is represented by P. For example, 200 per object) are sequentially set for each object, and the position (object position) of the object is determined probabilistically. The candidate position to be set is called a hypothesis or the like. Although the candidate position can be set in the xy coordinate system of the monitoring image, in this embodiment, it is set in the orthogonal coordinate system (XYZ coordinate system) of the monitoring space. The motion prediction is performed by applying a predetermined motion model to the past object position (the following (1)) or applying a predetermined motion model to the past candidate position (the following (2)).

(1) Prediction from object position An average velocity vector is calculated from the object position for T time (for example, T = 5) before the attention time. This average velocity vector is added to the object position one time ago to predict the object position at the time of interest. P candidate positions are randomly set within a predetermined range centered on the predicted object position. In this method, object positions for the past T times are circulated and stored in the storage unit 3.

(2) Prediction from candidate position An average velocity vector is calculated from candidate positions for T time (for example, T = 5) before the attention time. This average velocity vector is added to the candidate position one hour before, and a new candidate position at the time of interest is predicted. Prediction is performed for each of the P candidate positions, and the same identifier is assigned to the new candidate position and the past candidate position that is the origin of the new candidate position, and is circularly stored. Note that candidate positions whose likelihood is lower than a preset likelihood threshold are deleted from candidate positions one hour ago. On the other hand, in order to compensate for this deletion, new candidate positions one hour before the same number as the number deleted are randomly set within a predetermined range centered on the predicted object position one hour before, and these candidate positions And extrapolating the candidate positions before two hours before corresponding to the motion of the past object position. Therefore, in addition to the past candidate positions, the object positions for the past T times are also circulated and stored in the storage unit 3.

  When a new object is detected, the hypothesis setting unit 40 randomly sets P candidate positions within a predetermined range centered on the detected position.

  Further, the hypothesis setting means 40 includes the candidate posture in the hypothesis. For example, a standing posture (standing position) is set as a candidate posture for half of the hypotheses of each target object, and a falling posture (declining posture) is set as a candidate posture for the remaining half.

  Alternatively, the hypothesis setting unit 40 may use a predicted posture hypothesis as compared to other posture hypotheses for an object whose current posture can be predicted based on the shape of a group of changed pixels (change region), the magnitude of movement speed, and the like. Can also be set. For example, among the people being tracked, for a person whose vertical region of change in the vertical direction is extracted around the candidate position, the standing posture is set to 90% of the hypothetical candidate postures, and the remaining 10% of the candidates Set the posture to posture.

  The model image output unit 41 outputs a model image obtained by projecting the three-dimensional model of the object at the candidate position onto the coordinate system of the monitoring image to the visible region estimation unit 43. FIG. 3 is a functional block diagram of the model image output unit 41, and shows two types of configuration examples of the model image output unit 41. In the configuration shown in FIG. 3, the three-dimensional model storage unit 300, the camera parameter storage unit 301, and the model image storage unit 302 are realized by the storage unit 3, and the model image generation unit 420 and the model image reading unit 421 are realized by the image processing unit 4. Is done.

  The model image output means 41 shown in FIG. 3A is a form in which a model image is generated by projecting a three-dimensional model every time the visible region of the object is estimated. The three-dimensional model storage means 300, the camera parameter storage means 301 and model image generating means 420. The three-dimensional model storage unit 300 stores a three-dimensional model of the object, and the camera parameter storage unit 301 stores camera parameters of the photographing unit. The model image generation unit 420 reads the stereo model and camera parameters from the stereo model storage unit 300 and the camera parameter storage unit 301. Then, the model image generation unit 420 generates and outputs a model image by projecting the three-dimensional model of the target object at the candidate position onto the coordinate system of the monitoring image using the camera parameters.

  The model image output unit 41 shown in FIG. 3B reads a model image obtained by projecting a three-dimensional model in advance, and includes a model image storage unit 302 and a model image reading unit 421. The model image storage unit 302 stores the model image at each position in the three-dimensional space, and the model image reading unit 421 reads the model image stored corresponding to the candidate position from the model image storage unit 302. Output.

  Alternatively, as an intermediate form, the model image at the representative position in the three-dimensional space is stored, and the model image at the representative position is deformed by enlarging or reducing according to the relationship with the representative position of the candidate position. It is also possible to generate and output a model image of candidate positions by performing processing.

  The model distance calculation unit 42 receives the candidate position and the candidate posture as the hypothesis from the hypothesis setting unit 40, calculates the projection distance from the imaging unit to the candidate position, and outputs the projection distance to the visible region estimation unit 43. Here, for each candidate posture, a linear subspace in which the principal axis vector in the longitudinal direction of the three-dimensional model is defined in the three-dimensional space that is the monitoring space is referred to as a principal axis vector space, and the projection distance is the line-of-sight vector of the imaging means. It is defined as the size of the component orthogonal to the principal axis vector space. The projection distance calculated by the model distance calculation unit 42 is a distance with respect to a line-of-sight vector starting from the viewpoint of the camera serving as the imaging unit and having the candidate position as the end point. This projection distance from the photographing means to the candidate position is referred to as a model distance in order to distinguish it from the projection distance for the line-of-sight vector from the photographing means to the installation described later.

  Therefore, the model distance calculation unit 42 switches the model distance type according to the candidate posture of the target object input from the hypothesis setting unit 40. 4A and 4B are schematic views of the monitoring space for explaining a specific definition of the model distance according to the candidate posture, FIG. 4A is a perspective view, and FIG. 4B is along the YZ plane. It is sectional drawing. Hereinafter, a case where the candidate posture is standing and a case where the candidate posture is standing will be sequentially described.

(1) When Candidate Posture is Standing Since the main axis 601 of the standing human solid model 600 is parallel to the Z axis, that is, the vertical direction, the main axis vector is constrained to a one-dimensional space that is a straight line in the vertical direction. This becomes the principal axis vector space. In this case, the XY plane that is a plane perpendicular to the main axis 601 is the orthogonal complement space of the main axis vector space, and the size of the line-of-sight vector on this XY plane is the projection distance. Here, the projection distance is defined by the size of the image obtained by orthogonally projecting the line-of-sight vector onto the horizontal plane 610 (in the example of FIG. 4, the floor of the room that is the monitoring space). Therefore the model distance calculating unit 42 calculates a horizontal distance d _HM1 from photographing means with respect to the three-dimensional model 600 to the candidate position P ₁ having a vertical main axis 601 as a model distance.

The coordinates of the position C of the photographing means in the three-dimensional space represented by the orthogonal coordinate system XYZ are (X _C , Y _C , Z _C ), and the coordinates of the candidate position P ₁ of the standing person are (X ₁ , Y ₁ , If Z ₁ ), the horizontal distance d _HM1, which is the model distance, is given by the following equation.
d _HM1 = {(X ₁ −X _C ) ² + (Y ₁ −Y _C ) ² } ^1/2 (1)

The model image reading unit 421 takes the thickness 2W ₁ of the stereo model 600 into consideration, defines the candidate position P ₁ as a point 602 that is closer to the imaging unit by W ₁ than the main axis 601 of the stereo model 600, and sets the model distance. It may be calculated. This definition is effective, for example, when a wall is included in an installation. Including this example, the positional relationship between the candidate position P ₁ and the three-dimensional model 600 can be defined by the front and rear from the main axis 601 in a range of thickness of the three-dimensional model 600. This is because, within that range, the magnitude relationship between the model distance and the installation object distance described later is maintained, and the visible area estimation means 43 described later can correctly determine the presence or absence of concealment.

(2) In the case where the candidate posture is the supine position Since the main axis 621 of the three-dimensional model 620 of the supine person is in a direction parallel to the XY plane, that is, in the horizontal direction and has various orientations, the main axis vector is in a two-dimensional space that is a horizontal plane. This is the main axis vector space. In this case, the vertical axis is an orthogonal complement space of the principal axis vector space, and the projection distance is defined as a vertical distance that is the size of an image obtained by orthogonally projecting the line-of-sight vector onto the vertical axis.

Thus, for the three-dimensional model 620 having a horizontal main shaft 621, the model distance calculating unit 42 calculates the vertical distance d _VM2 of line-of-sight vector from the imaging means (point C) to the candidate position P ₂ as a model distance. When the coordinates of the candidate position P ₂ are (X ₂ , Y ₂ , Z ₂ ), the vertical distance d _VM2 is given by the following equation.
d _VM2 = | Z ₂ −Z _C | (2)

Incidentally, the model distance calculating unit 42, taking into account the thickness 2W ₂ of the three-dimensional model 620, the candidate position _{P 2} is defined as _{W 2} as close point 622 to the photographing unit than the main shaft 621 of the three-dimensional model 620 models the distance It may be calculated. This definition is effective when the floor is included in the installation. Including this example, it is possible for the same reason as in the case of horizontal distance, positional relationship between the candidate position P ₂ and the three-dimensional model 620 to define by longitudinal (vertical) from the spindle 621 in a range of thickness of the three-dimensional model 620 .

  When the horizontal distance and the vertical distance, which are the lengths of the images obtained by orthogonally projecting the line-of-sight vector of the photographing unit onto the horizontal plane and the vertical axis, are the two types of projection distance of the line-of-sight vector, the model distance calculating unit 42 is as described above. The projection distance of the line-of-sight vector from the imaging means to the candidate position is calculated as the model distance for at least one specified type predetermined for the candidate posture among the projection distance types. Specifically, the specified type of projection distance in the present embodiment is a horizontal distance when the candidate posture is standing, and a vertical distance when the candidate posture is saddle.

  The method for calculating the model distance according to the candidate posture has been described above. In this method, the horizontal distance and the vertical distance are both distances along a plane perpendicular to the main axis of the three-dimensional model. That is, in any of the candidate postures, the position of the three-dimensional model when determining concealment by the installation object is captured at a distance along a plane perpendicular to the main axis of the three-dimensional model. The significance of this method will be described with reference to the schematic diagram of FIG.

FIG. 5 is a schematic cross-sectional view along the YZ plane of the monitoring space in the arrangement example of the three-dimensional models 600 and 620 shown in FIG. Candidate positions P ₁ and P ₂ corresponding to the three-dimensional models 600 and 620 are positioned at the center of the three-dimensional models 600 and 620 on the horizontal plane 610. The standing solid model 600 illustrates points P _1a and P _1b having different positions in the direction along the main axis 601 (that is, Z-axis coordinates) as points for comparison with the candidate position P ₁ . The candidate position P ₁ is a contact point with the floor of the three-dimensional model 600, whereas the point P _1a is an intersection of the line-of-sight vector toward the center of the height of the three-dimensional model 600 and the surface of the three-dimensional model 600. _{1 b} is located at the head of the three-dimensional model 600. In addition, the prone three-dimensional model 620 illustrates points P _2a and P _2b having different positions in the direction along the main axis 621 (that is, Y-axis coordinates) as points for comparison with the candidate position P ₂ . To candidate position P ₂ that is a ground point of the floor in the longitudinal direction of the center of the three-dimensional model 620, the point P _2a is towards the head of the three-dimensional model 620 is a portion close to the photographing means from the candidate position P ₂ sight The intersection of the vector and the three-dimensional model 620, and the point P _2b is the intersection of the line-of-sight vector toward the leg of the three-dimensional model 620, which is a part farther from the imaging means than the candidate position P _2, and the three-dimensional model 620.

  As can be seen in FIG. 5, the distance from the camera to the object varies depending on the direction of the line-of-sight vector. In other words, the distance can be different for each pixel in the monitoring image even for the same object. Here, if the distance is calculated for each pixel with respect to the same object, the processing load becomes excessive. On the other hand, if the distance regarding the candidate position is commonly used for each pixel of the same object, the processing load can be reduced. On the other hand, when a distance that originally differs from pixel to pixel is replaced with a common distance, naturally an error occurs and the possibility of an erroneous concealment state being estimated increases.

  In this regard, in the present invention, the magnitude of the line-of-sight vector from the camera to the candidate position is represented by a distance in a direction orthogonal to the main axis of the three-dimensional model, and the distance is used in common for each pixel of the same object. The error is reduced.

In other words, the difference in the length of the line-of-sight vector to each point of the three-dimensional model of the same object is defined as the contribution from the difference in the vertical component of the line-of-sight vector (referred to as the vertical difference contribution) and the horizontal component. When divided into contributions from differences (referred to as horizontal difference contributions), in the standing solid model 600, the horizontal size (the width of the person) is smaller than the vertical size (the height of the person who is the object). Therefore, the horizontal difference contribution is smaller than the vertical difference contribution. For example, regarding three line-of-sight vectors CP ₁ , CP _1a , CP _{1b with} respect to the three-dimensional model 600, the difference in components in the direction orthogonal to the main axis 601 (the Y-axis direction in FIG. 5) is about the thickness of the three-dimensional model 600. This is smaller than the difference in height of the solid model 600 that occurs in the components in the parallel direction (Z-axis direction in FIG. 5). Since the component in the Z-axis direction is removed from the distance in the direction orthogonal to the main axis of the three-dimensional model, such as the horizontal distance, the length of the line-of-sight vector itself, the length of the image obtained by orthogonally projecting the line-of-sight vector onto the Z-axis, etc. The difference in the entire solid model 600 is small compared to the distance defined including the axial component. Thus, the candidate posture can be reduced in the above-described error by using in common to each pixel of the example (1) the same object the horizontal distance of the candidate positions P ₁ given by the formula in the case of standing.

On the other hand, since the vertical model is smaller in the vertical model than the horizontal model, the vertical difference contribution is smaller than the horizontal difference contribution. For example, regarding the three line-of-sight vectors CP ₂ , CP _2a , CP _{2b with} respect to the three-dimensional model 620, the difference in the components in the direction orthogonal to the main axis 621 (the Z-axis direction in FIG. 5) is the height of the three-dimensional model 620 (the thickness of the person). It becomes smaller than the difference in the length (person's height) of the three-dimensional model 620 generated in the component in the direction parallel to the main axis 621 (Y-axis direction in FIG. 5). Since the component in the Y-axis direction is removed from the distance in the direction perpendicular to the principal axis of the stereo model, such as the vertical distance, the length of the line-of-sight vector itself or the length of the image obtained by orthogonally projecting the line-of-sight vector to the Y-axis, etc. The difference in the entire three-dimensional model 620 is small compared to the distance defined including the axial component. Thus, if the candidate position is supine example (2) by using in common to the pixels of the same object in the vertical distance of the candidate position P ₂ given by equation can be reduced in the above-mentioned error.

  The installation object distance storage unit 30 stores the projection distance to the installation object in the three-dimensional space of the line-of-sight vector corresponding to each pixel in association with each pixel of the monitoring image. In this configuration, the storage capacity necessary for the installation is determined according to the number of pixels of the monitoring image, and is not affected even if the number of installations existing in the monitoring space and the complexity and size of the monitoring space increase.

  In this embodiment, projection distances of different types are used according to the candidate postures. In this case, the installation object distance storage unit 30 uses the photographing unit and the candidate position for use when the candidate posture is standing. The horizontal distance is stored, and the vertical distance is stored for use when the candidate posture is supine. In other words, the installation object distance storage unit 30 stores two types of projection distances to the installation object in the three-dimensional space of the line-of-sight vector corresponding to the pixel as the installation object distance in association with each pixel of the monitoring image.

  A model in which the horizontal distance or the vertical distance corresponding to the line-of-sight vector from the photographing means to the installation object stored in the installation distance storage means 30 is the horizontal distance or the vertical distance for the line-of-sight vector from the photographing means to the candidate position. In order to distinguish from the distance, it is referred to as an installation distance.

  The installed object distance stored in the installed object distance storage unit 30 may be calculated in advance by a simulation using a three-dimensional model of the installed object, or based on an actual measurement value obtained by a three-dimensional measuring instrument or the like. It may be calculated in advance.

FIG. 6 is a schematic diagram for explaining a specific example of the installation object distance. FIG. 6A is a perspective view of the monitoring space where the installation object 700 exists, and shows an imaging surface 710 on which points P _{3 to} P _{5 on} the surface of the installation object 700 are projected. The horizontal distance stored in the installation object distance storage means 30 constitutes the same number of horizontal distance groups 720 as the number of pixels of the monitoring image, and FIG. 6B shows the horizontal distance group 720 in the same xy coordinate system as the monitoring image. It is schematically represented in the form of Similarly, the vertical distance stored in the installed object distance storage means 30 constitutes the same number of vertical distance groups 730 as the number of pixels of the monitoring image, and FIG. 6C shows the same xy coordinate system as that of the monitoring image. This is schematically shown in the form of an image.

As shown in FIG. 6A, the projection points on the imaging plane 710 corresponding to the points P _{3 to} P ₅ on the surface of the installation object 700 are Q _{3 to} Q ₅ , and the projection points are the horizontal distance group 720 and the vertical point. It is shown as pixels Q _{3 to} Q _{5 in the} distance group 730. In the horizontal distance group 720 shown in FIG. _{6B, the} horizontal distances d _{HF3 to} d _HF5 shown in FIG. _6A are stored in the pixels Q _{3 to} Q ₅ respectively. Also, it represents the horizontal distance of the pixel _{Q 6} not reflected is in installed objects FIG 6 (B) and _{d HF 6.} Vertical distance _d VF3 _{to d VF5} shown in FIG. 6 (A) is stored in the pixel _Q 3 to Q ₅ each in a vertical distance group 730 shown in FIG. 6 (C). Also, the vertical distance of the pixel _{Q 7} not reflected is in installed objects FIG 6 (C) represents the _{d VF7.}

Specifically, the coordinate in the three-dimensional space of the point P _{3 on} the surface of the installation object 700 is (X ₃ , Y ₃ , Z ₃ ), and the position C of the photographing means and the point P _{3 on} the surface of the installation object 700 are intersection of the imaging plane 710 of the linearly with imaging means for connecting, i.e. when the coordinates (i.e., pixel positions) in the imaging plane 710 of the projection point _{Q 3} of the xy plane of the point _{P 3} and _(x 3, _{y 3),} installed horizontal distance _{d HF3} in the object distance storage means 30 corresponding to the coordinates of the pixel _{Q 3 (x 3, y 3} ), as the vertical distance _{d VF3,
^{d HF3 = {(X 3 -X}} C) 2 + (Y 3 -Y C) 2} 1/2
d _VF3 = | Z ₃ -Z _C |
Is memorized.

The same applies to the other two points P ₄ and P ₅ on the surface of the installation object 700, and the installation object distance storage means 30 stores the horizontal distance d _HF4 , corresponding to the coordinates (x ₄ , y ₄ ) of the pixel Q ₄ . As the vertical distance d _VF4 ,
^{_{d HF4 = {(X 4 -X}} C) 2 + (Y 4 -Y C) 2} 1/2
d _VF4 = | Z ₄ -Z _C |
There is stored, the coordinates of the pixel _{Q 5 (x 5, y 5} ) for installation thereof distance storage means 30 corresponding to the horizontal distance _{d HF 5,} as the vertical distance _{d VF5,
^{d HF5 = {(X 5 -X}} C) 2 + (Y 5 -Y C) 2} 1/2
d _VF5 = | Z ₅ -Z _C |
Is memorized.

Incidentally, exemplified as a horizontal distance in pixels Q ₆ in FIG. 6 (B), the the corresponding horizontal distance of no pixel of installed objects in the viewing direction d _{HF 6,} sets always larger than model distance. For example, the value of _dHF6 can be a line segment that can enter the monitoring space (imaging range) or a value that is greater than or equal to the maximum length of the line-of-sight vector that can be set in the monitoring space. Alternatively, if the monitoring space is a room, it is possible to treat the wall as being equivalent to an installed object, and in this case, _dHF6 can be the horizontal distance from the imaging means to the wall.

Similarly, exemplified as a horizontal distance in pixels Q ₇ in FIG. 6 (C), the the corresponding vertical distance no pixel of installed objects in the viewing direction d _VF7 also sets always larger than model distance . For example, the value of d _VF7 can be a line segment that can enter the monitoring space (imaging range) or a value that is greater than or equal to the maximum length of the line-of-sight vector that can be set in the monitoring space. Alternatively, the floor or the ground can be treated as equivalent to the installation object, and in this case, _dVF7 can be a vertical distance from the photographing means to the floor.

  The visible region estimation means 43 is a pixel of which the object distance stored in the object distance storage means 30 corresponding to the pixel is greater than or equal to the model distance of the three-dimensional model among the pixels constituting the model image of the three-dimensional model. The object is estimated as a visible pixel. Specifically, the visible region estimation unit 43 receives a model image corresponding to a hypothesis (that is, a candidate position and a candidate posture) from the model image output unit 41 and the model distance corresponding to the hypothesis from the model distance calculation unit 42. Is estimated, and the object visible region that is not concealed by the installation object in the model image is estimated. In the estimation process, among the pixels constituting the model image, a pixel whose installation distance corresponding to the pixel is equal to or greater than the model distance is a target visible pixel, and a pixel whose installation distance is less than the model distance is an installation object. The concealment pixel is concealed by. The visible area estimating means 43 outputs the object visible area including the estimated object visible pixels to the object determining means 44.

  As described above, the model distance type (specified type) is switched according to the candidate posture. The visible region estimation means 43 reads the installed object distance whose designation type is the same as the model distance from the installed object distance storage means 30 and uses it. When the candidate posture is standing, that is, when the model distance is a horizontal distance, the visible region estimation unit 43 determines that the horizontal distance of the installed object distance of the pixel is greater than or equal to the model distance for each pixel constituting the model image. Are estimated as pixels in the object visible region, and pixels whose installation object distance is less than the model distance are estimated as concealment pixels.

FIG. 7 is a schematic diagram illustrating a processing example of the visible region estimation unit 43 when the candidate posture is standing. For example, the candidate positions and the candidate orientation model image output unit 41 in response to output from the hypothesis setting means 40, the projection of the standing of the three-dimensional model 600 disposed in the candidate position P ₁ shown in FIG. 4 to the imaging surface xy model The image 800 is input to the visible region estimation means 43 (upper part of FIG. 7). Pixels constituting the model image 800 include a pixel Q _{3 in} the line-of-sight direction to the point P ₃ on the surface of the installation object 700 shown in FIG. 6 and a pixel Q _{6 in} the line-of-sight direction to a point having no installation object. Yes. Further, the model distance d _HM1 is input from the model distance calculation unit 42 to the visible region estimation unit 43 corresponding to the candidate position and candidate posture output by the hypothesis setting unit 40.

The visible region estimation unit 43 corresponds to the candidate posture being standing, and the pixels constituting the model image 800 from the horizontal distance group 720 of the installation object distances stored in the installation object distance storage unit 30. .., D _HF6 ,..., D _HF3,.

The visible region estimation means 43 compares the model distance with the installation object distance of the pixel for each pixel constituting the model image 800, and estimates the object visible region 801 including pixels whose model distance is equal to or less than the installation object distance ( FIG. 7 bottom). Specifically, since the model distance d _HM1 is less than the installation object distance d _HF6 , the pixel Q ₆ is determined as a visible pixel that forms the object visible region 801. On the other hand, the pixel _{Q 3} are model distance _{d HM1} is not determined visible pixels constituting the installed object distance _{d HF3} greater for objects visible region 801.

  Further, when the candidate posture is supine, that is, when the model distance is a vertical distance, the visible region estimation unit 43 determines that the vertical distance of the installation object distance of the pixel is a model for each pixel constituting the model image. A pixel that is equal to or longer than the distance is estimated as a pixel in the object visible region, and a pixel whose installation object distance is less than the model distance is estimated as a hidden pixel.

FIG. 8 is a schematic diagram for explaining a processing example of the visible region estimation means 43 when the candidate posture is the supine position. For example, from the model image output means 41 corresponding to the candidate location and the candidate orientation output from the hypothesis setting means 40, the projection of the recumbent three-dimensional model 620 disposed in the candidate position P ₂ shown in FIG. 4 to the imaging surface xy model An image 810 is input to the visible region estimation means 43 (upper part of FIG. 8). The pixels constituting the model image 810 include the pixel Q _{5 in} the line-of-sight direction to the point P ₅ on the surface of the installation object 700 shown in FIG. 6 and the pixel Q _{7 in} the line-of-sight direction to the point having no installation object. Yes. Further, the model distance d _VM2 is input from the model distance calculation unit 42 to the visible region estimation unit 43 corresponding to the candidate position and the candidate posture output by the hypothesis setting unit 40.

The visible region estimating means 43 corresponds to the candidate posture being the saddle position, and the pixels constituting the model image 810 from the vertical distance group 730 among the installation distances stored in the installation distance storage means 30. the corresponding horizontal distance _{(..., d VF5, ...,} d VF7, ...) read out (FIG. 8 middle).

The visible region estimation means 43 compares the model distance with the installation object distance of the pixel for each pixel constituting the model image 810, and estimates the object visible region 811 including pixels whose model distance is equal to or less than the installation object distance ( FIG. 8 bottom). Specifically, the pixel _{Q 7,} since the model distance _{d VM2} is less than the installed object distance _{d VF7,} it is determined that the visible pixels constituting the object visible region 811. On the other hand, the pixel _{Q 5,} the model distance _{d VM2} is determined not to visible pixels constituting the installed object distance _{d VF5} greater for objects visible region 811.

  The object determination means 44 determines the presence of the object image from the image feature of the object visible pixel in the monitoring image. Specifically, the object determining unit 44 receives information on the object visible region from the visible region estimating unit 43, determines the presence of at least the image of the object from the image features of the object visible region in the monitoring image, and determines The result is output to the abnormality determination means 45. In the present embodiment, the abnormality determination unit 45 further determines the position of the target object and the posture of the target object that are in the state of the target object. These processes are performed by the change pixel extraction unit 440, the likelihood calculation unit 441, and the position / posture determination unit 442 included in the object determination unit 44.

  The change pixel extraction unit 440 extracts change pixels from the monitoring image newly input from the imaging unit 2 and outputs information about the extracted change pixels to the likelihood calculation unit 441. Information on the changed pixels is also output to the hypothesis setting means 40 as necessary. Extraction of a change pixel is performed by a known background difference process. That is, the change pixel extraction unit 440 performs difference processing between the newly input monitoring image and the background image, and extracts pixels whose difference is equal to or greater than a preset difference threshold as change pixels. The change pixel can be extracted corresponding to a region where the object exists. The change pixel extraction unit 440 stores, as a background image, a monitoring image in a state where no object exists in the monitoring area in the storage unit 3. For example, the administrator basically activates the moving object tracking device 1 in a state where no object is present in the monitoring area, so that a background image can be generated from the monitoring image immediately after activation. Instead of the difference process, the changed pixels may be extracted by a correlation process between the newly input monitoring image and the background image, or the difference from the background model by learning the background model instead of the background image You may extract a change pixel by a process.

  The likelihood calculating means 441 extracts the feature amount of the target in the target visible region estimated for each hypothesis from the monitoring image, and determines the object position of the candidate position in the hypothesis according to the degree of feature extraction. Is calculated and output to the position / posture determination means 442. The following (1) to (4) are examples of likelihood calculation methods.

  (1) The object visible area is superimposed on the change pixel extracted by the change pixel extraction means 440, and the ratio (inclusion level) at which the change pixel is included in the object visible area is obtained. Inclusion degree is higher as the candidate position is closer to the position where the object is actually present, and is likely to be lower as the distance is further away. Therefore, the likelihood is calculated based on the degree of inclusion.

  (2) An edge is extracted from a portion corresponding to the contour of the object visible region in the monitoring image. The closer the candidate position is to the position where the object actually exists, the more the edge extraction degree (for example, the sum of the extracted edge intensities) increases because the contour of the object visible region matches the edge position, while the more the candidate position is located, The degree of extraction tends to decrease. Therefore, the likelihood is calculated based on the degree of edge extraction.

  (3) The feature amount extracted from the monitoring image at the past object position of each object is stored in the storage unit 3 as reference information of the object. As the candidate position is closer to the position where the target object actually exists, the feature quantity of the background and other target objects will not be mixed, so the similarity between the feature quantity extracted from the target visible area and the reference information becomes higher. The degree of similarity tends to decrease as the distance increases. Therefore, the feature quantity in the object visible region is extracted from the monitoring image, and the similarity between the extracted feature quantity and the reference information is calculated as the likelihood. As the feature amount here, for example, various image feature amounts such as an edge distribution, a color histogram, or both of them can be used.

  (4) The likelihood may be calculated according to a weighted addition value of a plurality of degrees of the above-described inclusion degree, edge extraction degree, and similarity degree.

  By using the object visible region estimated by the visible region estimating means 43 in this way, the likelihood suitable for the concealment state of each object can be calculated, so that the tracking reliability is improved.

The position / posture determination unit 442 determines the position (object position) of the target object from each hypothesis of the target object and the likelihood calculated for each hypothesis, and stores the determination result in the storage unit 3 for each target object. Accumulate in series. When all the likelihoods are less than a predetermined lower limit (likelihood lower limit), it is determined that there is no object position, that is, disappeared. The following (1) to (3) are examples of the object position calculation method.
(1) For each object, a weighted average value of candidate positions weighted by likelihood is calculated, and this is set as the object position of the object.
(2) For each object, a candidate position where the maximum likelihood is calculated is obtained, and this is set as the object position.
(3) For each object, an average value of candidate positions at which likelihoods equal to or higher than a preset likelihood threshold value are calculated is set as an object position. Here, the likelihood threshold> the likelihood lower limit value.

  As described above, the likelihood calculating unit 441 and the position / posture determining unit 442 in the object determining unit 44 have a function of determining the presence of the image of the object from the image features of the object visible region in the image. Moreover, the target object determination means 44 determines the candidate posture set as the maximum likelihood hypothesis when the disappearance is not determined as the posture of the target object.

  The abnormality determination unit 45 refers to the time-series object positions accumulated in the storage unit 3, determines that a suspicious movement that stays for a long time or a suspicious movement that deviates from the normal flow line is abnormal, and determines the abnormality. The alarm signal is output to the output unit 5.

[Operation of Moving Object Tracking Device 1]
Next, the tracking operation of the moving object tracking device 1 will be described. 9 and 10 are schematic flowcharts of the tracking process of the moving object tracking device 1. FIG.

  The image processing unit 4 acquires a monitoring image from the imaging unit 2 every time the imaging unit 2 images the monitoring space (step S1). Hereinafter, the time when the latest monitoring image is input is called the current time, and the latest monitoring image is called the current image.

  The current image is compared with the background image by the change pixel extraction unit 440, and the change pixel extraction unit 440 extracts the change pixel (step S2). Here, the isolated change pixel is excluded from the extraction result as being caused by noise. Note that immediately after the start of an operation without a background image, the current image is stored in the storage unit 3 as a background image, and there is no change pixel for convenience.

  Further, the hypothesis setting means 40 sets P hypotheses (candidate postures and candidate positions) for each object being tracked based on motion prediction (step S3). Note that P hypotheses are set in a wider range centering on the appearance position because the candidate position of the object determined to be a new appearance in step S16 to be described later cannot be predicted. Further, the hypothesis of the object determined to be lost in step S16 described later is deleted.

  In the case where the changed pixel is not extracted in step S2 and the hypothesis is not set in step S3 (there is no object being tracked) (if “YES” in step S4), the image processing unit 4 Returning to step S1, the input of the next monitoring image is awaited.

  On the other hand, if “NO” in the step S4, the processes of the steps S5 to S16 are performed. First, the hypothesis setting means 40 determines the front-rear relationship of the object (step S5). Specifically, the hypothesis setting means 40 calculates the distance between the center of gravity (average value) of the candidate positions and the camera position, and creates a contextual list in which the identifiers of the objects are arranged in ascending order of the distance.

  The image processing unit 4 sequentially sets each target object as a target object in order from the top of the context list (step S6). Subsequently, the image processing unit 4 sequentially sets each hypothesis of the target object as the target hypothesis (step S7). However, a hypothesis having a candidate position outside the field of view of the monitoring image is excluded from the target hypothesis setting target, the object visible region of the hypothesis is not estimated, and the likelihood is set to zero.

  The model image output unit 41 reads the object model image corresponding to the candidate posture and candidate position input from the hypothesis setting unit 40, for example, the stereo model read from the stereo model storage unit 300, and the camera parameter read from the camera parameter storage unit 301. Is projected onto the coordinate system of the monitoring image and generated and output to the visible region estimation means 43 (step S8). As described above, the model image output unit 41 may be configured to read the target object model image corresponding to the input candidate posture and candidate position from the model image storage unit 302 and output it to the visible region estimation unit 43. it can.

  The visible region estimation means 43 estimates the concealment by the installation object in the object model image input from the model image output means 41 (step S9). FIG. 11 is a schematic flowchart of the object concealment estimation process in step S9.

If the hypothesis input from the hypothesis setting unit 40 is a standing hypothesis with the candidate posture standing (“YES” in step S100), the model distance calculating unit 42 determines the horizontal distance d for the candidate position of the hypothesis. _HM is calculated and output to the visible region estimation means 43 as a model distance (step S101).

The visible region estimation unit 43 sequentially sets the pixels in the object model image input from the model image output unit 41 as the target pixel (step S102), and stores the monitored image as the installation object distance in the installation object distance storage unit 30. Of the horizontal distance and the vertical distance stored in association with each pixel, the horizontal distance d _HF (x, y) corresponding to the xy coordinate of the target pixel is read (step S103).

If the visible region estimating means 43 installed object distance _d HF (x, y) compared with the model distances _{d HM} and installed object distance _d HF (x, y) is the model distance _{d HM} or more (step S104 " In the case of “YES”), it is estimated that the target pixel is an object visible pixel, that is, a non-hidden pixel that is not concealed by the installation (step S105), and the process proceeds to step S106. On the other hand, if the installation object distance d _HF (x, y) is less than the model distance d _HM (in the case of “NO” in step S104), the target pixel is treated as a concealed pixel concealed by the installation object. Therefore, the target pixel is not estimated to be an object visible pixel, and the process proceeds to step S106.

  The visible region estimating means 43 repeats the processing of steps S102 to S105 until the processing is completed for all the pixels of the object model image (in the case of “NO” in step S106), and when the processing is completed for all the pixels (step If “YES” in S106), the process proceeds to step S10 in FIG.

Further, the model distance calculation means 42, if the hypothesis input from the hypothesis setting means 40 is not a standing hypothesis with the candidate posture standing, that is, a supine hypothesis (in the case of “NO” in step S100), The vertical distance d _VM is calculated for the candidate position of the hypothesis, and this is output as a model distance to the visible region estimating means 43 (step S107).

The visible region estimation unit 43 sequentially sets the pixels in the object model image input from the model image output unit 41 as the target pixel (step S108), and stores the monitored image as the installation object distance in the installation object distance storage unit 30. Of the horizontal distance and the vertical distance stored in association with each pixel, the vertical distance d _VF (x, y) corresponding to the xy coordinate of the target pixel is read (step S109).

If the visible region estimating means 43 installed object distance _d VF (x, y) compared to a model distance _{d VM,} installed objects distance _d VF (x, y) is the model distance _{d VM} above (step S110 " In the case of “YES”), it is estimated that the target pixel is an object visible pixel, that is, a non-hidden pixel that is not concealed by the installation (step S111), and the process proceeds to step S112. On the other hand, if the installation object distance d _VF (x, y) is less than the model distance d _VM (in the case of “NO” in step S110), the target pixel is treated as a concealed pixel concealed by the installation object. Accordingly, the target pixel is not estimated to be a target visible pixel, and the process proceeds to step S112.

  The visible region estimating means 43 repeats the processing of steps S108 to S111 until the processing is completed for all the pixels of the object model image (in the case of “NO” in step S112), and when the processing is completed for all the pixels (step If “YES” in S112), the process proceeds to step S10 in FIG.

  The visible region estimation means 43 refers to the context list, and if there is an object in front of the object of interest (“YES” in step S10), the object model image corresponding to the object position of the object in front is shown. The object visible area is updated by performing a mask process for removing the area from the object visible area (step S11). Note that if there are a plurality of objects in front, mask processing may be attempted for all of them, and the angle between the projection line from the camera position to the object position and the projection line to the object of interest is 2 W in width. The exclusion process may be executed by limiting only to objects having an angle less than the corresponding angle. On the other hand, if there is no object in front (in the case of “NO” in step S10), the mask process is not performed.

  When the concealment area by the installation object and the front object is removed and the target visible area at the candidate position of the target hypothesis is obtained, the likelihood calculation unit 441 calculates the likelihood for the target hypothesis (step S12). ). The likelihood calculating unit 441 receives a portion of the captured image corresponding to the changed pixel extracted by the changed pixel extracting unit 440, and the feature of the target in the target visible region estimated for the target hypothesis from the portion. The amount is extracted, and the likelihood as the object position of the hypothesis corresponding to the degree of feature amount extraction is calculated and output to the position / posture determination unit 442.

  The image processing unit 4 repeats the processes in steps S7 to S12 when a hypothesis whose likelihood is not calculated remains (in the case of “NO” in step S13). When the likelihood is calculated for all P hypotheses (in the case of “YES” in step S13), the position / posture determination means 442 calculates each hypothesis of the target object and the likelihood calculated for each of the hypotheses. Using this, the object position of the object of interest is calculated and the posture is determined (step S14). The object position calculated for the current time is additionally recorded in association with the object position of the object of interest stored in the storage unit 3 one hour before. In the case of a newly appearing object, a new identifier is assigned and registered. If the likelihood at all predicted positions is less than the lower limit of likelihood, it is determined that there is no object position.

  When an unprocessed object remains (when “NO” in step S15), the image processing unit 4 repeats the processes of steps S6 to S14 for determining the object position of the object. On the other hand, when the object position is determined for all the objects (in the case of “YES” in step S15), the appearance and disappearance of the object are determined (step S16). Specifically, the image processing unit 4 synthesizes the object visible area estimated for each object position, and detects a change pixel outside the synthesis area among the change pixels extracted by the change pixel extraction unit 440. The adjacent change pixels among the detected change pixels are labeled. If the label is a size that can be regarded as an object, a new appearance is recorded in the storage unit 3 together with the label position (appearance position). If there is an object without an object position, the fact that the object has disappeared is recorded in the storage unit 3. When the above processing is completed, the processing returns to step S1 in order to perform processing on the monitoring image at the next time.

[Modification]
In the above embodiment, as the projection distance when the candidate posture is standing, the horizontal distance, which is the length of the vector obtained by orthogonally projecting the line-of-sight vector of the photographing means on the horizontal plane, is calculated, and the model distance defined by the horizontal distance, respectively. An example in which the presence or absence of concealment is determined by comparing the distance between the installation object and the object is shown. On the other hand, the X component or the Y component of the orthogonal projection vector corresponding to the horizontal distance is defined as the projection distance, and the concealment can also be determined by comparing the model distance and the installation object distance in this definition.

The coordinates of the position C of the photographing means are set to (X _C , Y _C , Z _C ), and the point P ₃ (X on the surface of the installation object with respect to the standing person candidate position P ₁ (X ₁ , Y ₁ , Z ₁ ) ₃ , Y ₃ , Z ₃ ) This modification will be specifically described with reference to the determination of the presence or absence of concealment. Visible region estimating unit 43, for example, if one of the following: (3a) of the formula ~ (3d) expression Naritate, pixels corresponding to P ₃ among the pixels of the model image of the three-dimensional model of the candidate position P ₁ is the installed object It is determined that the pixel is not hidden, that is, a visible pixel.
| (X ₁ -X _C ) |> | (Y ₁ -Y _C ) | and (X ₁ -X _C )> 0,
(X ₁ −X _C ) <(X ₃ −X _C ), that is, X ₁ <X ₃ (3a)
| (X ₁ -X _C ) |> | (Y ₁ -Y _C ) | and (X ₁ -X _C ) <0,
(X ₁ −X _C )> (X ₃ −X _C ), that is, X ₁ > X ₃ (3b)
| (X ₁ −X _C ) | <| (Y ₁ −Y _C ) | and (Y ₁ −Y _C )> 0,
_{(Y 1 -Y C) <(} Y 3 -Y C), i.e. _{Y 1 <Y 3 ......... (3c} )
| (X ₁ −X _C ) | <| (Y ₁ −Y _C ) | and (Y ₁ −Y _C ) <0,
(Y ₁ -Y _C )> (Y ₃ -Y _C ), that is, Y ₁ > Y ₃ (3d)

In this case, for example, the model distance calculation unit 42 includes an X component (X ₁ −X _C ) and a Y component (X component (X ₁ −X _C )) of the orthogonal projection vector corresponding to the horizontal distance from the imaging unit to the candidate position P ₁ of the standing stereo model 600. Y ₁ -Y _C ) is output to the visible region estimating means 43, and the installed object distance storing means 30 outputs the X component (X ₃ -X _C ) and Y component (Y ₃ -Y) of the point P _{3 on} the surface of the installed object 700. _C ) is stored and output to the visible region estimation means 43. In this configuration, the data amount of the installation distance stored corresponding to the standing position in the installation distance storage means 30 is twice that of the above-described embodiment in which the horizontal distance is stored. Therefore, calculation of the power and square root included in the equation (1) is unnecessary, and the processing amount can be reduced.

  Incidentally, the vertical distance used when the candidate posture in the above embodiment is the supine position is the magnitude of the Z component of a vector obtained by orthogonally projecting the line-of-sight vector of the imaging means on the vertical plane perpendicular to the main axis 621 of the three-dimensional model 620, This modification has technical common points.

  DESCRIPTION OF SYMBOLS 1 Moving object tracking apparatus, 2 Image pick-up part, 3 Storage part, 4 Image processing part, 5 Output part, 30 Installation object distance storage means, 40 Hypothesis setting means, 41 Model image output means, 42 Model distance calculation means, 43 Visible region Estimating means, 44 object judging means, 45 abnormality judging means, 300 three-dimensional model storing means, 301 camera parameter storing means, 302 model image storing means, 420 model image generating means, 421 model image reading means, 440 change pixel extracting means, 441 Likelihood calculation means, 442 Position / attitude determination means, 600,620 solid model, 601,621 main axis, 610 horizontal plane, 700 installation object, 710 photographing plane.

Claims (4)

An object image estimation apparatus that estimates an object visible region in which an image of an object appears in an image obtained by photographing a predetermined three-dimensional space by an imaging unit using a three-dimensional model of the object in the three-dimensional space. And
As the two types of projection distances of the line-of-sight vector of the photographing means, a horizontal distance and a vertical distance, which are the lengths of images obtained by orthogonal projection of the line-of-sight vector onto the horizontal plane and the vertical axis, respectively, are defined.
An installation distance storage means for storing the two types of projection distances to the installation object in the three-dimensional space of the line-of-sight vector corresponding to the pixel in association with each pixel of the image;
Candidate setting means for setting candidate positions of the object and candidate positions in the three-dimensional space where the object can exist;
Model image output means for outputting a model image obtained by projecting the three-dimensional model at the candidate position at the candidate position onto the coordinate system of the image;
Model distance calculation means for calculating, as a model distance, the projection distance of the line-of-sight vector to the candidate position for at least one specified type predetermined for the candidate posture among the types of the projection distance;
Among the pixels constituting the model image of the three-dimensional model, a pixel in which the specified object distance of the specified type of the pixel is greater than or equal to the specified model distance of the three-dimensional model is estimated to be an object visible pixel. Region estimation means;
The object image estimation apparatus characterized by having. The object is a person;
When the candidate posture is set to the standing position, the horizontal distance is determined as the projection distance of the designated type,
The object image estimation apparatus according to claim 1. The object is a person;
The vertical distance is determined as the projection distance of the designated type when the candidate posture is set to the saddle position,
The object image estimation apparatus according to claim 1, wherein the object image estimation apparatus is an object image estimation apparatus. The object image estimation apparatus according to any one of claims 1 to 3,
An object determination means for determining presence of an image of the object from an image feature of the object visible pixel in the image;
The object image determination apparatus characterized by having.

Images