JP5027758B2 - Image monitoring device - Google Patents

Description

  The present invention relates to an image monitoring apparatus that detects a target object from a monitoring image obtained by imaging a monitoring space, and in particular, stores an image feature of the target object and detects the target object having the image feature in consideration of concealment. The present invention relates to an image monitoring apparatus.

  In order to detect an intruder or the like, a technique is known in which an image related to a target object to be monitored is extracted from a monitoring image obtained by imaging a monitoring space, and the target object is detected and tracked based on the image information. Patent Document 1 below determines whether the object candidate area of the current frame and the object candidate area of the immediately preceding frame include an image of the same object based on the similarity of the image feature values of these areas, and the object of the immediately preceding frame A technique for tracking a moving object by obtaining an object candidate area of the current frame associated with the candidate area is disclosed.

That is, when associating the moving object image at the time before and after with the image feature, the image feature of the moving object image at the immediately preceding time or several hours before is sequentially stored and used.
JP 2006-318064 A

  Conventionally, a single object appearing in a monitoring image at each time is often grasped and extracted as a group of images, that is, a two-dimensional image. At that time, basically, the three-dimensional structure of the target object is discarded. However, in reality, the target object has a three-dimensional structure, and when the arrangement of the object with respect to the shooting direction of the camera changes due to the movement of the object, the state of occlusion that occurs between the parts constituting the object changes. The appearance of the object changes on the monitoring image taken by the camera. For example, when taking a bird's-eye shot with a camera, as the person moves away from just below the camera, the torso hidden in the head and the legs hidden in the torso appear on the image. Due to such a change in the concealment state, the conventional method of storing the image feature of the immediately preceding frame or several frames cannot cope with the occurrence of a new image feature because the previously hidden part of the object appears in the image. . For this reason, there is a problem that it becomes difficult to associate the image of the target object existing in the monitoring image captured at the current time with the stored image feature, and the target object cannot be detected or tracked continuously. .

  The present invention has been made to solve the above-described problems, and provides an image monitoring apparatus that can suitably realize detection and tracking of a target object even if the concealment state changes as the target object moves. For the purpose.

  An image monitoring apparatus according to the present invention includes a projection condition for a monitoring image obtained by perspective projection of a monitoring space, a partial model representing a three-dimensional shape for each of a plurality of constituent parts of a target object to be monitored, and an arrangement relationship between the partial models. A storage unit that stores an object model of the target object represented by the reference object, a reference image and a reference image concealment degree for each component of the target object, the monitoring image, and a position in the monitoring space are input. Virtually disposing the object model at the position and performing perspective projection based on the projection condition, and for each partial model, the other partial model positioned in front of the perspective projection among the projection images The partial model visible area excluding the concealed area by the image and the concealment degree of the projection image by the concealed area are obtained, and the part corresponding to the partial model visible area from the monitoring image is the target. The partial image extraction means for extracting as a partial image for each component of the body, the partial image at the candidate position where the target object may exist and the reference image are compared for a plurality of candidate positions, and the comparison matches Object reference means for determining an object position where the target object is present based on a candidate position, and the reference image and the reference for the constituent part in which the concealment degree at the object position is equal to or less than the reference image concealment degree Object information updating means for updating the image concealment degree with the partial image at the concealment degree and the object position.

  In another image monitoring apparatus according to the present invention, the object determination unit maps the partial image on the reference projection image obtained by the object model arranged at the reference position based on the projection condition. Similarly, a common area where a standard reference image obtained by mapping the reference image on the standard projection image overlaps is obtained, and the object position is determined by comparing the standard partial image and the standard reference image in the common area. judge. In a preferred aspect of the image monitoring apparatus according to the present invention, the object determination unit determines the candidate position and the position of the object model arranged at the time of acquiring the reference image that is farther from the viewpoint position. The image monitoring apparatus is set to the reference position.

  In another image monitoring apparatus according to the present invention, the object determination unit sets a plurality of horizontal strip regions in the height direction on the partial model, and the common region corresponds to the horizontal strip region. The image is divided into a plurality of small areas, and the standard partial image and the standard reference image are compared for each small area.

  According to the present invention, since a reference image with a small concealment can be held, even if the target object moves and the concealment state changes, the image information of the concealment portion is not lost from the reference image, and the accuracy of detection and tracking of the target object is improved. improves.

  Further, according to the present invention, since the comparison is performed except for the area concealed in either the partial image or the reference image, the comparison based on the mutually corresponding areas (common areas) of the partial image and the reference image is performed. This improves the accuracy of target object detection and tracking.

  In addition, since one of the partial images and the reference image that has a short distance from the viewpoint and a high resolution is converted in accordance with the other image that has a long distance and a low resolution, comparison can be made with the same resolution. That is, a comparison in which an error caused by a high-frequency component included in an image with high resolution is reduced is performed, and the accuracy of detection and tracking of the target object is improved.

  It is also possible to accurately determine a target object in which a plurality of different image features are distributed in the height direction within the same component.

  Hereinafter, an image monitoring apparatus 1 according to an embodiment of the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. For example, the image monitoring apparatus 1 detects and tracks a person as an object in a monitoring space such as an entrance of a building or an ATM corner of a financial institution. The image monitoring apparatus 1 is further configured to perform processing such as detecting a suspicious person who stays for a long time and notifying an abnormality based on the tracking result.

  FIG. 1 is a block configuration diagram of an image monitoring apparatus 1 according to the embodiment. The image monitoring apparatus 1 includes an imaging unit 2, an operation unit 3, a storage unit 4, a control unit 5, and an output unit 6.

  The imaging unit 2 is a monitoring camera, is installed in the monitoring space, and images the monitoring space at a predetermined time interval. The captured surveillance image of the surveillance space is output to the control unit 5. Hereinafter, the imaging timing of the imaging unit 2 is referred to as time. When the monitoring space is represented by an orthogonal coordinate system composed of X, Y, and Z axes, and the vertically upward direction is set in the positive direction of the Z axis, the imaging unit 2 is basically used to grasp the position and movement of a person in the XY plane. For example, in this embodiment, the imaging unit 2 is installed on the ceiling with the optical axis of the surveillance camera facing vertically downward (Z-axis negative direction).

  The operation unit 3 is an input means for the administrator to make various settings for the control unit 5, and is a user interface device such as a touch panel display, for example.

  The storage unit 4 is a storage device such as a RAM, a ROM, and a hard disk, and is connected to the control unit 5. The storage unit 4 stores programs and data used by the control unit 5. The data stored therein includes object information 40, a movement trajectory 41, and an imaging condition 42.

  The object information 40 is information regarding the target object detected in the monitoring space, and is stored so as to be identifiable for each target object. The object information 40 includes an object model 400, a reference image 401, a reference image concealment degree 402, and a reference image acquisition position 404.

  The object model 400 is data describing a partial model representing a three-dimensional shape for each of a plurality of constituent parts constituting the target object, and an arrangement relationship between the partial models. A target object to be monitored by the image monitoring apparatus 1 is a person. In the present embodiment, for example, three parts of a head, a torso, and a leg are set as a human component. In addition, as a partial model that approximately represents the three-dimensional shape of each component, an ellipse having a major axis set in the vertical direction and a minor axis set in the horizontal direction is obtained by rotating the ellipse around the vertical axis (Z axis). A spheroid is used. FIG. 2 schematically shows an object model 400 in the XYZ space. In the object model 400, the lower end of the leg partial model is grounded to a reference surface (Z = 0) such as a floor surface or the ground surface, and the trunk and head partial models are sequentially placed on the leg partial model. It has an arrangement relationship that is stacked in the Z-axis direction. The height h from the reference plane to the center of the head and the maximum width w of the trunk (the trunk minor axis diameter) are set for each target object. The length from the head center to the trunk center with respect to the height h, the length from the head center to the leg center, the ratio of the head major axis radius, the trunk major axis radius and the leg major axis radius, and the width w The ratios of the head short axis radius and the leg short axis radius with respect to are set to values common to each target object. In this way, the height and width of the object model 400 are set for each target object, so that estimation of a concealment state and image extraction of each component described later are less affected by the physique difference of the target object. A good extraction process is possible. Note that the head is based on the center of the head because the head is not easily concealed by other components due to the positional relationship with the imaging unit 2 and appears well in the monitoring image.

The reference image 401 is image information that characterizes the target object. The reference image 401 is stored for each component for each target object. An image (partial image) of each constituent part of each target object is extracted from the image captured by the imaging unit 2 during the tracking process, and the part image selected according to the processing described later is stored as the reference image 401. The Hereinafter, the reference image 401 of the component part _p is represented as Fp.

The reference image concealment degree 402 is data representing the degree of concealment of the reference image 401 and is stored corresponding to each reference image 401. Specifically, the ratio of the portion that is not captured in the reference image Fp and is hidden in the other component portion of the image that would have been captured if there was no other component portion for the component portion _p. The reference image hiding degree 402 of p. Hereinafter, representative of the reference image concealment of 402 of the reference image _{F p} and _{E p.} Of the hiding degree it is estimated with the extraction of a partial image in the process of tracking, which corresponds to the partial image is a reference image F _p is stored as a reference picture concealment of E _p.

  The reference image acquisition position 404 is a position of the target object when each reference image 401 of the target object is acquired, and is stored corresponding to each of the reference images 401.

  The movement trajectory 41 is a history of the position of each target object (hereinafter, object position). The object position is a three-dimensional coordinate of the head center.

  The imaging condition 42 includes information regarding a projection condition when the imaging unit 2 captures a monitoring image obtained by perspective projection of the monitoring space. For example, external parameters such as the installation position and imaging direction of the imaging unit 2 in the actual monitoring space, focal length, angle of view, lens distortion, and other lens characteristics of the imaging unit 2, and internal parameters such as the number of pixels of the imaging element are included. These parameters obtained by actual measurement or the like are input in advance from the operation unit 3 and stored in the storage unit 4 when the image monitoring apparatus 1 is used.

  The imaging condition 42 is used for the process of projecting the object model 400 virtually placed in the monitoring space onto the imaging surface of the imaging unit 2. For example, by applying the imaging condition 42 to a known camera model such as a pinhole camera model, a coordinate conversion formula for converting an arbitrary position coordinate in the monitoring space into a coordinate system of the monitoring image, or an arbitrary position coordinate in the monitoring image Can be set to a coordinate conversion formula that converts to the coordinate system of the surveillance space. FIG. 2 is a schematic diagram for explaining the coordinate transformation, and shows the transformation between the object model 400 arranged in the monitoring space 70 defined by the XYZ coordinate system and the projected image on the imaging surface 71.

  Using the above coordinate conversion formula, the object model 400 virtually arranged at the position Q0 in the monitoring space 70 can be projected onto the imaging surface 71 (arrow 73 in the figure), and a two-dimensional image 72a can be obtained as a projected image. . Further, by using the coordinate conversion formula, it is possible to perform back projection from the imaging surface 71 to the monitoring space 70, whereby the two-dimensional image 72a on the monitoring image of the object model 400 placed at the position Q0. A three-dimensional texture that is projected onto the surface (arrow 74 in the figure) and the texture of the two-dimensional image 72 a is pasted on the surface of the object model 400 can be obtained.

  Furthermore, using the coordinate conversion formula, the two-dimensional image 72a on the monitoring image at the position corresponding to the position Q0 can be mapped to the position on the monitoring image corresponding to another position Q1 in the monitoring space 70. it can. In this mapping, the two-dimensional image 72a is projected onto the surface of the object model 400 virtually arranged at the position Q0 (arrow 74 in the figure), and this object model 400 is moved or copied to the position Q1 (arrow in the figure). 75), and is projected onto the imaging surface 71 (arrow 76 in the figure).

  The control unit 5 is configured using an arithmetic device such as a DSP (Digital Signal Processor) or MCU (Micro Control Unit), and is connected to the imaging unit 2, the storage unit 4, and the output unit 6. The control unit 5 reads out and executes the program from the storage unit 4 and functions as the object image extraction unit 50, the tracking unit 52, and the abnormality determination unit 54. The control unit 5 performs image processing on the monitoring image input from the imaging unit 2 to perform tracking of an object, detection of a suspicious person, and the like, and outputs an abnormal signal to the output unit 6 when a suspicious person is detected.

  The object image extraction unit 50 extracts the object image of the target object from the monitoring image obtained by the imaging unit 2, and outputs the extracted object image information to the object determination unit 526. Specifically, the object image extraction unit 50 sets a monitoring image for determining that the target object is not reflected as a background image and stores the background image in the storage unit 4. When the monitoring image is input from the imaging unit 2, the object image extraction unit 50 performs a difference process between the monitoring image and the background image, detects an image that does not exist in the background image, and detects a predetermined image among the detected images. An image having a size larger than the value is extracted as an object image. Here, the predetermined value is a threshold value for removing an image due to noise, and is set in advance.

  The tracking means 52 detects the target object at each time by identifying an image that matches the reference image 401 in the monitoring image, and generates a movement trajectory. The tracking unit 52 includes a position prediction unit 520, a partial image extraction unit 522, an object determination unit 526, and an object information update unit 528.

  The position prediction unit 520 predicts the object position at the current time by applying a motion model such as a constant velocity motion model or a prediction filter such as a Kalman filter to the movement trajectory 41 (past object position). The position prediction unit 520 further sets a plurality of predicted positions around the current position predicted position obtained directly by the motion model or the prediction filter in order to absorb prediction errors due to motion changes. At that time, it is preferable to set a probability function such as a normal distribution with the central predicted position as the mode value, and to distribute the predicted positions more densely in regions with higher probability. Note that the height h of the object model 400 is set as the height of the predicted position based on the constraint that the target object stands on the reference plane.

When the position of the target object is input from the position prediction unit 520, the partial image extraction unit 522 virtually arranges the object model 400 at the position in the monitoring space 70, and uses the imaging condition 42 to set the object model 400. Projecting onto the imaging surface 71 of the imaging unit 2. The partial image extraction unit 522 extracts a part of the monitoring image that overlaps the projection image of the partial model corresponding to each component as a partial image of the component, and also outputs the projection image of the partial model between the components. The degree of concealment of each component is calculated from the overlapping relationship. Hereinafter, the partial image extracted corresponding to the component part _p is represented as f _p , and the concealment degree calculated for the component part _p is represented as ep.

  The processing by the partial image extraction unit 522 is performed in the following three cases, and in each case, the position input to the partial image extraction unit 522 is different. In the first case, the object information 40 of a newly appearing object is generated. In this case, the position where the object image extracting unit 50 extracts the object image is input. In the second case, an object is identified. In this case, the predicted position set by the position prediction unit 520 is input. In the third case, the object information 40 is updated. In this case, the current position of the object determined by the object determination unit 526 is input.

  FIG. 3 is an explanatory diagram of the processing of the partial image extraction means 522. With reference to FIG. 3, the process of the partial image extraction means 522 is demonstrated in detail.

  First, each partial model is independently projected on the imaging surface using the imaging condition 42, and the entire area 80 of the projected image of each component when there is no concealment is calculated. As a result, an entire area 80a of the head, an entire area 80b of the trunk, and an entire area 80c of the legs are obtained.

  Next, a partial model of a constituent part closer to the imaging unit 2 than the constituent part is projected for each constituent part, and a concealment area 81 that is an area where the constituent part can be concealed on the monitoring image is calculated. In the present embodiment, the distance between each component and the imaging unit 2 is close to the order of the head, the trunk, and the legs, and this order can be set in advance. For the partial model of the head, there is no other partial model on the projection line, and there is no concealment area. On the other hand, since the partial model of the head is positioned in front of the partial model of the trunk, the projection image becomes a concealment region 81b for the trunk. Further, since the partial model of the head and torso is positioned in front of the partial model of the leg part, the projection image thereof becomes a concealment region 81c for the leg part.

  The order may be determined by calculating and comparing the distance between the imaging unit 2 and each partial model. Further, the concealment area 81 can be further calculated by subtracting an area where a partial model of another target object located in front of the target object is projected.

  Subsequently, by removing an overlapping portion with the concealment area 81 of the component part from the entire region 80 of each component part, an observation region 82 (partial model visible region) that is actually an area where the component part can be observed on the monitoring image. ) Is calculated. As a result, an observation region 82a that is not concealed from the head is obtained, while observation regions 82b and 82c having a shape in which a part of an ellipse is concealed from the trunk and legs are obtained.

The partial image extraction unit 522 extracts a portion of the monitoring image that overlaps the observation region 82 of each component thus obtained as a partial image for each component. Moreover, for each component, as shown in the following equation to calculate the degree of hiding e _p by dividing the value obtained by subtracting the area S2 of the observation region 82 from the area S1 of the entire area 80 in the area S1 of the entire area 80 .
e _p = (S1−S2) / S1 (1)

When identifying the object, the partial image extraction unit 522 sets the object model 400 as the candidate position in the monitoring space where the target object can exist, with the predicted position input from the position prediction unit 520 as the candidate position. arranged virtually, we obtain a partial image f _p and hiding of e _p.

The object determination unit 526 compares the partial image f _p extracted by the partial image extraction unit 522 and the reference image F _p with respect to the image feature for each component part p, and the comparison result is a plurality of component parts constituting the target object. And the integrated similarity U is calculated. Then, when the integrated similarity U satisfies a predetermined determination criterion, it is determined that the same target object as the reference image exists at the candidate position. Then, the candidate position is determined as the object position of the target object, and additionally stored in the movement locus 41 of the target object.

Specifically, first, the object judging unit 526 extracts the image feature quantity from the partial image f _p of the current time that has been extracted by partial image extraction unit 522, this and the storage unit 4 as image information of the tracking target object by comparing the image feature amount is extracted from the reference image 401 stored in each component p, the degree of similarity is calculated u _p per component as a result of the comparison. For example, the partial image f _p is analyzed to calculate the color histogram h _fp , and the reference image F _p is analyzed to calculate the color histogram h _Fp , for example, both Euclidean as defined by the following equation: distance value obtained by multiplying by -1 may be used as u _p. Note that Σ means the sum of all elements of the color histogram.
u _p ≡−Σ‖h _Fp −h _fp ‖ ² (2)

Next, the object determination unit 526 calculates the above-described integrated similarity U for each target object by adding the similarity u _p of each component to each target object. Then, the integrated similarity U is compared with a predetermined threshold. If U is equal to or greater than the threshold, it is determined that the object image extracted at the current time is the same as the target object of the reference image 401.

Since the similarity u _p per component are those calculated by using the partial image f _p above that only the image portion relating to the partial accurately extracted, performs identification of the object image by integrating the u _p As a result, even when concealment occurs between components, high-precision identification is possible.

Here, as shown in the following equation, the object determination unit 526 weights the partial similarity u _p with the reliability g _p calculated for each constituent part p, and integrates the weighted similarity for each constituent part p. Thus, the integrated similarity U can be calculated. Here, Σ in the following expression means the summation of p, that is, summing up the constituent parts (head, trunk, and legs). By doing so, the component part with higher reliability can contribute more to the identification, and the object image can be identified with high accuracy.
U≡Σg _p u _p ……… (3)

Reliability g _p is degree of hiding ep, is preset as a function of the reference image concealment of E _p. An example of reliability g _p is the g _p defined by the following equation.
g _p ≡αmax (E _p , e _p ) + β (where α> 0, β ≧ 0) ……… (4)

Incidentally, (4) By using the reliability _{g p} of equation among partial similarity _{u p} is the reference image _{F p} and the partial image _{f p,} the partial image _{f p} hiding degree _{e p} and the reference image concealment degree E Identification is performed in consideration of having only the reliability corresponding to the image that is higher of _p , that is, the image with the smaller amount of information.

Here, the partial image f _p and the reference image F _p to be compared are generally based on object models and target objects placed at different object positions, and the shapes and concealment states of the images are different. And when things with different concealment states are compared, an error is likely to be included in the comparison result. That is, an area that is hidden by only one of the partial image f _p and the reference image F _p and not hidden by the other reduces the accuracy of the comparison result. Therefore, the reference image F _p with partial image f _p using the imaging condition 42, and the coordinate transformation to the shape of the case of the target object or object model exists in a common position to one another (reference position), the converted The same position in both the images f _p and F _p is associated with a common position in the component. Then, the coordinate-converted partial image f _p (standard partial image) and the coordinate-converted reference image F _p (standard reference image) are overlapped with each other to obtain an overlapping common area. Make a comparison. As a result, the comparison is performed excluding the area that is hidden by only one of the partial image f _p and the reference image F _p , so that the accuracy of the comparison result can be improved.

4, FIG. 5 is a schematic diagram illustrating a specific example of the coordinate conversion on the reference projection images obtained by the common object model of arranging the reference image F _p with partial image f _p to the reference position.

In the example shown in FIG. 4, the reference position is set to the object position Q _R stored in the reference image acquiring position 404. The partial image f _p (image 90) is projected onto the projection image of the object model placed at the reference image acquisition position 404 (position Q _R ) to generate a projected image 92. This image 92 is projected at the position of the reference image F _p (image 91). Then, a common area between the projection image 92 and the reference image F _p (image 91) is calculated, and the color histogram h _fp of the projection image 92 in the common area and the color histogram h _Fp of the reference image F _p are compared. To describe from the image 90 of the projection of the image 92 in greater detail, the image 90 is a projection image of the object model 400a of the position Q _I, and backprojected to the object model (backprojection step), the position QR to the object model The coordinates are converted into (moving step). Coordinate transformation of the object model, after the movement surface facing the imaging unit 2 in the object model 400a of the position Q _I to position Q _R also to face the image pickup section 2, about the optical axis of the imaging unit 2 It is preferable to perform the conversion by combining the rotational movement and the translational movement in the distance direction from the optical axis. After moving the object model 400a to position Q _R, and projection thereof onto the imaging surface 71 of the imaging unit 2 (the projection step), it is possible to obtain a projection image 92. These back projection step, movement step, and projection step are represented by thick arrows in FIG. FIG. 4 (b), each partial image _{f p} in the image 90 and 92, and the reference image _{F p} in the image 91 shown schematically.

In the example shown in FIG. 5, the reference position, the position of the object model 400b as a projection original partial image f _p, that is, set in the candidate position Q _I. The reference image F _p (image 91) is projected onto the projection image of the object model 400b placed at the position Q _I to generate a projected image 93. This image 93 is projected to the position of the partial image f _p (image 90). Then, it calculates a common region of the projected image 93 and the partial image _{f p} (image 90), to compare the color histogram _{h fp} color histogram _{h Fp} partial image _{f p} of the projection image 93 in the common area. The projection from the image 91 to the image 93 is performed in the opposite direction to that shown in FIG. 4, and the series of backprojection steps, movement steps, and projection steps are represented by thick arrows in FIG. Yes. FIG. 5B schematically shows the reference images F _p in the images 91 and 93 and the partial image f _p in the image 90.

Further, the comparison between the partial image f _p and the reference image F _p can be performed in the coordinate system of the monitoring space 70. In FIG. 6, the partial image f _p and the reference image F _p are respectively back-projected onto the object models 400a and 400b arranged in the monitoring space 70, and the partial image f _p and the reference image F _p are compared between the object models. It is a schematic diagram explaining the example to do. In this case, the partial image f _p (image 90) is back-projected onto the object model 400a virtually arranged at the candidate position (position Q _I ) to calculate the three-dimensional texture 94, and the reference image F _p (image 91) is calculated. A three-dimensional texture 95 is calculated by back-projecting the object model 400b virtually arranged at the reference image acquisition position 404 (position Q _R ), and a common area is calculated by superimposing the two object models 400a and 400b. Compare color histograms. FIG. 6A is a schematic diagram showing back projection from the images 90 and 91 onto the object models 400a and 400b, and FIG. 6B shows a three-dimensional texture 94 corresponding to the partial image f _p and the reference image F _p. It is a schematic diagram with the three-dimensional texture 95 corresponding to.

Here, the comparison error between the partial image f _p extracted by the partial image extraction means 522 and the reference image F _p also occurs due to a difference in resolution. That is, when a high resolution is compared with a low resolution, a high frequency component existing only in the color histogram on the higher resolution side causes a comparison error. The resolution decreases as the distance between the target object and the imaging unit 2 increases. Therefore, the distance d _R between the imaging unit 2 and the reference image acquisition position 404 is determined from the imaging condition 42 and the reference image acquisition position 404, and the query between the imaging unit 2 and the prediction position is determined from the imaging condition 42 and the prediction position (candidate position). calculating a distance d _I respectively, and compares these distances, distances are projected to the far side performs the comparison. In other words, one of the partial images and the reference image that has a short distance from the viewpoint and a high resolution is converted in accordance with the other image that has a long distance and a low resolution, so that the resolution can be matched and compared. As a result, a comparison in which an error caused by a high-frequency component included in an image with high resolution is reduced is performed, and the accuracy of detection and tracking of the target object is improved.

Specifically, when d _R ≦ d _I , as described with reference to FIG. 5, the reference image F _p is projected onto the image corresponding to the predicted position, and when d _R > d _I , FIG. as described with reference to, projected to the image corresponding to the partial image f _p to the reference image acquisition position 404. Thereby, the comparison error caused by the difference in resolution is reduced, and the accuracy of the comparison result can be improved.

  Here, there is a target object in which a plurality of features are distributed in a single component, such as a person wearing a short skirt or short pan. Therefore, a plurality of horizontal belt-like regions are set in the height direction on the surface of the partial model constituting the object model 400, and the common region is set corresponding to the projection image projected on the imaging surface 71. Is divided into a plurality of small areas, and the color histograms of the small areas are compared. As a result, it is possible to accurately determine a target object in which image features are distributed in a layered manner within the component.

  On the monitoring image, the height h direction of the object model 400 can be specified as the direction of a straight line connecting the center of the monitoring image and the point corresponding to the reference image acquisition position 404 or the point corresponding to the predicted position. The straight line can also be obtained by projecting a straight line connecting the lowermost end and the uppermost end of the object model 400 virtually arranged in the monitoring space 70 onto the monitoring image. Incidentally, in the monitoring space 70, the height h direction is the vertical direction.

  Further, the division into small areas may be performed by dividing the height of the small area as a fixed value, dividing the number of small areas as a fixed value, or dividing the scale on the monitoring image. Alternatively, it may be divided on a scale on the object model 400.

Object information updating unit 528, the object whose position is determined by the object determination unit 526, when degree of hiding e _p calculated by the partial image extracting unit 522 in the determined position is below the reference image concealment of E _p The object information 40 is updated with the image information at the current time. Conversely, it does not update the object information 40 if the degree of hiding e _p is greater than the reference image concealment of E _p.

In the update of the object information 40, specifically, the reference image F _p stored as the reference image 401 is overwritten with the partial image f _p extracted by the partial image extraction unit 522 at the determined position. Reference image concealment of E _p stored in the reference picture concealment degree 402 is overwritten by the degree of hiding e _p calculated by the partial image extracting unit 522 in the determined location. Further, the stored content of the reference image acquisition position 404 is overwritten with the determined object position.

  For example, as shown in FIG. 7, when the image of the target object shown in the monitoring image changes in order of the images 100, 101, 102, 103, and 104, object information based on the image 101 having the smallest concealment degree is stored. In this case, when the image 104 is obtained, if the image 103 one hour ago is stored as a reference image, the similarity of the legs is reduced. However, according to the image monitoring apparatus 1 of the present embodiment, The image 101 is stored as a reference image, which can prevent a decrease in similarity. That is, since the object information 40 when the degree of concealment was the smallest in the past is held, even if the concealed state changes and the concealed area expands, the information of that part is lost from the reference image 401. Can be prevented. Therefore, object determination can be performed using the maximum information acquired in the past, and tracking failure due to concealment can be minimized.

  Further, the object information updating unit 528 has an object image extracted by the object image extracting unit 50 when there is an object image that does not exist in any of the positions of the target objects determined by the object determining unit 526. New object information 40 relating to the image is created. On the other hand, when there is a target object determined to have no object position by the partial image extraction means 522, the object information 40 of the target object is deleted.

  The abnormality determination unit 54 analyzes the movement trajectory 41 stored in the storage unit 4 and determines that there is an abnormality if there is an object (suspicious person) staying for a long time, and outputs an abnormality signal to the output unit 6. . Specifically, the number of position data included in each of the movement trajectories 41 is counted, and if there is a movement trajectory 41 including position data of a predetermined number or more (for example, equivalent to 10 minutes or more), it is determined that there is an abnormality.

  The output unit 6 includes a communication circuit that is connected to an external device such as a center device through a communication line and transmits an abnormal signal to the external device.

  Next, the operation of the image monitoring apparatus 1 will be described. FIG. 8 is a schematic flowchart of the operation of the image monitoring apparatus 1.

  When the administrator turns on the power after confirming that the monitoring space is unattended, each unit and each means of the image monitoring apparatus 1 starts operating and predetermined initialization is performed (S10). At this time, the control unit 5 stores the monitoring image from the imaging unit 2 in the storage unit 4 as a background image.

  Subsequently, the control unit 5 checks whether or not the imaging condition 42 is set in the storage unit 4 (S15). If the imaging condition 42 is not set, the control unit 5 receives and inputs the imaging condition 42 from the operation unit 3 by the administrator. The imaging condition 42 is stored in the storage unit 4 (S20). Thereafter, each time a monitoring image is newly input from the imaging unit 2 to the control unit 5, a series of processes from S25 to S75 is repeated.

  First, the control unit 5 acquires a monitoring image newly captured by the imaging unit 2 (S25), and the object image extraction unit 50 of the control unit 5 uses the acquired monitoring image as a background image stored in the storage unit 4. The object image (person image) is extracted in comparison with (S30).

  In the subsequent object position determination process, the tracking unit 52 of the control unit 5 determines the current position of the target object by identifying the target object image represented by the reference image 401 in the monitoring image (S35).

  FIG. 9 is a schematic flowchart of the object position determination process S35, and the process will be described using this. The position predicting means 520 of the control unit 5 predicts the current position of the target object from the movement trajectory 41 of the target object, and sets a plurality of predicted positions for each target object (S350).

  The tracking unit 52 obtains the distance between the predicted position (central position of the distribution) of each target object and the imaging unit 2, sets the target objects in order from the closest target object (S351), and sets the target object to be processed. Processes S352 to S385 are performed.

  Processes S352 to S384 are a loop process for a plurality of predicted positions of the target object selected as a process target. The predicted positions are sequentially set (S352), and processes S353 to S380 are performed for the set predicted positions. Done.

First, the partial image extraction unit 522 of the control unit 5, extracts a partial image f _p corresponding to each component by projecting by arranging the object model 400 to predict the position of the processing target on the imaging plane 71 of the imaging unit 2 and (S353), and calculates the degree of hiding _{e p} for each of the extracted partial image _{f p} (S354).

Then, the object judging unit 526 of the control unit 5 calculates the reliability _{g p} calculated degree of hiding _{e p,} and a reference image concealment of _{E p} (4) is substituted into equation at S354 (S355) .

Subsequently, the object judging unit 526 of the control unit 5 compares the partial image _{f p} extracted by the processing S353 and the reference image _{F p} to calculate the partial similarity _{u p} (S360).

  FIG. 10 is a schematic flowchart of the partial similarity calculation process S360, and the process will be described using this. The object determination unit 526 of the control unit 5 sequentially sets each component as a processing target (S361), and performs processing S361 to S370 for the processing target component.

First, the object determination unit 526 of the control unit 5 calculates a distance d _R between the reference image acquisition position 404 and the installation position of the imaging unit 2 stored as a part of the imaging condition 42 (S362). Further, a distance d _I between the predicted position set as the processing target in step S352 of FIG. 9 and the position of the imaging unit 2 is calculated (S363), and the distance d _R and the distance d _I are compared. (S364).

If the distance _{d R} is the distance _{d I} or less (YES in S364), partial towards the image _{f p} is projected onto the image of the reference image _{F p} corresponding to the predicted position as a low resolution, (S365), morphism It calculates a common region of the shadow image and a partial image _{f p} (S366).

On the other hand, if distance d _R is larger than distance d _I (NO in S364), partial image f _p is projected onto an image corresponding to reference image acquisition position 404, assuming that reference image F _p has a lower resolution ( S367), calculates a common region of the reference image _{F p} and the projection image (S368).

Thus the common area is calculated, to calculate the partial similarity u _p by comparing the color histograms in the common region (S369).

  When the above process ends for all components (YES in S370), the process proceeds to process S380 in FIG.

  Returning to the flowchart of FIG. 9, the continuation of the object position determination process will be described.

Object determining means 526 of the control unit 5, the partial similarity u _p calculated in partial similarity calculating process S360, weighted addition by the reliability g _p calculated in the process S355, calculates the integrated similarity U (S380). When the integrated similarity U is calculated for all predicted positions set for the object to be processed in this way (YES in S381), the object determination unit 526 of the control unit 5 selects and selects the maximum integrated similarity U. It is determined whether or not the integrated similarity U is equal to or greater than a preset reference value (S382).

  If it is equal to or greater than the reference value (YES in S382), the image information existing at the predicted position where the maximum integrated similarity U is calculated is identified as the image of the object to be processed, and the predicted position is the object of the object to be processed. The position is determined (S383).

  On the other hand, if it is less than the reference value (NO in S382), the object to be processed is not identified, the object to be processed is lost, and the object position is determined to be unknown (S384).

  When the above process is completed for all objects (YES in S385), control unit 5 returns the process to process S40 in FIG.

  The continuation of the entire process will be described with reference to FIG. 8 again.

  When the object position determination process ends, the object determination unit 526 of the control unit 5 additionally stores the determined object position in the movement locus 41 of the storage unit 4 (S40). If there is an object image that does not exist in the object position determined in the object position determination process S35 in the object image extracted in the process S30, the object locus indicated by the object image is used as information on the new object. 41 is stored.

  Subsequently, the object information updating unit 528 of the control unit 5 updates the object information 40 in preparation for the object position determination process at the next time (S45).

  FIG. 11 is a schematic flowchart of the object information update process S45, and the process will be described using this. The object information update unit 528 of the control unit 5 sequentially sets each object represented by the object information 40 as a processing target (S450), and performs processing S451 to S456 for the processing target object.

  First, the object information updating unit 528 of the control unit 5 confirms whether or not the object to be processed is the object whose object position is determined in the process S383 of FIG. 9 (S451), and the object position is determined. If it is not an object, it is determined that the object is a lost object, and the object information 40 of the object is deleted (S452).

On the other hand, if the object position is determined, the object information update unit 528 of the control unit 5 sequentially sets each component as a processing target (S453), and the processing target component p is S354 in FIG. comparing the calculated degree of hiding _{e p} a reference image concealment of _{E p} at (S454), (YES at S454) if hiding of _{e p} is less than the reference image concealment of _{E p,} components to be processed The object information 40 is updated for p (S455).

That is, the processing object information updating means 528 of the control unit 5 at S455, the reference image F _p, in the determined object position rewritten to the partial image f _p extracted by the process S353 of FIG. 9, the reference image concealment of rewritten E _p the degree of hiding e _p, the reference image acquisition position 404 rewrites the determined object position. In this way, by holding the object information when the degree of concealment is the smallest, it is possible to perform highly accurate object identification regardless of the size of concealment.

  When the above update is completed for all components (YES in S456) and for all target objects (YES in S457), the process proceeds to process S458.

  In process S458, the object information update unit 528 of the control unit 5 includes an object image that does not exist in any of the object positions determined in S383 of FIG. 9 among the object images extracted in process S30 of FIG. Check if it exists. If there is a corresponding object image, it is determined that the object has newly appeared, and new object information 40 is created from the image information of the object image (S459).

  That is, a plurality of search positions are set in the object image, the whole body of the object model 400 is projected at each search position, and the shapes of the projection image and the object image are compared. As a result of the comparison, the search position with the most matching shape is determined as the object position.

Then, the determined object position is given to the partial image extracting means 522 to perform extraction of the partial image and calculation of the concealment degree, and each extracted partial image is referred to as the reference image F _p , and each calculated concealment degree is referred to as the reference image. The concealment degree E _p and the object position are stored in the storage unit 4 as the reference image acquisition position 404, respectively.

  At this time, the height and width of the object model 400 are set within a predetermined range based on the height and width of the standard target object, and a search is performed. Is the height h and width w of the object model 400 of new object information.

  When the object information update process ends in this way, the control unit 5 returns the process to step S55 in FIG.

  The continuation of the entire process will be described with reference to FIG. 8 again.

  When the object information update process is finished, the control unit 5 confirms whether or not all objects have disappeared (S55). When all the objects have disappeared (YES in S55), the object image extraction means 50 of the control unit 5 updates the background image stored in the storage unit 4 with the monitoring image (S60). Then, the control part 5 returns a process to S25.

  On the other hand, when one or more objects exist (NO in S55), abnormality determination unit 54 of control unit 5 analyzes movement locus 41 and determines whether or not an object having abnormal movement locus 41 exists. If there is an abnormality (YES in S70), the abnormality determination means 54 outputs an abnormality signal including the position of the object determined to be abnormal and the monitoring image to the output unit 6 (S75). The output unit 6 to which the abnormal signal is input transmits the signal to the center device. The center device displays information included in the abnormality signal to notify the monitoring person of the abnormality, and the monitoring person who sees the display takes measures such as removing a suspicious person.

  On the other hand, when abnormality is not determined (NO in S70), process S75 is skipped.

  When the above process ends, the control unit 5 returns the process to S25 again.

  In the above-described embodiment, the example in which the partial image extraction unit 522 calculates the concealment degree is shown, but instead, the non-concealment degree may be calculated. The non-concealment degree can be calculated for each component as a ratio (S2 / S1) of the area of the observation region of the component to the area of the entire region of the component. In this case, the object information update unit 528 may update the object information 40 when the non-concealment degree is high.

  In the above-described embodiment, the candidate position having the maximum integrated similarity U is determined as the object position. In another embodiment, the object position is calculated by averaging a plurality of candidate positions for which the integrated similarity U satisfies the determination criterion or by averaging the candidate positions using the integrated similarity U.

  The image feature amount is not limited to a color histogram, and various image feature amounts such as a luminance histogram, an edge intensity histogram, an edge direction histogram, or a texture can be employed. Two or more image feature amounts may be combined.

It is a block block diagram of the image monitoring apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows conversion between the object model arrange | positioned in the monitoring space defined by an XYZ coordinate system, and the projection image on an imaging surface. It is explanatory drawing of the process of a partial image extraction means. It is a schematic diagram explaining the specific example which carries out coordinate conversion of the partial image on the projection image obtained by the object model arrange | positioned at the reference image acquisition position similarly to a reference image. It is a schematic diagram explaining the specific example which carries out coordinate conversion on the projection image obtained by the object model which has arrange | positioned the reference image to the candidate position similarly to the partial image. It is a schematic diagram explaining the example which backprojects a partial image and a reference image to the object model each arrange | positioned in the monitoring space, and compares a partial image and a reference image between these object models. It is a typical monitoring image showing the change of the concealment degree in the projection image according to the change of the position of the target object in the monitoring space. It is a schematic flowchart of operation | movement of an image monitoring apparatus. It is a general | schematic flowchart of an object position determination process. It is a general | schematic flowchart of a partial similarity calculation process. It is a general | schematic flowchart of an object information update process.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Image monitoring apparatus, 2 imaging part, 3 operation part, 4 memory | storage part, 5 control part, 6 output part, 40 object information, 41 movement locus, 42 imaging conditions, 50 object image extraction means, 52 tracking means, 54 abnormality determination Means, 400 object model, 401 reference image, 402 reference image concealment degree, 404 reference image acquisition position, 520 position prediction means, 522 partial image extraction means, 526 object determination means, 528 object information update means.

Claims (4)

A projection condition for a surveillance image obtained by perspective projection of the surveillance space, a partial model representing a three-dimensional shape for each of a plurality of constituent parts of the target object to be monitored, and an arrangement relationship between the partial models. A storage unit for storing an object model, a reference image for each component of the target object, and a reference image concealment degree;
The monitoring image and a position in the monitoring space are input, the object model is virtually arranged at the position, and perspective projection is performed based on the projection condition, and for each partial model, the projection image The partial model visible region excluding the concealed region by the other partial model positioned in front by perspective projection and the concealment degree of the projected image by the concealed region are obtained, and the partial image visible region from the monitoring image is obtained. Partial image extraction means for extracting a corresponding portion as a partial image for each component of the target object;
An object that compares the partial image at the candidate position where the target object can exist and the reference image with respect to a plurality of candidate positions, and determines an object position where the target object exists based on the candidate positions that have been compared and matched A determination means;
Object information for updating the reference image and the reference image concealment degree with the concealment degree and the partial image at the object position for the constituent part in which the concealment degree at the object position is equal to or less than the reference image concealment degree Update means;
An image monitoring apparatus comprising: The image monitoring apparatus according to claim 1,
The object determination means includes a reference partial image obtained by mapping the partial image on the basis of the projection condition on a reference projection image obtained by the object model arranged at a reference position, and the reference image on the reference projection image. An image monitoring apparatus, comprising: obtaining a common area in which a reference reference image mapped with each other overlaps and determining the object position by comparing the reference partial image and the reference reference image in the common area. The image monitoring apparatus according to claim 2,
The object determination unit sets the reference position as a distance that is farther from a viewpoint position, out of the candidate position and the position of the object model that is arranged when the reference image is acquired. Monitoring device. In the image monitoring apparatus according to claim 2 or 3,
The object determining means sets a plurality of horizontal strip regions in the height direction of the partial model, divides the common region into a plurality of small regions corresponding to the horizontal strip regions, And comparing the reference partial image with the reference reference image.

Images