JP5101429B2 - Image monitoring device - Google Patents

Description

  The present invention relates to an image monitoring apparatus that detects a target object from a monitoring image obtained by imaging a monitoring space, and more particularly to detection of a target object that generates an image having characteristics similar to a background.

In order to detect an intruder or the like, a technique is known in which an image related to a target object that is a monitoring target is extracted from a monitoring image obtained by imaging the monitoring space, and the target is detected and tracked based on the image information. As this technique, there is a method of comparing a monitoring image with a background image obtained when there is no target object in the monitoring space, and detecting the target object based on the difference area.
JP 2007-201933 A

  However, when there is a portion having a feature similar to the background image in the target object image, the portion is difficult to appear in the difference area, and there is a problem that the detection accuracy of the target object is lowered.

  The present invention has been made to solve the above-described problem, and an object thereof is to provide an image monitoring apparatus that can suitably detect a target object that generates an image having characteristics similar to a background image.

  The image monitoring apparatus according to the present invention includes a first imaging unit that captures a monitoring space from a viewpoint from which a first background image is obtained and outputs the first monitoring image, and a second background image that is different from the first background image. A second imaging unit that images the monitoring space from the obtained viewpoint and outputs a second monitoring image; the first background image; the second background image; and an arbitrary imaging surface on the second imaging unit A storage unit for storing a correspondence rule for associating a gazing point projection image passing through the gazing point onto an imaging surface of the first imaging unit with respect to the gazing point, and the first monitoring image as the first monitoring image. A change area extracting means for extracting a first change area by comparing with a background image, and extracting a second change area by comparing the second monitoring image with the second background image; The gazing point projection image of the gazing point included in the second change region A change area combining means for determining a combined change area by combining the projection area corresponding to the area on the imaging plane of the first imaging unit in which the gazing point projection image is formed and the first change area; And an object detection means for detecting a target object in the monitoring space based on the composite change area.

In another image monitoring apparatus according to the present invention, the change area synthesizing unit extracts a predetermined image feature in the second change area, and a portion having the image feature in an area where the gazing point projection image is formed. Is used as the projection area to determine the composite change area. In the image monitoring apparatus according to the present invention, the imaging direction of the two imaging units is set to have a portion that appears in common in the first monitoring image and the second monitoring image on the surface of the target object. And the change area synthesis means extracts the image feature in an edge area provided along a portion of the outline of the second change area on the front side with respect to the first imaging unit; can do.

  In another image monitoring apparatus according to the present invention, the change area extraction unit extracts a portion where a pixel value exceeds a first reference value as the first change area, and the change area synthesis unit includes A second reference value set to be less than the first reference value is used, and a pixel value between the first monitoring image and the first background image in the region where the gazing point projection image is formed is the second value. A portion that is different from the reference value is defined as the projection region.

  A preferred aspect of the present invention includes a tracking unit that tracks a position of the target object in a predetermined object moving plane, and the first imaging unit is set with an imaging direction facing the object moving plane. The image capturing unit sequentially captures the first monitoring image and the second monitoring image in time series, and the tracking unit detects an area in which the target object is detected at a time before and after the object detecting unit. It is an image monitoring apparatus that compares image information and identifies the target object between the preceding and following times based on the comparison result.

  According to the present invention, by capturing a monitoring image from two directions, even if the target object is similar to the background in one monitoring image, there is a significant difference between the target object and the background in the other monitoring image. Can appear. Therefore, by using one of the change areas extracted from each of these two monitoring images and complementing the other, it is possible to improve the detection accuracy of the target object similar to the background.

  Hereinafter, an unauthorized traffic detection apparatus 1 according to an embodiment of the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. The unauthorized traffic detection device 1 detects an unauthorized passerby, for example, by detecting and tracking a person who is a target object from an image obtained by imaging an entrance of a building.

  FIG. 1 is a schematic block diagram of an unauthorized traffic detection device 1 according to the embodiment. The unauthorized traffic detection device 1 includes an imaging unit 2, an operation unit 3, an authentication unit 4, a storage unit 5, a control unit 6, and an output unit 7.

  The imaging unit 2 is a surveillance camera. The unauthorized traffic detection device 1 is set to a monitoring space including at least an entrance and an authentication unit 4 disposed in the vicinity thereof, and the imaging unit 2 captures the monitoring space by imaging the monitoring space at a predetermined time interval. The fluoroscopically projected images are sequentially output to the control unit 6. Hereinafter, the imaging timing of the imaging unit 2 is referred to as time.

  FIG. 2 is a schematic diagram showing the arrangement of the imaging unit 2 in the present apparatus. 2A is a view of the monitoring space from the side of a person standing in front of the authentication unit 4, and FIG. 2B is a view of the monitoring space from the back of the person standing in front of the authentication unit 4. It is a figure. In this apparatus, as shown in FIG. 2, two imaging units 2-1 and 2-2 having different shooting directions are installed. Here, the monitoring space is represented by an orthogonal coordinate system composed of the X, Y, and Z axes, and the vertically upward direction is set to the positive direction of the Z axis. In the unauthorized traffic detection device 1, in order to grasp the position and movement of a person in the XY plane, one imaging unit 2-1 is basically installed at a height at which a person can be seen from an overhead view. For example, in the present embodiment, the imaging unit 2-1 is installed on the ceiling directly above the position where the person who operates the authentication unit 4 stands with the optical axis 80-1 vertically downward (Z-axis negative direction). The other imaging unit 2-2 is installed so as to capture a direction approximately perpendicular to the imaging direction of the imaging unit 2-1. For example, the imaging unit 2-2 is installed on the wall surface facing the person who operates the authentication unit 4 with the optical axis 80-2 facing in the horizontal direction. Hereinafter, the ceiling camera refers to the imaging unit 2-1, and the wall camera refers to the imaging unit 2-2. An image captured by the imaging unit 2-1 is referred to as a first monitoring image or a ceiling camera image, and an image captured by the imaging unit 2-2 is referred to as a second monitoring image or a wall camera image.

  The ceiling camera image is more suitable for tracking the target object because the target camera is less likely to be occluded (occlusion) than the wall camera image. Further, when a target object such as a person having a three-dimensional shape formed of a curved surface is photographed with the setting where the optical axes of the imaging unit 2 are substantially perpendicular to each other, the portion of the surface of the target object that appears in common in both images is For example, the target object image in the wall camera image is unevenly distributed at the edge near the ceiling camera. By utilizing this property, it becomes easy to extract the characteristics of the target object from the image. When the imaging unit 2 automatically adjusts exposure and image quality in response to a change in illumination, the image brightness and color balance of the plurality of imaging units 2 are synchronized and controlled to be the same value. For example, it is preferable from the viewpoint of facilitating the process of detecting the target object based on the comparison of both images.

  The operation unit 3 is an input unit for the administrator to perform various settings on the control unit 6 and is, for example, a user interface device such as a touch panel display.

  The authentication unit 4 is an authentication device such as a card authentication device, a biometric authentication device, or a personal identification number authentication device, and is installed near the entrance / exit. The authentication unit 4 is connected to the control unit 6 and an electric lock (not shown) installed at the entrance and exit from a person who is permitted to pass through the entrance and exit from the biometric information such as card ID, fingerprint, voice, face, iris, etc. Alternatively, when unique information such as a password is acquired, an authentication signal indicating successful authentication is output to the control unit 6 and the electric lock.

  The storage unit 5 is a storage device such as a RAM, a ROM, and a hard disk, and is connected to the control unit 6. The storage unit 5 stores programs and data used by the control unit 6. Data stored therein includes a background image 50 and a projection rule 51.

  The background image 50 is a background image captured from each viewpoint of the plurality of imaging units 2. The background image 50 is compared with an image (monitoring image) captured by the imaging unit 2, and a portion that differs between the two images is extracted as a change region by the target object. In the present apparatus, the two image capturing units 2-1 and 2-2 capture the monitoring space from different directions, and thus the background images are basically different from each other.

  Specifically, the floor is mainly imaged in the background image 50-1 from the viewpoint of the ceiling camera, and the wall is mainly imaged in the background image 50-2 from the viewpoint of the wall camera. For example, different colors are used on the floor and walls in a general building, and images that can be easily distinguished from each other based on the color are obtained as the background images 50-1 and 50-2. For example, if the clothes of a person existing in the surveillance space are similar in color to the floor, the clothes part of the person may not be extracted as a change area even if the ceiling camera image is compared with the background image 50-1. However, if the wall camera image is compared with the background image 50-2, the clothes portion of the person can be extracted as the change area. As described above, by using a plurality of background images 50 having different viewpoints, even if it is difficult to extract a change area related to a part or all of the target object at any viewpoint, the corresponding part at another viewpoint. Can be extracted.

  The projection rule 51 projects a projection line passing through the gazing point onto an imaging surface of the ceiling camera (imaging unit 2-1) with respect to an arbitrary gazing point on the imaging surface of the wall camera (imaging unit 2-2). This is a correspondence rule for associating viewpoint projection images. Specifically, one pixel in the monitoring image by the wall camera is associated with one line called an epipolar line as a gazing point projection image obtained by projecting a projection line passing through the pixel onto the imaging surface of the ceiling camera. The projection rule 51 includes camera parameters including external parameters such as an installation position and an imaging direction of each imaging unit 2 in an actual monitoring space, and internal parameters such as a focal length, an angle of view, the number of pixels, and lens distortion of each imaging unit 2. It can be set as a program of a coordinate conversion function applied to a camera model such as a known pinhole camera model. The camera parameters are obtained by actually measuring and the like, and are input from the operation unit 3 and stored in the storage unit 5 prior to operation of the apparatus.

  In general, a projection line defined in a monitoring space that is a three-dimensional space is associated with a pixel (gaze point) on the imaging surface of the wall camera, or a line (epipolar line) on the imaging surface of the ceiling camera. In this specification, an operation for associating a gazing point with an epipolar line is also referred to as a projection.

  As will be described later, by repeating the generation of epipolar lines based on the projection rule 51 for pixels in the change area extracted from the monitoring image of the wall camera, an area having a positional relationship corresponding to the change area viewed from the wall camera (Hereinafter referred to as a projection area) can be defined in the monitoring image of the ceiling camera.

  Hereinafter, the projection from the wall camera image to the ceiling camera image will be further described with reference to FIG. 3A is a wall camera image, FIG. 3B is a ceiling camera image, and FIG. 3C is a schematic diagram showing three-dimensional coordinates of the monitoring space. For simplicity, the wall camera image and the ceiling camera image are obtained by correcting the lens distortion using the distortion parameter of each camera, and description will be made assuming that a pinhole camera model can be applied.

  A wall camera is arranged at the origin 84, and a right-handed XYZ coordinate system is set with the optical axis direction of the wall camera as the Y axis and the vertical upper direction as the Z axis. Therefore, the X axis is set to the left from the monitoring area toward the wall camera. In the coordinate system, the optical axis of the wall camera is in the direction of the vector [0, 1, 0]. The coordinates of the ceiling camera are [0, Yc, Zc] (position 83 in FIG. 3C), and the optical axis of the ceiling camera is in the direction of the vector [0, 0, −1]. The wall camera image is a plane defined by XZ coordinates, and the positive direction of the Z axis is defined as upward in the image. The ceiling camera image is a plane defined by XY coordinates, and the positive direction of the Y axis is defined as the upward direction in the image.

  The coordinates of an arbitrary gazing point 81 on the wall camera image are [u, v] (the center is the image center), the number of horizontal pixels of the wall camera image is 2 × W, and the number of vertical pixels is 2 × H. And Also, the horizontal field angle of the wall camera is 2 × α, and the vertical field angle is 2 × β. At this time, when the gazing point 81 is projected back onto the monitoring space, a straight line as a line of sight is obtained. If the vector on the straight line starting from the origin is V1, the X direction component of the angle formed by the vector V1 and the Y axis is arctan (u / W · tanα) (hereinafter referred to as θu), and the Z direction component is arctan (v / H · tanβ) (hereinafter referred to as θv). Therefore, the vector V1 can be expressed as [tan θu, 1, tan θv].

  If the end point of a vector obtained by multiplying the vector V1 by a factor T is P1, the coordinates [XP1, YP1, ZP1] of P1 are expressed as [T · tanθu, T, T · tanθv] using the coefficient T. . This point P1 is projected onto the imaging surface of the ceiling camera. For simplicity in the following calculation, the internal parameters of the ceiling camera are the same as those of the wall camera. The angle between the vector from the ceiling camera to the point P1 and the optical axis vector [0, 0, -1] of the ceiling camera is such that the X-direction component is arctan [XP1 / (Zc-ZP1)] (hereinafter, θuc), Y The direction component is calculated as arctan [(YP1-Yc) / (Zc-ZP1)] (hereinafter, θvc). Therefore, the coordinate P1c on the image when the point P1 is projected onto the ceiling camera image is obtained as [tan θuc / tan α · W, tan θvc / tan β · H], where the image center is the origin.

  Here, the end point of the vector obtained by changing the coefficient T and multiplying the vector V1 by a constant is set as a point P2, and similarly, the coordinates on the image when the point P2 is projected onto the ceiling camera image are set as P2c. When the points P1c and P2c obtained by mapping the points P1 and P2 to the ceiling camera image in this way are connected on the ceiling camera image, the epipolar line 82 is obtained. That is, the line segment connecting the point P1 and the point P2 is formed by back projecting the gazing point 81 onto the monitoring space, and the line segment obtained by projecting this line segment onto the ceiling camera image is the epipolar line 82.

  Note that the vector V1 is multiplied by a factor T to set the points P1 and P2, but the Y coordinate of the point P1 becomes the minimum value of the Y coordinate of the range to be monitored, and the Y coordinate of the point P2 is the Y coordinate of the range of the monitoring target. What is necessary is just to set a coefficient so that it may become the maximum value. For example, the coefficient is set so that the Y coordinate of the point P1 is 50 cm and the Y coordinate of the point P2 is 200 cm.

  The control unit 6 is configured using an arithmetic device such as a DSP (Digital Signal Processor) or MCU (Micro Control Unit), and is connected to the imaging unit 2, the operation unit 3, the authentication unit 4, the storage unit 5, and the output unit 7. The The control unit 6 reads the program from the storage unit 5 and executes it, and functions as a change region extraction unit 60, a change region synthesis unit 61, an object detection unit 62, a tracking unit 63, an abnormality determination unit 64, and the like. The control unit 6 processes the image input from the imaging unit 2 to perform detection of the target object, tracking of the target object, detection of unauthorized traffic, and the like, and outputs an abnormal signal to the output unit 7 when unauthorized traffic is detected.

  The change area extraction means 60 compares the first monitoring image input from the imaging unit 2-1 with the background image 50-1, extracts the first change area, and the second monitoring input from the imaging unit 2-2. A second change area is extracted by comparing the image with the background image 50-2, and information on each extracted change area is output to the change area combining means 61.

  The change area extraction means 60 generates the first background image 50-1 using the first monitoring image input prior to the extraction of the change area, stores the first background image 50-1, and stores the first background image 50-1 in the same manner. 2 A second background image 50-2 is generated using the monitoring image and stored in the storage unit 5. Specifically, immediately after the start-up in which the background image 50 is not stored, the first monitoring image and the second monitoring image are directly stored in the storage unit 5 as the background images 50-1 and 50-2, and thereafter at least the target object The background images 50-1 and 50-2 stored in the storage unit 5 are updated by the first monitoring image and the second monitoring image for the area determined to be absent.

  When the first monitoring image is newly input, the change area extraction unit 60 compares each pixel of the first monitoring image with the corresponding pixel of the background image 50-1, and a pixel value is set in advance. A portion that exceeds the first threshold TH1 and is different is extracted as a first change region. Specifically, a difference in luminance value or color component is calculated between the target pixels of the first monitoring image and the background image 50-1, and pixels whose calculated difference is equal to or greater than the first threshold value TH1 are detected and detected. A region in which the pixels whose distances are within a predetermined range is collected as the first change region. Further, by calculating the correlation value instead of the difference and detecting the pixel whose calculated correlation value is less than the first threshold value TH1, the pixels constituting the first change region can be detected.

  Similarly, when the second monitoring image is newly input, the change area extraction unit 60 compares each pixel of the second monitoring image with the corresponding pixel of the background image 50-2, and the first threshold TH1. A group of pixels that are different from each other is extracted as a second change region.

  As the first threshold value TH1, a value that does not erroneously extract background changes caused by shadows or light is obtained and set through a prior experiment. Note that the threshold value for extracting the second change region may not be the same value as the first threshold value TH1, but may be the third threshold value TH3 determined by the same policy as the first threshold value TH1.

  When the first change area, the second change area, and, if necessary, the first monitoring image and the second monitoring image are input, the change area combining unit 61 first includes a plurality of gazing points included in the second change area. Is projected by the projection rule 51 to obtain a gazing point projection image, and a projection area corresponding to an area where the gazing point projection image is formed in the imaging plane of the imaging unit 2-1. Thus, the second change area is associated with a projection area having the same coordinate system as the first change area. The projection area may be obtained simply as the sum of the areas where the gazing point projection images are formed. Details of the predetermined condition will be described later. Next, the change region combining unit 61 combines the obtained projection region and the first change region, determines a combined change region (hereinafter referred to as a combined change region), and outputs the information to the object detection unit 62. . The process for obtaining the composite change area can be a process for obtaining the sum area of the projection area and the first change area. Further, for the purpose of noise removal and shaping, a predetermined number of contraction / expansion processes may be performed on the obtained sum area.

  In this way, by synthesizing the projection area based on the second change area with the first change area, it is possible to supplement part or all of the target object that cannot be extracted from the first monitoring image with the projection area.

  Here, since the directions in which the first monitoring image and the second monitoring image are captured are perpendicular to each other, the contour that faces the gazing point in the direction in which the first monitoring image is captured in the second change region. It can limit to the area | region (henceforth edge area | region) along a line or an outline. By doing so, it is not necessary to project all the pixels in the second change area, so that the processing can be reduced.

  The above-mentioned contour line indicates that the first monitoring point is detected by scanning in the same direction as the direction in which each pixel of the first monitoring image is imaged in the coordinate system of the second monitoring image. Extraction can be performed by sequentially repeating while shifting in a direction perpendicular to the direction in which the image is captured. FIG. 4 is a schematic diagram for explaining the extraction processing of the contour line and the edge region. The hatched portion 90 in FIG. 4A is the second change region extracted from the wall camera image, and the bold line indicates the contour line. To express. Each arrow from the top to the bottom represents the above-described scanning state, and the direction of the arrow is the scanning direction, that is, the imaging direction of the ceiling camera. In the above scanning, the scanning lines are set in parallel to simplify the calculation. However, if the line connecting the imaging unit 2-1 and each pixel is set as a scanning line in the coordinate system of the second monitoring image, Strict contour detection is possible. The edge region is obtained as a region in which the contour line is shifted inward by a predetermined number of pixels. FIG. 4B is an enlarged view of a part of the second change area of FIG. 4A. Each square represents a pixel, and the hatched pixel is shifted inward by one pixel from the boundary line. It represents the edge region.

  Here, the area where the gazing point projection image is formed includes an extra portion related to the background. Therefore, a portion satisfying the following two conditions is set as a projection area target. The first condition is that the image features are the same, and the second condition is that there is a change with respect to the background image 50-1.

  First, the first condition will be described. The image feature of the target object is extracted in the second change region, and the region having the same image feature as the extracted image feature in the region where the gazing point projection image is formed in the first monitoring image is set as the projection region. By doing so, an accurate projection area from which the background portion is removed can be set as a synthesis target. The image feature may be a luminance value, a color component, or the like.

  Specifically, an image feature of a pixel set in the second change area as a gazing point is extracted from the pixels of the second monitoring image. In the first monitoring image, for each pixel existing in the gazing point projection image (on the epipolar line) corresponding to the gazing point, the degree of difference (or similarity) between the image feature of the pixel and the image feature extracted from the gazing point And a pixel having a calculated difference of less than a predetermined value (or a similarity of at least a predetermined value) is selected as a projection region.

  By applying the first condition described here for each gazing point, the search range of the pixels to be synthesized is limited to the epipolar line corresponding to each gazing point, and therefore based on the commonality / similarity of image features. Thus, the pixels of the first monitoring image associated with the gazing point can be obtained with suitable position accuracy.

  Here, since the resolution of the image is limited, the luminance component and the color component of the background and the target object are often mixed in the pixels corresponding to the outline of the second change area in the second monitoring image. . Therefore, the region from which the gazing point is extracted is defined as the edge region described above. In this way, image features that are not mixed with background image components are reliably extracted. Further, limiting the gazing point to the edge region means that when the first monitoring image and the second monitoring image are captured in directions perpendicular to each other, the first monitoring image and the surface of the target object It also means that image features are extracted from only the area captured in both of the second monitoring images. Due to this limitation, the image feature of the portion of the target object (such as the leg portion in FIG. 4A) not captured in the first monitoring image is not extracted. As described above, by limiting the gazing point to the edge region, it is possible to prevent an error in which the background portion is a synthesis target.

  Next, the second condition will be described. Under the second condition, a portion of the region where the gazing point projection image is formed that differs from the first monitoring image and the background image 50-1 with a pixel value exceeding the second threshold value TH2 is a projection region. Is done. Here, the second threshold value TH2 is set looser (lower) than the first threshold value TH1. Specifically, a difference in luminance value or color component is calculated between each pixel of the first monitoring image and a corresponding pixel in the background image 50-1, and the calculated difference is equal to or greater than the second threshold value TH2. Pixels can be detected, and an area composed of the detected pixels can be extracted as a projection area. Further, it is also possible to extract the projection area by calculating a correlation value instead of the difference and detecting a pixel whose calculated correlation value is less than the second threshold value TH2.

  The second threshold value TH2 is set to a value that can eliminate a change due to noise generated in the camera or in the transmission system based on a prior experiment or the like. For example, when the luminance values of the first monitoring image and the second monitoring image are 256 gradations and the projection area is extracted based on the difference between the luminance values, the threshold values are set as TH1 = 20, TH2 = 5, and the like.

  As a result, the region of the target object that is too strict in comparison with the first threshold value TH1 and cannot be extracted is extracted in comparison with the second threshold value TH2. On the other hand, for excessive extraction due to standard relaxation, suppression by the condition that it is included in the region where the gazing point projection image is formed or the above-described condition of the identity of the image feature works. That is, based on the two conditions described above, a portion in which a change due to the target object is extracted from the region in which the gazing point projection image is formed, and the projection region in which the background portion is suitably removed is defined as the first change region. And can be the target of synthesis.

  FIG. 5 is a schematic diagram for explaining a specific example of the process for obtaining the combined change region by the change region combining means 61. The shaded area in FIG. 5A is the second change area extracted from the wall camera image, and the bold line along the boundary facing the ceiling side represents the edge area. 5B and 5C are respectively a ceiling camera image, FIG. 5D is a projection area to be synthesized, FIG. 5E is a first change area extracted from the ceiling camera image, and FIG. (F) represents the composite change region.

One pixel 92 in the edge region is projected to obtain an epipolar line 93 as a gazing point projection image. The luminance value Iw of the pixel 92 is extracted as an image feature, and a pixel that satisfies the expressions (1) and (2) is selected for each pixel on the epipolar line 93 to obtain a region 94. Here, Ic (x, y) is the luminance value of the ceiling camera image, and Icb (x, y) is the luminance value of the background image 50-1. TH3 is a threshold value that defines a range in which the image features can be regarded as the same, and is obtained and set in advance through an experiment or the like.
| Ic (x, y) -Iw | <TH3 (1)
| Ic (x, y) -Icb (x, y) | ≧ TH2 (2)

  When the projection and the selection of pixels according to equations (1) and (2) are repeated for each pixel in the edge region, a projection region 95 to be synthesized is obtained.

  On the other hand, in the example shown in FIG. 5, the person of the target object wears clothes having a color similar to the color of the floor, and as shown in FIG. 5 (e), the first change area 96 is extracted for most of the clothes. It has been damaged. For this reason, the first change region 96 does not satisfy the reference as the target object in both size and shape. However, in the combined change area 97 obtained by combining the projection area 95 with the first change area 96, most of the clothes are supplemented, and both the size and the shape satisfy the standard as the target object, Can be detected.

  When the composite change region is input, the object detection unit 62 detects the target object based on the region, and outputs information on the region where the target object is detected to the tracking unit 63. Specifically, one or both of the size and shape of the composite change area is obtained and compared with a predetermined reference, and the presence of the target object is detected in the composite change area that satisfies the reference. With respect to the shape of the composite change area, for example, the aspect ratio of the composite change area can be obtained as a value representing the shape, and an allowable range for the aspect ratio of the target object can be set as a reference for the shape. In addition, regarding the size of the composite change area, for example, the area of a circumscribed rectangle surrounding the composite change area or the number of pixels of the composite change area is obtained as a value representing the size, and an allowable range as the area or the number of pixels of the target object is obtained. It can be set as a standard for size.

  As described above, by combining the projection area of the second change area extracted from the second monitoring image with the first change area extracted from the first monitoring image, the target object that cannot be extracted from the first monitoring image is synthesized. Since a part or the whole is supplemented by the projection area, a target object similar to the background can also be suitably detected.

  When the information of the detected area of the target object is input, the tracking unit 63 calculates a feature amount such as a color histogram for the area in the first monitoring image, and calculates the storage unit 5 for each tracking object calculated at the previous time. The similarity with the feature quantity stored in the is calculated, and the same object is identified between the current time and the previous time based on the similarity being equal to or greater than a predetermined value. The position of the area is determined as the position of the tracking object at the current time (current position).

  Thus, the position of the target object is tracked by comparing the image information of the areas where the target object is detected at the preceding and following times.

  The tracking unit 63 additionally stores the determined position at the current time in the storage unit 5 for each tracking object. As a result, the movement trajectory (position history) is accumulated for each tracking object. In addition, an attribute that identifies “authenticator” and “non-authenticated person” is associated with the movement locus and managed. Incidentally, the initial value of the attribute is “non-authenticated”. When an authentication signal is input from the authentication unit 4 at the current time, the tracking object whose current position is closest to the position of the authentication unit 4 is detected, and the attribute of the detected movement trajectory of the tracking object is set to “authenticator”. Change to

  The abnormality determination unit 64 determines whether or not there is an abnormality in illegal traffic. If it is determined that there is an abnormality, the abnormality determination unit 64 performs output according to the abnormality. It is confirmed whether or not a movement trajectory whose current time position is in the entrance / exit area and the attribute is “non-authenticated” is stored in the storage unit 5. It is determined that there is an abnormality, and an abnormality signal is output to the output unit 7.

  The output unit 7 is a speaker that outputs a warning sound that warns unauthorized traffic when an abnormal signal is input, a sound output means such as a buzzer, and a report that reports unauthorized traffic via a communication line when an abnormal signal is input. Communication means for sending a signal to an external device such as a center device is included.

  Next, the operation of the unauthorized traffic detection device 1 will be described. FIG. 6 is a schematic process flow diagram of the operation. When the administrator who has confirmed that the monitoring space is unmanned, the respective units and means start operating and predetermined initialization is performed (S10).

  Subsequently, the control unit 6 checks whether or not the projection rule 51 is stored in the storage unit 5, that is, whether or not a camera parameter that is a part of the projection rule 51 is stored (S15). Then, the camera parameter input by the administrator is received from the operation unit 3, and the projection rule 51 is complemented by the camera parameter and stored in the storage unit 5 (S20).

  Thereafter, each time a new image is input from the imaging unit 2 to the control unit 6, the processes of S25 to S60 described below are repeated.

  First, when the control unit 6 acquires the first monitoring image newly captured by the ceiling camera and the second monitoring image newly captured by the wall camera (S25), whether or not the background image 50 is stored in the storage unit 5 is determined. Is confirmed (S30). If the background image 50 is not stored immediately after activation (NO in S30), the change area extraction means 60 of the control unit 6 stores the first monitoring image in the storage unit 5 as the background image 50-1, The second monitoring image is stored in the storage unit 5 as the background image 50-2 and the background image 50 is initialized (S35). After initialization, the process returns to S25.

  On the other hand, if background image 50 is stored (YES in S30), the process proceeds to S40 and the target object detection process is performed. Details of the target object detection process will be described later.

  In subsequent S45, the control unit 6 refers to the result of the target object detection process to check whether or not the target object has been detected. If the target object is not detected (NO in S45), the change area extraction means 60 of the control unit 6 updates the background image 50-1 with the first monitoring image and the background image 50-2 with the second monitoring image. Is updated (S50). After updating, the process returns to S25.

  On the other hand, if the target object is detected (YES in S45), tracking unit 63 of control unit 6 performs the target object tracking process (S55).

  That is, the tracking unit 63 of the control unit 6 compares the detected feature quantity of the target object region with the feature quantity of the tracking object stored in the storage unit 5 to determine which of the tracking objects is the target object detected. And the current position of each tracking object is determined. Then, the movement trajectory of the tracking object is updated with the determined current position, and the feature amount of the tracking object is updated with the detected feature amount of the target object.

  Further, the tracking unit 63 of the control unit 6 confirms the presence or absence of an authentication signal from the authentication unit 4. If the authentication signal is received, the attribute of the tracking object whose current position is closest to the authentication unit 4 is determined as “authenticator”. Change to

  Note that a target object that has not been identified as any tracking object is determined as a new tracking object, and the initial value “non-authenticated person” of the current position, feature amount, and attribute of the target object is newly stored in the storage unit 5. . Further, it is determined that a tracking object that has not been identified as any target object has moved out of the field of view, and the movement locus, feature amount, and attribute of the tracking object are deleted from the storage unit 5.

  When the tracking progresses in this way, the abnormality determination unit 64 of the control unit 6 analyzes the movement trajectory stored in the storage unit 5 and determines the presence or absence of an unauthorized passerby (S60). If it is determined that there is an unauthorized passerby (YES in S60), the abnormality determining means 64 of the control unit 6 outputs an abnormal signal to the output unit 7, and the output unit 7 outputs a warning sound to identify the unauthorized passerby. While threatening, a notification signal to the center device is transmitted (S65). The center device that has received the notification signal performs notification of abnormality, display of a monitoring image, and the like, and a guard who receives the notification performs a coping action.

  When the above process is completed, the control unit 6 returns the process to S25. FIG. 7 is a schematic process flow diagram of the target object detection process S40. The target object detection process will be described with reference to FIG.

  First, the change area extraction means 60 of the control unit 6 compares the first monitoring image with the background image 50-1, extracts the first change area, and compares the second monitoring image with the background image 50-2. A second change area is extracted (S401).

  Hereinafter, the process of combining the second change area with the first change area is performed. The change area combining means 61 of the control unit 6 detects an edge area from the second change area (S402), sequentially sets the pixels in the detected edge area as a gazing point (S403), and about the gazing point. The loop processing of S403 to S411 is performed. The change area synthesis means 61 of the control unit 6 extracts the luminance value Iw of the gazing point (S404), and projects the gazing point according to the projection rule 51 to obtain a gazing point projection image (epipolar line) (S405).

  Subsequently, the change area composing unit 61 of the control unit 6 sequentially sets the pixels in the obtained gazing point projection image as processing targets (S406), and selects the compositing target pixels from the processing target pixels. The loop process of S410 is performed.

  The change area synthesis means 61 of the control unit 6 determines whether or not the luminance value Ic (x, y) of the pixel to be processed and the luminance value Iw of the gazing point satisfy the expression (1) (S407). Difference | Ic (x, y) -Icb (x, y) | between the luminance value Ic (x, y) of the pixel to be processed and the luminance value Icb (x, y) of the corresponding pixel in the background image 50-1 Whether or not satisfies the expression (2) is determined (S408). If the pixel to be processed satisfies the expressions (1) and (2) (YES in S407 and S408), the pixel is set as a composition target (S409).

  When the processing is completed for all the pixels in the gazing point projection image (YES in S410), partial selection of the projection area corresponding to one gazing point in the edge area is completed, and all the pixels in the edge area are finished. When the process is finished (YES in S411), the extraction of the entire projection area is finished.

  Subsequently, the change area synthesizing unit 61 of the control unit 6 calculates the composite change area by synthesizing the projection area thus obtained with the first change area extracted in S401 (S412).

  The composite area is calculated by at least the number of target objects existing in the monitoring space. The object detection means 62 of the control unit 6 confirms whether the shape and size of each composite area is within the reference range, and if within the reference range, determines that the composite change area is due to the target object. (S413).

  When the above process is completed, the control unit 6 returns the process to S45 of FIG.

  In the above-described embodiment, the target object is a person, but the target object is not limited to this, and can be applied to various three-dimensional objects.

  Further, even when the angle formed by the optical axis of the wall camera and the ceiling camera is not vertical, this technique can be suitably applied if the backgrounds are different from each other and the formed angle is smaller than 90 degrees. In such a case, the portion of the surface of the target object that looks common to both cameras becomes large. Therefore, it is more preferable to set the gazing point set in the second change area not only to the pixels in the edge area but also to the pixels in the inner area.

  Further, the projection rule 51 may be stored as a table describing the correspondence relationship between the gaze point calculated in advance and the coordinates of the gaze point projection image.

  As an application example of the present invention, an example of the unauthorized traffic detection device 1 that tracks a detected target object and determines an abnormality according to the tracking result has been shown, but when the target object is detected without tracking The present invention can also be applied to an intrusion detection device or the like that outputs an abnormal signal by determining that an abnormality has occurred.

  Note that the projection area can be extracted even if either one of the processes S407 and S408 is omitted.

  FIG. 8 is a schematic diagram for explaining another example of processing for detecting a person by applying the present invention. FIG. 8A shows the second change region, and FIG. 8B shows the first change region. In the example shown here, the wall is similar to the color of the skin, and the body ( The head (the area surrounded by the dotted line) could not be extracted, while the floor was similar to the color of the clothes and the head (the hatched area) could be extracted in the first change area 101, but the torso This is a case where a part (area surrounded by a dotted line) could not be extracted. FIGS. 8C to 8E are diagrams for explaining other processing examples in this case.

  FIG. 8C is a diagram for explaining a calculation process of a synthesis target that can be used in place of the processes in steps S406 to S410. In this processing, the maximum value Ymax and the minimum value Ymin of the Y coordinate of the first change area 101 are obtained, and the area where the Y coordinate is less than Ymin and the area where the Y coordinate exceeds Ymax from the area 102 where the gazing point projection image is formed. An area obtained by deleting is calculated as a projection area. As a result, a hatched area in FIG. 8C is obtained as a combined change area obtained by combining the projection area and the first change area.

  FIGS. 8D and 8E are schematic diagrams for explaining projection calculation processing and object detection processing that can be used in place of the processing in steps S406 to S410. In the processes according to FIGS. 8D and 8E, processes S406 to S410 are not performed as the projection calculation process, and the area where the gazing point projection image calculated in process S405 exists is directly used as the projection area. That is, in the processes according to FIGS. 8D and 8E, the area 102 and the first change area 101 are combined in the process S412 to obtain a combined change area. This composite change area can be used, for example, for intruder detection without tracking the target object. FIG. 8D shows one example of the process (S413) for detecting the presence of the target object from this composite change area, and FIG. 8E shows another example.

  In the process shown in FIG. 8D, an angle θi for looking at the combined change area is calculated from the position of the second imaging unit 2-2 in the coordinate system of the first monitoring image, and θi is a predetermined value (the target object It is determined that the target object is present if it exceeds the minimum width.

  In the process shown in FIG. 8E, the width W1 of the composite change region is calculated, and it is determined that the target object exists when the calculated width W1 exceeds the predetermined minimum width of the target object. The width W1 is calculated as follows. First, the coordinates of the lower end (foot) of the second change area in the coordinate system of the second monitoring image are obtained, and the second imaging unit 2-2 is obtained using the coordinates and the camera parameters of the second imaging unit 2-2. The distance dw from the target object is calculated. Next, in the three-dimensional coordinate system of the monitoring space, a distance from the position of the second imaging unit 2-2 is dw, and a straight line that is orthogonal to both of the optical axes of the two imaging units is obtained. A straight line La is calculated by projecting onto the coordinate system of one imaging unit 2-1. A length in which the straight line La and the composite change region overlap is obtained, and this is defined as a width W1.

1 is a schematic block diagram of an unauthorized traffic detection device according to an embodiment of the present invention. It is a schematic diagram which shows arrangement | positioning of the two imaging parts in this apparatus. It is a schematic diagram which shows the relationship of the three-dimensional coordinate of a wall camera image, a ceiling camera image, and a monitoring space. It is a schematic diagram explaining the extraction process of an outline and an edge area | region. It is a schematic diagram explaining the specific example of the process which calculates | requires a synthetic | combination change area | region by a change area synthetic | combination means. It is a processing flow figure of an outline of operation of an unauthorized traffic detection device. FIG. 10 is a schematic process flow diagram of target object detection processing. It is a schematic diagram explaining the other process example which detects a person by applying this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Unauthorized traffic detection apparatus, 2 Imaging part, 3 Operation part, 4 Authentication part, 5 Storage part, 6 Control part, 7 Output part, 50 Background image, 51 Projection rule, 60 Change area extraction means, 61 Change area synthesis means, 62 object detection means, 63 tracking means, 64 abnormality determination means.

Claims (5)

A first imaging unit that images the monitoring space from a viewpoint from which the first background image is obtained and outputs the first monitoring image;
A second imaging unit that images the monitoring space from a viewpoint from which a second background image different from the first background image is obtained, and outputs a second monitoring image;
Projecting a projection line passing through the gazing point onto the imaging surface of the first imaging unit with respect to the first background image, the second background image, and an arbitrary gazing point on the imaging surface of the second imaging unit. A storage unit for storing a correspondence rule for associating the gazing point projection image,
A first change area is extracted by comparing the first monitoring image with the first background image, and a second change area is extracted by comparing the second monitoring image with the second background image. Means,
According to the correspondence rule, the gazing point projection image for the gazing point included in the second change area is obtained, and the gazing point projection image in the imaging plane of the first imaging unit is formed according to the area formed. Change area combining means for combining the projected area and the first change area to obtain a combined change area;
Object detection means for detecting a target object in the monitoring space based on the composite change region;
An image monitoring apparatus comprising: The image monitoring apparatus according to claim 1,
The change area synthesizing unit extracts a predetermined image feature in the second change area, and obtains the synthesized change area using the portion having the image feature as the projection area in an area where the gazing point projection image is formed. An image monitoring apparatus characterized by that. The image monitoring apparatus according to claim 2,
The imaging directions of the two imaging units are set so as to have portions that appear in common in the first monitoring image and the second monitoring image on the surface of the target object,
The change area synthesizing unit extracts the image feature in an edge area provided along a portion of the outline of the second change area on the front side with respect to the first imaging unit;
An image monitoring apparatus characterized by the above. The image monitoring apparatus according to any one of claims 1 to 3,
The change area extraction unit extracts a portion where the pixel value is different from the first reference value as the first change area,
The change area synthesis unit uses a second reference value set to be less than the first reference value, and includes the first monitoring image and the first background image in an area where the gazing point projection image is formed. A portion where the pixel value differs between the second reference value and the projection region,
An image monitoring apparatus characterized by the above. The image monitoring apparatus according to any one of claims 1 to 4,
Tracking means for tracking the position of the target object in a predetermined object movement plane;
The first imaging unit is set with an imaging direction facing the object moving plane,
The imaging units sequentially capture the first monitoring image and the second monitoring image in time series,
The tracking unit compares the image information of the area where the target object is detected at the time before and after the object detection unit, and identifies the target object between the time before and after based on the comparison result. thing,
An image monitoring apparatus characterized by the above.

Images