JP2012079340A - Suspicious object detection system, suspicious object detection device, suspicious object detection method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a suspicious person detection system capable of integrally using results of image processing methods having different properties, and especially performing effective suspicious person detection processing according to a security level.SOLUTION: The suspicious person detection system in the embodiment includes: multiple cameras for imaging a monitoring object within a monitoring range; multiple image processing parts which are means for performing image processing of a video output from each of the multiple cameras, having different detection properties; a detection part for detecting behaviors of the monitoring object by using either one of image processing results from the image processing parts or the integrated processing result generated by integrating all the image processing results; and a determination part for determining whether the monitoring object is a suspicious person or not based on behavior information output by the detection part. The multiple image processing parts include simple eye image processing part(s) for performing simple eye image processing of the video output from each of the cameras and stereo image processing part(s) for performing stereo image processing of the video output from each of the cameras.

Description

  Embodiments described herein relate generally to a suspicious person detection system that uses video from a camera.

  In recent years, a monitoring system for monitoring a suspicious person or the like using a video (moving image) taken by a video camera has been developed. Various methods have been proposed for the monitoring system, and an intruder monitoring system for tracking an object by processing images obtained from a plurality of cameras has been proposed. This intruder monitoring system includes means for selecting one of three-dimensional recognition processing for tracking an object by stereo image processing and two-dimensional recognition processing for tracking an object by monocular image processing.

JP 2003-61075 A

  In general, when detecting a suspicious person or an intruder using the result of image processing of a video from a camera, the method of performing monocular image processing as an image processing method is accurate for the person being monitored. It is difficult to detect (calculate) position information and a walking trajectory. On the other hand, in the method of performing stereo image processing, accurate position information and walking trajectory can be detected, but the monitoring range tends to be narrower than the monocular image processing method.

  Therefore, a system that utilizes the advantages of the detection characteristics of the monocular image processing method and the stereo image processing method and uses each image processing result in an integrated manner, and in particular, realizes effective suspicious person detection performance according to the security level. Construction of is desired.

  Accordingly, an object of the present invention is to provide a suspicious person detection system capable of performing effective suspicious person detection processing according to the security level by using the results of image processing methods having different characteristics in an integrated manner. is there.

  A suspicious person detection system according to an aspect of the present invention includes a plurality of cameras that shoot a monitoring target within a monitoring range, and a unit that performs image processing on images output from the cameras, and has a plurality of different detection characteristics The image processing means, a detection means for detecting the behavior of the monitoring target using any one of the image processing results from the image processing means, or an integrated processing result obtained by integrating the image processing results, and the detection Determination means for determining whether or not the monitoring target is a suspicious person based on behavior information output from the means, and the plurality of image processing means are configured to output images output from the cameras as a single eye. The image processing apparatus includes a monocular image processing unit that performs image processing, and a stereo image processing unit that performs stereo image processing on video output from each of the cameras.

The block diagram which shows the principal part of the suspicious person detection system regarding 1st Embodiment. The block diagram for demonstrating the structure of the image process part regarding this embodiment. The figure for demonstrating the specific application example of the suspicious person detection system regarding this embodiment. The flowchart for demonstrating the process sequence of the suspicious person detection system regarding this embodiment. The figure for demonstrating the case where it applies to the wide monitoring range in the specific application example of the suspicious person detection system regarding this embodiment. The block diagram which shows the principal part of the suspicious person detection system regarding 2nd Embodiment. The flowchart for demonstrating the process sequence of the suspicious person detection system regarding 2nd Embodiment.

  Embodiments will be described below with reference to the drawings.

[First Embodiment]
FIG. 1 is a block diagram illustrating a main part of a suspicious person detection system according to the present embodiment.

  As shown in FIG. 1, this system includes a plurality of video cameras (here, three cameras) 10 </ b> A to 10 </ b> C, a suspicious person detection device 11, and a monitoring processing device 15. Each of the cameras 10A to 10C is installed at each location within the monitoring range, and transmits an image obtained by photographing to the suspicious person detection device 11.

  The suspicious person detection device 11 includes an image processing unit 12, a controller 13, and an output unit 14. The image processing unit 12 includes a plurality of image processing units (here, four types of image processing units) 20A to 20D that use image processing methods having different detection characteristics. Each of the image processing units 20A to 20D performs image processing on each video image transmitted from the cameras 10A to 10C by using a respective image processing method.

  The controller 13 uses each image processing result output from the image processing unit 12 to detect a behavior of a person (monitoring target) existing in the monitoring range, and behavior information obtained by the detection function. And a determination function for finally determining whether or not the person is a suspicious person indicating a suspicious behavior. As will be described later, the controller 13 selectively uses each of the image processing results or the integrated result of integration according to the monitoring mode (security level or the like).

  The output unit 14 outputs the determination result determined by the controller 13, that is, information indicating the person who is the monitoring target specified as the suspicious person to the monitoring processing device 15. The monitoring processing device 15 is a device that displays information from the output unit 14 on a screen and executes an alarm process for reporting the presence of a suspicious person.

(Configuration of image processing unit)
Hereinafter, a specific configuration of the image processing unit 12 will be described with reference to FIG.

  As shown in FIG. 2, the image processing unit 12 includes a monocular image processing unit 20A and a stereo image processing unit 20B. The monocular image processing unit 20A performs image processing on images from the cameras 10A to 10C to calculate two-dimensional position information (including coordinate information and time information) in real space, and the two-dimensional position. It has a tracking processing unit 121A that tracks the movement of a person to be monitored using information. That is, the monocular image processing unit 20A generates information such as the presence of a person in the two-dimensional space, movement, background difference, interframe difference, and optical flow.

  On the other hand, the stereo image processing unit 20B performs, for example, stereo image processing on images from the two cameras 10A and 10B, and calculates the three-dimensional position information in the real space, and the three-dimensional position information. The tracking trace processing unit 121B is used to track the movement of a person to be monitored. That is, the stereo image processing unit 20B generates information such as the presence, movement, binocular parallax method, etc. of a person in a three-dimensional space.

  Note that the image processing unit 12 performs, for example, RGB color image processing as an image processing unit having other detection characteristics, and generates an image processing unit 20C that generates information such as hue and saturation to be monitored, and 3 The image processing unit 20D executes processing such as categorization such as 3D model matching based on the shape in the dimensional space.

(Suspicious person detection operation)
Hereinafter, the operation of the system of this embodiment will be described with reference to FIGS.

  FIG. 3 is a diagram illustrating an example of a monitoring range to which the system of the present embodiment is applied. Here, it is assumed that a plurality of monitoring areas 300, 310, and 320 are monitored by two cameras 10A and 10B. The monitoring area 300 is, for example, an entrance of a server room or the like, and entry of persons other than outsiders is strictly restricted, and the monitoring area 300 has a high security level. In addition, the areas 310 and 320 outside the monitoring area 300 are areas where the security level is not so high.

  As shown in FIG. 4, the suspicious person detection apparatus 11 inputs the video image | photographed from each camera 10A, 10B (step S1). In the image processing unit 12, the monocular image processing unit 20 </ b> A performs monocular image processing on each video from the cameras 10 </ b> A and 10 </ b> B and outputs the image processing result to the controller 13. The stereo image processing unit 20 </ b> B performs stereo image processing on the video from the two cameras 10 </ b> A and 10 </ b> B and outputs the image processing result to the controller 13.

  The controller 13 performs suspicious person detection processing on the monitoring areas 310 and 320 having a relatively low security level by using the monocular image processing result from the monocular image processing unit 20A (steps S2, S5, and S5). S4). That is, the controller 13 generates behavior information indicating the behavior of a person from the tracking processing result using the two-dimensional position information from the monocular image processing unit 20A, and whether or not there is a suspicious person based on the behavior information. Determine.

  Here, in general, monocular image processing is inferior in detection accuracy of a person compared with stereo image processing, but an effective monitoring range can be widened. Therefore, for the monitoring areas 310 and 320 having a relatively low security level, detection processing using a monocular image processing result that prioritizes the breadth of the monitoring range over detection accuracy is effective.

  On the other hand, stereo image processing has relatively high human detection accuracy, but on the other hand, the effective monitoring range is narrowed. Therefore, the controller 13 performs suspicious person detection processing using the stereo image processing result for the monitoring area 300 having a high security level as a relatively narrow monitoring range where the imaging ranges of the cameras 10A and 10B overlap. (Steps S2, S3, S4).

  That is, the controller 13 uses the integrated image processing result obtained by integrating the image processing results of the monocular image processing unit 20A and the stereo image processing unit 20B to generate behavior information indicating human behavior, and based on the behavior information. To determine whether there is a suspicious person. As a specific method, as shown in FIG. 2, the controller 13 supplies the three-dimensional position information calculated by the stereo image processing unit 20B to the tracking processing unit 121A of the monocular image processing unit 20A, thereby Recalculate the operations associated with the tracking process that tracks This makes it possible to track the movement of a person with high accuracy.

  Here, the monocular image processing unit 20A executes person extraction based on general background differences, and performs tracking processing by clustering two-dimensional extraction regions. On the other hand, the stereo image processing unit 20B performs tracking processing by clustering three-dimensional extraction regions using the binocular parallax method. In the tracking process in the monocular image processing unit 20A, the controller 13 converts the three-dimensional position information calculated by the stereo image processing unit 20B into two-dimensional information.

  The output unit 14 outputs the determination result determined by the controller 13, that is, information indicating the person who is the monitoring target specified as the suspicious person to the monitoring processing device 15. As a result, the monitoring processing device 15 displays the information from the output unit 14 on the screen or executes an alarm process for reporting the presence of a suspicious person.

  As described above, for the monitoring area 300 having a high security level, for example, in the vicinity of an entrance such as a server room, the integrated processing result obtained by integrating the monocular image processing result and the stereo image processing result is used. The behavior information for detecting the behavior is generated, and it is determined whether or not the person is a suspicious person based on the behavior information. Therefore, although the effective monitoring range is narrowed, it is possible to detect the number of people and the behavior of the person with high accuracy using the stereo image processing result that can improve the detection accuracy of the person. Thereby, a suspicious person's detection process can be performed with the high precision according to a high security level.

  In addition, for the monitoring areas 310 and 320 having a relatively low security level, a detection process for a suspicious person is performed using a monocular image processing result in which the breadth of the monitoring range is given priority over the detection accuracy. Can do.

(Modification)
FIG. 5 is a diagram illustrating an example of a monitoring range to which the system of the present embodiment is applied, and is a case where the present invention is applied to a wide monitoring range as compared with the example illustrated in FIG. Here, it is assumed that a plurality of monitoring areas 400, 410, 420, 430, 440, and 450 are monitored by four cameras 10A to 10D. The monitoring area 430 is, for example, an entrance of a server room or the like, where entry by persons other than outsiders is strictly restricted, and is a high security level. In addition, the monitoring areas 400, 410, 420, 440, and 450 are areas where the security level is not so high. The monitoring area 400 is an area corresponding to the shooting range of the camera 10A. The monitoring area 410 is an area corresponding to the shooting range of the camera 10B. The monitoring area 420 is an area where the imaging ranges of the two cameras 10A and 10B overlap.

  Next, the image processing unit 12 includes image processing units 20C and 20D having other detection characteristics as described above, in addition to the monocular image processing unit 20A and the stereo image processing unit 20B in the present embodiment. The controller 13 uses, for example, the results obtained by integrating the image processing results from the image processing units 20C and 20D, for example, the detected person's height, physique (thin type, chubby type), age, color of clothes, Attributes and features such as clothing, hats, sunglasses, and other wearing equipment may be extracted and used as supplementary information in the judgment of suspicious persons or detection.

  Further, the controller 13 sets a weight for the suspicious person's determination criterion for each image processing result of the image processing unit 12 based on the monitoring status or the instruction information from the operator, and uses this weight information for the final determination process. You can use this method. For example, when the position in human behavior is important as a suspicious person's criterion, the controller 13 weights each image processing result of the monocular image processing unit 20A and the stereo image processing unit 20B. On the other hand, when human characteristics are important as a suspicious person's criterion, the controller 13 weights each image processing result from the image processing units 20C and 20D. Specifically, the controller 13 executes a calculation process for multiplying each image processing result of the image processing unit 12 by a weighting coefficient.

  Further, the controller 13 includes a mechanism capable of canceling the tracking of the person set as the certifier within the monitoring area of the suspicious person. As an authentication mechanism, for example, face authentication or a wireless authentication method is adopted, and time information at which a human is authenticated and information on whether authentication is possible are obtained from an authentication device. The controller 13 cancels the tracking of the person from the result of identifying the person existing at the location of the authentication apparatus, and realizes systematic error report suppression and calculation cost reduction. In the authentication process, if there are a plurality of persons around the authentication apparatus and it is difficult to specify the authenticator, the tracking cancel process is not executed.

  The security level varies depending on the place to be monitored. For example, the security level is generally low in a passage through which a person passes, and the security level is set high in an intrusion prohibited passage. On the other hand, regarding an area that must be protected even in a passage through which a person passes, the grade of attention or warning may be changed even if the same behavior is performed depending on, for example, whether it is in the vicinity thereof. Specifically, for example, when a situation is assumed in which a certain person is stagnant for a certain period of time, the grasped event is matched with the logically set security level area, and the set security level is If found in a low area, alert the administrator. On the other hand, if a discovery is made in an area with a high security level, the administrator can be alerted. In this case, attention and warning can be issued not only to the administrator but also to the parties.

[Second Embodiment]
FIG. 6 is a block diagram illustrating a main part of the suspicious person detection system according to the second embodiment. FIG. 7 is a flowchart for explaining the suspicious person detection operation.

  The system of this embodiment is a structure which has the action analysis apparatus 16, and since it is the same as that of the system shown in FIG. 1 except for this structure, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  The behavior analysis device 16 executes behavior analysis processing based on the behavior information of the person detected by the controller 13, and uses the behavior analysis result and information indicating whether or not the user is actually a suspicious person. In this configuration, learning information in which a suspicious person's determination process is ruled is generated and fed back to the controller 13. Specifically, the behavior analysis device 16 performs behavior analysis processing by a method such as pattern recognition or machine learning (for example, SVM: Support Vector Machine) to clarify a rule that has not been clarified.

  Hereinafter, the processing procedure of the suspicious person detection system of this embodiment will be described with reference to the flowchart of FIG.

  First, the suspicious person detection apparatus 11 inputs the video image | photographed from each camera 10A-10C (step S11). For example, the image processing unit 12 executes integrated image processing that integrates the image processing of the monocular image processing unit 20A and the stereo image processing unit 20B, and outputs the image processing result to the controller 13 (step S12). The controller 13 performs a suspicious person detection process using the image processing result from the image processing unit 12 (step S13). That is, the controller 13 generates behavior information indicating a human behavior using the image processing result, and determines whether or not there is a suspicious person based on the behavior information. Based on the determination result of the controller 13, the monitoring processing device 15 displays on the screen information indicating the person who is the monitoring target identified as the suspicious person, or executes an alarm process for reporting the presence of the suspicious person.

  On the other hand, the behavior analysis apparatus 16 according to the present embodiment executes a behavior analysis process based on human behavior information from the controller 13 (step S14). If the behavior analysis device 16 can describe the suspicious person determination process as a rule using the behavior analysis result and information indicating whether or not the person is actually a suspicious person, the behavior analysis device 16 displays the ruled determination logic. The learning information is set in the controller 13 (steps S15 and S16). If it cannot be described as a rule, the behavior analysis device 16 sets the behavior identification function in the controller 13 (steps S15 and S16).

  Specifically, for example, information indicating the suspicious person's waving behavior is acquired from the controller 13 as human behavior information. Based on this information, if it can be described as a suspicious person after 10 round trips, the behavior analysis device 16 learns the rule of “when 10 round trips” as the suspicious person's decision logic to the controller 13. Set as information. On the other hand, if it cannot be described as a rule, the behavior analysis device 16 sets the controller 13 with the suspicious person's wandering behavior identification function for the controller 13.

  When the operator designates a person on the screen by a GUI input operation, the behavior analysis apparatus 16 may set a new ruled determination logic in the controller 13 as learning information. Thereby, when it is desired to find a person showing a specific behavior, learning information indicating the discrimination logic can be automatically created on the screen and added to the controller 13.

For example, when you want to find a person who turns to the right in the passage, if you specify a person who turns to the right from the video that has been captured and accumulated, learning information that shows the discrimination logic of `` turn right in the passage '' is automatically created. Can be added to the controller 13. As a result, the controller 13 can later find a person turning right on the passage based on the behavior information. In the first and second embodiments described above, the setting of the security level is the place to be monitored. Depending on the security level, the sensing sensitivity of suspicious behavior may be controlled according to the security level. For example, in the case where a person is determined to be a suspicious person when staying there for a certain period of time, matching is performed between the grasped event and the logically set security level area. In an area where the security level is low, for example, if the user stays for 10 seconds, attention is urged. In addition, when a discovery is made in an area with a high security level, for example, the administrator can be alerted in 5 seconds. In this case, attention and warning can be issued not only to the administrator but also to the parties. Regarding sensitivity, it may be possible to set each parameter for determining the moving distance and the intensity of behavior.

  Furthermore, in the first and second embodiments described above, when there is a person who is equipped with a sound pressure sensor or a voice recognition sensor and generates a strange voice, in addition to sensitivity control depending on the security level, Sensing sensitivity of suspicious behavior to humans may be controlled. When voice and human beings cannot respond, sensing sensitivity may be controlled as a region, or the sensitivity of all regions may be increased.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

10A to 10D ... Video camera, 11 ... Suspicious person detection device, 12 ... Image processing unit,
13 ... Controller, 14 ... Output unit, 15 ... Monitoring processing device,
16 ... Behavior analysis device, 20A ... Monocular image processing unit, 20B ... Stereo image processing unit,
300, 310, 320 ... monitoring area,
400, 410, 420, 430, 440, 450 ... monitoring area.

Embodiments described herein relate generally to a suspicious object detection technique using a camera image.

In general, a monocular image is used as an image processing method when detecting a suspicious person, an intruder, or the like (hereinafter sometimes referred to as a suspicious object) as a suspicious object using the result of image processing of a video from a camera. In the method of performing the process, it is difficult to detect (calculate) accurate position information and a walking trajectory of a person to be monitored. On the other hand, in the method of performing stereo image processing, accurate position information and walking trajectory can be detected, but the monitoring range tends to be narrower than the monocular image processing method.

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a suspicious object detection system that can use the results of image processing methods having different characteristics in an integrated manner and perform effective suspicious object detection processing according to the security level. is there.

According to the embodiment, the suspicious object detection device is a unit that performs image processing on the video output from the imaging unit, and includes a plurality of image processing units having different detection characteristics, and an image processing result from each of the image processing units. Or a detection unit that detects the behavior of the imaging target using an integrated processing result obtained by integrating the image processing results, and the imaging target is suspicious based on behavior information output from the detection unit. Determination means for determining whether or not it is a target, wherein the plurality of image processing means outputs a monocular image processing means for monocular image processing of the video output from each imaging means, and outputs from each imaging means And a stereo image processing means for performing stereo image processing on the video to be processed .

Embodiments will be described below with reference to the drawings. In addition, this embodiment is a case where a suspicious person is made into a detection object as a suspicious object, and is applicable also in the case of an intruder.

Claims (10)

A plurality of cameras for photographing the monitoring target within the monitoring range;
A means for image-processing video output from each of the cameras, a plurality of image processing means having different detection characteristics;
Detecting means for detecting the behavior of the monitoring target using any one of the image processing results from each of the image processing means or an integrated processing result obtained by integrating the image processing results;
Determination means for determining whether or not the monitoring target is a suspicious person based on behavior information output from the detection means,
The plurality of image processing means include
Monocular image processing means for monocular image processing of video output from each camera;
A suspicious person detection system comprising: stereo image processing means for performing stereo image processing on video output from each of the cameras. When the monitoring range is divided into a plurality of monitoring areas set with different security levels,
2. The suspicious person detection system according to claim 1, wherein the detection unit detects the behavior of the monitoring target using the integrated processing result in a monitoring area in which a specific security level is set. When the monitoring range is divided into a plurality of monitoring areas set with different security levels,
The said detection means detects the behavior of the said monitoring object using the integrated process result of the said monocular image process and the said stereo image process in the monitoring area | region to which the specific security level was set. The suspicious person detection system according to 1. The detection means includes
The behavior of the monitoring target is detected using an integrated processing result of the monocular image processing and the stereo image processing in a monitoring region where the imaging ranges of the cameras overlap within the monitoring range. The suspicious person detection system according to 1. The determination means includes
It has a suspicious person identification function that rules the characteristics of a suspicious person based on human behavior information in advance, and the monitoring target is a suspicious person by using the behavior information from the detection means by the suspicious person identification function. The suspicious person detection system according to claim 1, wherein it is determined whether or not there is any.   The image processing results from the respective image processing means are mutually complemented or fed back to the respective image processing means, and the image processing calculation using the image processing results calculated by the respective image processing means as different parameters is re-executed. The suspicious person detection system according to claim 1, further comprising control means to be executed. The monocular image processing means includes means for calculating two-dimensional position information of the monitoring target, and means for executing tracking processing of the monitoring target using the two-dimensional position information.
The stereo image processing means includes means for calculating three-dimensional position information of the monitoring target, and means for executing tracking processing of the monitoring target using the three-dimensional position information.
The three-dimensional position information calculated by the stereo image processing unit is supplied to the monocular image processing unit, and the monocular image processing unit re-executes the monitoring target tracking process using the three-dimensional position information. The suspicious person detection system according to claim 1, further comprising control means.   Based on the behavior information output from the detection means, the behavior analysis process of the monitoring target is executed, and the behavior analysis result and information indicating whether or not the user is actually a suspicious person are used. 8. The method according to claim 1, further comprising behavior analysis means for generating learning information in which a determination process is made into a rule and reflecting the learning information in the determination means. 9. Suspicious person detection system.   The suspicious person detection system according to any one of claims 1 to 8, further comprising means for executing a notification process when a determination result of the determination means is a suspicious person. . A suspicious person detection method applied to a suspicious person detection system that uses a plurality of cameras that shoot a monitoring target within a monitoring range,
Image processing of the video output from each camera by a plurality of image processing methods of a monocular image processing method and a stereo image processing method having different detection characteristics;
Detecting the behavior of the monitoring target using any one of the image processing results of the image processing methods or an integrated processing result obtained by integrating the image processing results;
And a step of determining whether or not the monitoring target is a suspicious person based on behavior information indicating the behavior of the monitoring target.

Images