JP2008152586A - Automatic identification monitor system for area surveillance - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve identification accuracy of a monitoring target and to detect also a monitoring target whose behavior is suspicious. <P>SOLUTION: A monitoring target identification part 22 registers monitoring target patterns and identifies the monitoring target from a photographed image by using a pattern matching technique. A noise eliminating part 21 registers noise patterns to be eliminated; and when noise in the video coincides with any of the noise patterns, the noise is eliminated by using the pattern-matching technique. A behavior suspicious pattern registration part 25 registers behavior suspicious patterns of monitoring targets and a behavior suspicious pattern identification part 24 identifies whether the moving action pattern of the monitoring target matches any of the registered behavior suspicious patterns of the monitoring targets, by using an optical flow image processing technique. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an area warning automatic identification and monitoring system using a pattern matching technique and an optical flow technique which are image processing techniques.

  In a conventional area warning automatic identification and monitoring system, an intrusion of a suspicious person, a vehicle, or the like is detected from an image captured using a monitoring camera (not related to a known invention in the literature). Such an area warning automatic identification and monitoring system, for example, installs a surveillance camera at a gate or fence at a pier, a ship or a wharf, extracts video information of an object from video information taken by the surveillance camera, and selects a person or vehicle. When it is detected and identified, an alarm is issued. When the image monitor of the surveillance center displays the video captured by the surveillance camera and the surveillance staff finds a suspicious person or a suspicious vehicle, a warning alert is immediately issued and the police are notified.

  However, in the visual monitoring of surveillance camera images by surveillance personnel, when a small number of surveillance personnel are monitoring a large number of image monitors, all the image monitor screens do not receive attention and overlook suspicious behavior and abnormal behavior. There is a fear. In addition, in the wharf work area where cameras are monitored, sensors may be installed on fences and gates. However, since the sensor alerts are performed by the human beings to be checked by the surveillance staff, they may fail, be cut or wind. There is a problem that it may be falsely reported due to external factors such as natural phenomena. In this case, it is necessary for the monitor to check the image monitor screen every time. Furthermore, an image processing method by pattern matching for extracting video information such as a person and a vehicle from video information is also well known.

  As a patent document describing this type of prior art, an image captured and input without any object entering the monitoring area is obtained for each initial image memory, and the initial image is subdivided into a predetermined size for each area. The distance to the object indicated by the image is stored in the distance map memory, and when the object enters the monitoring area and the monitoring image is obtained in the monitoring image memory, the initial value stored in the initial image memory is stored. For example, a difference process is performed between the image and the image component that does not exist in the initial image is detected, and the moving object image is stored in the moving object image memory. A “monitoring device” is known in which a distance to a moving object is obtained by obtaining distance information of a subdivision area where the moving object is detected with reference to a memory (for example, see Patent Document 1).

  In addition, a block of pixels having a predetermined value included in a digital image obtained by digitizing a difference image between images before and after shooting with respect to the monitoring region is detected, and an area having a predetermined shape including the block in order of increasing area of the detected block. `` Intruder identification system '' that sets and determines the total area of clusters present in this area, and when it is within a predetermined range, the clusters present in the area are recognized as intruder images Is also known (see, for example, Patent Document 2).

Japanese Patent Publication No. 6-095008 (2nd page-3rd page, Fig. 1) Japanese Patent No. 2612662 (2nd page to 3rd page, FIG. 2)

  However, in the image processing method based on the pattern matching described above, even if people and vehicles can be detected, an unspecified number of people and vehicles come and go, and in places where work is performed, suspicious persons and It is very difficult to tell if it is a suspicious vehicle. For example, when climbing or destroying a gate or fence in the wharf area, suspicious or abnormal behavior and normal work can be seen from the surveillance camera images of workers or work vehicles in the wharf. It is not possible to automatically discriminate between actions and alerts.

  In addition, since the technology described in Patent Document 1 issues a warning by determining the size of a moving object, it does not warn by determining the shape of a moving object such as a person, a motorcycle, a car, or a ship. Since the identification accuracy of the monitoring object is low, there is a problem that there are very many false detections that identify even an object that is not the object. In this technique, since the image portion obstructed by the approaching object exists at least at a position farther than the approaching object, the approaching object exists at the position of the shortest distance information obtained from the distance map memory. This is because it is intended to obtain the distance to the object without complicated image processing such as three-dimensional image processing.

  In the technique described in Patent Document 2, it is also possible to detect and detect other objects, and thus behave suspiciously, only by judging only the person based on the area size of the object moving within the designated area. There is a problem that it is not possible to automatically detect the monitoring object, and it is impossible to distinguish a normal worker, a passerby, a work vehicle, and a person who behaves suspiciously. In addition, it is judged by the area size of the object that noises in the video (for example, shaking of the tree, waves) are excluded as unnecessary objects to be monitored. There is.

  Therefore, the object of the present invention is to increase the identification accuracy of people, vehicles, motorcycles, etc., which are the monitoring objects in the monitoring area, maintain the identification accuracy of the monitoring objects, and also detect monitoring objects with suspicious behavior. It is to provide an area warning automatic identification and monitoring system that can be used.

  The area warning automatic identification and monitoring system of the present invention registers a monitoring object pattern, and performs a monitoring object identification unit (22 in FIG. 1) for identifying and processing the monitoring object from a captured image using pattern matching image processing technology. And a non-monitoring object exclusion unit (23 in FIG. 1) that excludes objects other than the monitoring object from the video according to the result of the identification process. The monitored object pattern to be registered is preferably a simulated figure obtained by extracting and connecting the outermost vertices of the image pixels of the monitored object.

  In addition, a roughness pattern to be excluded is registered, and when the roughness in the video matches the roughness pattern, a roughness exclusion unit (21 in FIG. 1) that eliminates it using pattern matching image processing technology. It is provided.

  Furthermore, the behavior suspicious pattern registration unit (25 in FIG. 1) for registering the behavior suspicious pattern of the monitoring target object, and the monitoring target that has been subjected to the registration processing of the movement behavior pattern of the monitoring target that has been identified by the monitoring target identification unit A behavior suspicious pattern identification unit (24 in FIG. 1) that performs an identification process using an optical flow image processing technique that matches an object behavior suspicious pattern, or a movement suspicious pattern of a monitoring target object depending on the result of the identification process 1 is provided with a suspicious behavior reporting section (26 in FIG. 1) that performs a processing for issuing a notification when it matches. The movement behavior pattern of the monitoring object may be obtained by following the transition of blocks or pixels obtained by dividing the video screen.

  The first effect of the present invention is to detect the monitoring object using the image processing technology of pattern matching. Therefore, the identification accuracy of the person, vehicle, motorcycle, etc., which is the monitoring object in the monitoring area is increased. This means that it is possible to prevent the identification of non-target objects.

  The second effect of the present invention is that the identification accuracy of the monitoring target can be increased by about 10 times or more because the image roughness of the image is eliminated by using the image processing technique of pattern matching.

  The third effect of the present invention is that the behavior suspicious pattern is registered, and the coincidence with the behavior movement pattern of the monitored object grasped by using the optical flow image processing technology is checked. This means that it is possible to automatically discriminate between behavior monitoring objects and normal workers, passers-by, and work vehicles, thereby further improving the identification accuracy.

  According to the fourth effect of the present invention, any number of suspicious behavior patterns can be additionally registered, and the accuracy of suspicious person detection can be further increased. In addition, registration of a normal movement pattern (exclusion pattern) can be additionally registered.

  Next, embodiments of the present invention will be described in detail with reference to the drawings.

[Description of configuration]
FIG. 1 is a block diagram showing an area warning automatic identification monitoring system of the present invention. This area warning automatic identification and monitoring system is a system for warning and monitoring the inside of a wharf area, and is composed of a camera 1, an object take-in detection device 10, an object detection device 20, and a display device 30.

  The object take-in detection device 10 is connected to the camera 1 and the object detection device 20, and transmits to the object take-in detection device 10 video information of the state in the wharf area taken by the camera 1. At this time, the object capture detection device 10 converts the analog video signal received from the camera into a digital video signal, specifies the area range to be subdivided and warned, and detects the moving object to detect the video information. Transmit to device 20. The object detection device 20 detects and identifies the object by pattern matching from the video information transmitted from the object take-in detection device 10, detects and identifies the movement trajectory by the optical flow, and transmits it to the display device 30. To do. The display device 30 displays a buzzer sound and a screen when an alarm is issued for an object or a movement locus. Pattern matching and optical flow are well known techniques.

  Referring to FIG. 1, the object capture detection device 10 includes an A / D conversion unit 11, an image processing unit 12, a warning area designation unit 13, and a moving object detection unit 14. The A / D converter 11 converts the analog video signal from the camera 1 into a digital video signal. The image processing unit 12 converts the color image of the digital video signal to black and white (binarization), and further performs a subdivision process by performing correction that makes the outline of the video information stand out. The alert area designating unit 13 designates an area to be alerted in the video information. The moving object detection unit 14 detects a moving object in the alert area and transmits the video information to the object detection device 20.

  The object detection device 20 includes a wobble exclusion unit 21, a monitoring target identification unit 22, a non-monitoring target exclusion unit 23, a behavior suspicious pattern identification unit 24, a behavior suspicious pattern registration unit 25, a behavior suspicious notification unit 26, and an out-of-pattern exclusion unit. 27. In addition, in a memory (not shown) in the object detection apparatus 20, a rough pattern, a monitoring object pattern, a behavior suspicious pattern, and a non-monitoring pattern to be excluded are registered in advance. Note that the roughness pattern, the monitoring target pattern, the behavior suspicious pattern, and the behavior suspicious pattern can be newly registered in the middle of the process in addition to being registered in advance.

  The wrinkle removal unit 21 registers in advance a wobble pattern to be excluded, and removes the wobble pattern (swings, waves, etc.) in the video transmitted from the moving object detection unit 14 if it matches the wobble pattern. To process. The monitoring target identifying unit 22 registers a monitoring target pattern in advance, identifies a monitoring target (a person, a car, a motorcycle, a ship, etc.) from the image after the processing for eliminating the wobble, and performs a non-monitoring target exclusion unit 23. Removes objects other than the monitoring target from the video.

  The suspicious behavior pattern registration unit 25 performs registration processing of the suspicious behavior pattern of the monitoring target in advance. The behavior suspicious pattern identifying unit 24 identifies whether the movement behavior pattern of the monitoring object identified by the monitoring object identifying unit 22 matches the behavior suspicious pattern of the monitored object registered by the behavior suspicious pattern registering unit 25. To process. The suspicious behavior reporting unit 26 performs a reporting process when the movement behavior pattern of the monitored object is matched by the behavior suspicious pattern identifying unit 24. The non-monitoring exclusion unit 27 registers a non-monitoring target pattern in advance, and performs a removal process when the movement behavior pattern of the monitoring target does not match or matches the non-monitoring target pattern in the behavior suspicious pattern identification unit 24.

  The suspicious behavior display unit 31 displays the contents on the screen when the suspicious behavior reporting unit 26 reports. It can also be confirmed by a buzzer sound and lamp lighting. The moving object detection display unit 32 displays the monitoring target on the screen when the monitoring target identification unit 22 identifies the monitoring target.

[Description of operation]
FIG. 2 is a flowchart showing an operation procedure of the area warning automatic identification monitoring system of the present invention configured as described above. The operation of this area warning automatic identification and monitoring system will be described below with reference to FIG. 2 and with reference to FIGS.

  First, the A / D converter 11 of the object take-in detection device 10 converts the analog video signal captured by the camera 1 into a digital video signal (step S1 in FIG. 2). Next, for example, the image processing unit 12 converts the original image of the digital video signal (color) representing the scenery of the wharf into black and white (binarization), and further performs correction to make the contour of the video information stand out. Subdivision processing is performed (step S2 in FIG. 2). This contour correction is performed by inverting the black and white image data, extracting the difference between the shades of the color, emphasizing the dark portion, and eliminating fine noise (such as a thin point).

  The alert area designating unit 13 designates an area to be alerted in such video information (step S3 in FIG. 2). This designation is performed by designating the coordinates of four corners, for example, when an area is designated by a rectangle.

  The moving object detection unit 14 detects a moving object in the alert area of the contour-corrected image (step S4 in FIG. 2). If there is no moving object detection (step S5 in FIG. 2), the process is first performed (FIG. 2). Step S1) is repeated. FIG. 3 is a first diagram for explaining the moving object detection, and shows that the person image in the alert area designated by the alert area designation 13 is subdivided.

  FIG. 4 is a second diagram for explaining the moving object detection. The image frame is updated with time as shown in FIG. FIG. 4B shows one image frame taken from such an image frame and showing a human face. Such an image frame is divided into, for example, 450 blocks, and each block is divided into, for example, 64 pixels as shown in FIG.

  The moving object detection unit 14 examines the same coordinate pixel for a total of seven image frames including the current image frame and the past six image frames. As a result, when four or more are the same, it is determined that the pixel is “no change”. If it is different, it is judged as a “moving object”. This is processed for all the pixels on the screen to calculate 8 × 8 pixels. When 25 pixels or more (which can be changed by setting) of these 64 pixels are white, a block moving this block is determined. When the moving blocks are 25 blocks or more (can be changed by setting), the moving object is recognized and transmitted to the object detection device 20.

  In the target object detection device 20, the roughness eliminating unit 21 eliminates the roughness (such as tree shaking and waves) in the video (step S6 in FIG. 2). FIG. 5 is a schematic diagram for explaining the roughness elimination process. FIG. 5A is an example of a registered wavy pattern (pattern blur component). As shown in FIG. 5B, the image is such a pattern blur component ρ (X, Y) and a moving object φ. In the case of combining (X, Y), the pattern blur component ρ (X, Y) that matches the registered pattern blur component in the image is excluded. Moving objects are not excluded. When the roughness is large, the operation of reducing the false detection is performed by increasing the subdivision unit described in FIG.

  FIG. 6 is a more realistic diagram for explaining the roughness elimination process. Here, as shown in FIG. 6A, rain and snow are shown as the pattern blur component ρ (X, Y), and a human face is shown as the moving object φ (X, Y). As shown in FIG. 6B, the existence probability on the image of rain or snow is constant regardless of the position on the screen, whereas the existence probability on the image of the human face is shown in FIG. ), The center of the screen is high. Therefore, it is possible to exclude objects other than moving objects such as pattern blur components by such existence probability characteristics.

  Next, the monitoring target identifying unit 22 performs a monitoring target identifying process on the video from which the roughness is eliminated (step S7 in FIG. 2). When the object subjected to the moving object detection process (step S4 in FIG. 2) does not apply to the registered monitoring object pattern (step S8 in FIG. 2), the objects other than the monitoring object are excluded by the non-monitoring object exclusion unit 23. The process is repeated from the beginning (step S1 in FIG. 2).

  Note that the monitoring object pattern can be obtained by extracting the outermost vertex of the image pixel. For example, it is simulated by a vertically long rectangle for a person, a horizontally long rectangle for a car, a triangle for a motorcycle, and a trapezoidal figure for a ship. Such a bold simulation can also be affirmed as being reasonable in terms of the appearance probability if it is specially considered in an environment to which the area warning automatic identification and monitoring system is applied.

  On the other hand, when the monitoring target pattern is applicable, the moving object detection display unit 32 displays the monitoring target (person, vehicle, motorcycle, ship, etc.) on the screen (step S10 in FIG. 2), and the behavior suspicious pattern identification unit 24. 2 performs identification processing to determine whether the movement behavior pattern of the monitoring object identified by the monitoring object identification unit 22 matches the behavior suspicious pattern of the monitoring object registered by the behavior suspicious pattern registration unit 25 (see FIG. 2). Step S11).

  FIG. 7 is a schematic diagram for explaining the movement transition of the monitoring object. In FIG. 7, an arrow indicates that a person is moving from the lower left of the screen to the upper side, the right side, and between the cars to the lower side and the right side. Actually, since a person is treated as a block, the locus of the block is traced. The movement behavior pattern of the monitoring object can be obtained from the combination of such arrows. Note that the movement behavior pattern of the monitoring object can also be performed in units of pixels.

  Strictly speaking, the optical flow method is well known as an analysis method for obtaining the moving direction and speed of a moving object from a plurality of images observed over time. The optical flow method is a method in which a motion (vector) of a light and shade pattern in a moving image is displayed at each pixel point. FIG. 8 is a diagram for explaining the concept of detecting the moving direction of the moving body in the suspicious behavior pattern identifying unit 24.

Now, as shown in FIG. 8 (A), if the object of the shading pattern ρ located at the coordinate (x, y) moves to the coordinate (x + δx, y + δy) after δt seconds, the moving object moves. The direction can be displayed as a vector as shown in FIG. Further, since the shading pattern ρ does not change before and after the movement, ρ (x, y, t) = ρ (x + δx, y + δy, t + δt)
As a result of the behavior suspicious pattern identification process (step S11 in FIG. 2), when the movement behavior pattern of the monitored object matches the behavior suspicious pattern (step S12 in FIG. 2), the behavior suspicious reporting unit 26 performs a reporting process. In response to this, the suspicious behavior display unit 31 of the display device 30 displays the contents of the notification on the screen, and notifies by a buzzer sound, lamp lighting, or the like.

  On the other hand, when the movement behavior pattern of the monitored object does not match or matches the non-target pattern in the behavior suspicious pattern identification (step S12 in FIG. 2), the non-pattern exclusion unit 27 performs the exclusion process first (FIG. 2). 2. Repeat from step S1).

  9 to 16 show various behavior suspicious patterns registered by the behavior suspicious pattern registration unit 25. Multiple patterns can be registered. The detection setting time can be set arbitrarily. Further, even if there is a certain amount of variation in the pattern, a close movement trajectory is determined as an identification target.

  FIG. 9 shows a lightning type movement identification pattern. This pattern moves with an electric light type zigzag (bently bent), and the movement locus is not constant. FIG. 10 shows an intrusion prohibition direction identification pattern. This pattern represents a movement from a to b in the x direction.

  FIG. 11 shows a climbing motion identification pattern, that is, a pattern exceeding a predetermined position in the y direction. FIG. 12 is a dive-through operation identification pattern, that is, a pattern that dives in a predetermined position in the y direction.

  FIG. 13 shows a frequently appearing and appearing motion identification pattern, that is, frequently moving up and down to appear and hide (disappear). The vertical movement detection time can be arbitrarily set. FIG. 14 shows an expansion / contraction movement operation identification pattern, that is, how a person performs an expansion / contraction movement operation such as bending. The travel time detection can be arbitrarily set.

  As explained above, according to this area warning automatic identification and monitoring system, an abnormal state is detected by pattern matching and optical flow pattern matching from the video information from the surveillance camera that images the wharf area, and an alarm is generated. can do.

[Industrial applicability]
Although the present invention has been described on the assumption that it is applied to a wharf monitoring system, a customs illegal entry and exit monitoring system, a parking lot entry and exit number monitoring system, a store entrance and exit number monitoring system, a railway stone monitoring system, a landslide disaster It can be applied to a wider range of disaster prevention / crime prevention / security sites such as monitoring systems, suspicious ship monitoring systems, illegal dumping monitoring systems, factory illegal intrusion monitoring systems, unmanned facility illegal intrusion monitoring systems, and dam river monitoring systems.

The block diagram which shows the area warning automatic identification monitoring system of this invention The flowchart which shows the operation | movement procedure of the area warning automatic identification monitoring system of this invention. 1st figure for demonstrating a moving body detection 2nd figure for demonstrating a moving body detection Schematic diagram for explaining the roughness elimination process A more realistic diagram to explain the noise elimination process Schematic diagram for explaining the movement transition of the monitored object The figure for demonstrating the concept of the moving direction detection of a moving body in a suspicious pattern identification part The figure which shows the 1st behavior suspicious pattern The figure which shows the 2nd behavior suspicious pattern The figure which shows the 3rd behavior suspicious pattern The figure which shows a 4th behavior suspicious pattern The figure which shows a 5th suspicious pattern The figure which shows a 6th behavior suspicious pattern

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Camera 10 Object taking-in detection apparatus 11 A / D conversion part 12 Image processing part 13 Warning area designation | designated part 14 Moving body detection part 20 Object detection apparatus 21 Roughness exclusion part 22 Monitoring object identification part 23 Monitoring object exclusion part 24 Behavior Suspicious pattern identification unit 25 Suspicious behavior pattern registration unit 26 Suspicious behavior reporting unit 27 Outside pattern exclusion unit 30 Display device 31 Suspicious behavior display unit 32 Monitoring object identification display unit

Claims (5)

A monitoring object identification unit that registers a monitoring object pattern and identifies the monitoring object using image processing technology of pattern matching from the captured video;
An area warning automatic identification and monitoring system, comprising: a non-monitoring exclusion unit that removes objects other than the monitoring object from the video according to the result of the identification process.   2. The area warning automatic identification and monitoring system according to claim 1, wherein the monitoring object pattern to be registered is a simulated figure obtained by extracting and connecting vertices on the outermost side of image pixels of the monitoring object.   The present invention is characterized in that a roughness pattern to be excluded is registered, and when the roughness in the video matches the roughness pattern, there is provided a roughness exclusion unit that eliminates it using a pattern matching image processing technique. The area warning automatic identification monitoring system according to claim 2 or claim 2. A suspicious behavior pattern registration unit for registering a behavior suspicious pattern of the monitoring object;
The behavior suspicious pattern in which the movement behavior pattern of the monitored object that has been subjected to the identification processing by the monitoring target identifying unit matches the behavior suspicious pattern of the monitored processing target that has been registered or is identified using an optical flow image processing technique An identification unit;
4. A suspicious behavior reporting unit for performing a reporting process when a movement behavior pattern of the monitoring object matches the behavior suspicious pattern according to a result of the identification processing is provided. The area warning automatic identification and monitoring system described.   5. The area warning automatic identification and monitoring system according to claim 4, wherein the movement behavior pattern of the monitoring object is obtained by following a transition of blocks or pixels obtained by dividing the video screen.

Images