KR101877294B1 - Smart cctv system for crime prevention capable of setting multi situation and recognizing automatic situation by defining several basic behaviors based on organic relation between object, area and object's events - Google Patents

Abstract

According to the present invention, a smart CCTV system for crime prevention capable of automatically recognizing a situation comprises: a basic information storage unit to store information for a plurality of basic action patterns; a crime prevention area setting unit to set information for a crime prevention area; a non-allowed pattern setting unit to set a non-allowed action pattern which is not allowed in the crime prevention area among the plurality of basic action patterns; a postprocess setting unit to set one or more postprocesses in accordance with the non-allowed action pattern; an object information generation unit to generate path data of a target object detected by using image data of a detection camera, and use a position relation between the path data and the crime prevention area to generate action pattern information of the target object; and a driving control unit to control the one or more postprocesses to be driven if the generated action pattern information corresponds to the non-allowed action pattern.

Description

TECHNICAL FIELD [0001] The present invention relates to an intelligent security CCTV system capable of recognizing a plurality of basic behavior patterns based on an organic relationship between events caused by objects, regions and objects, RECOGNIZING AUTOMATIC SITUATION BY DEFINING SEVERAL BASIC BEHAVIORS BASED ON ORGANIC RELATION BETWEEN OBJECT, AREA AND OBJECT'S EVENTS}

The present invention relates to an intelligent CCTV security system, and more particularly, to an intelligent CCTV security system that more accurately detects and tracks an object moving at a long distance by organically fusing two or more detection tracking algorithms, By associating with the behavior pattern, the behavior pattern of the dynamic object is systematically recognized accurately, and by matching the behavior pattern based on the organic relation of the event triggered by the object and the plural basic action pattern, And an intelligent CCTV system implemented so that the follow-up action thereof is automatically performed.

In order to realize various purposes such as crime prevention, prevention of accidents, disaster monitoring, monitoring of buildings and facilities, identification of passengers using public transportation, parking, traffic monitoring, etc., CCTV and related systems or devices are installed , And these devices and systems are gradually increasing due to various social needs.

Such CCTV and related systems are used in various ways. Generally, a CCTV is installed in a plurality of locations in a specific area, and the CCTV is output to a screen display unit for controlling video data photographed from the plurality of CCTVs. Etc.) to monitor specific events while watching them, or to use images of various places or locations as a database and to use post-test evidence.

In addition, a conventional method of monitoring an object of interest in a captured image using a video analysis (VA) technique or the like and monitoring the movement of the tracked object is applied to a conventional CCTV or control system, When an abnormal behavior or an action of an object is detected, a method of transmitting alarming information to the manager is also used little by little.

Methods for detecting and tracking such objects include a method of detecting an object included in image data using an image difference technique, an optical flow technique, and the like, and further, a method of detecting an object included in the image data, (Kalman Filter) are used to track objects.

However, since the techniques related to object detection and tracking are designed to be processed by limited specific environment or feature parameters that can be detected and tracked by the corresponding technique, However, in situations or environments that are not suitable for these techniques, the efficiency of object detection and tracking is significantly degraded.

For this reason, the conventional techniques and methods are sensitive to the noise change of the external environment, so that the reliability of detection / tracking is degraded. Moreover, since it is difficult to continuously detect and track the object accurately, There is a limit.

Even if the dynamic object is detected or tracked, the conventional method only applies the method of expressing the result to the manager or simply storing it in a large form. Therefore, in applying the result, (Human) dependent method, it is possible to automatically recognize an intruder or the like performed in the crime prevention area by classifying the type of the intruder or the like, as well as to automatically follow the action pattern of the intruder or the differentiated action for each action have.

Furthermore, in the system in which the conventional method is implemented, only the method of confirming all stored image data in order to secure evidence for various crime scenes can be performed. Therefore, the process of recognition and confirmation of the crime scene is considerably inefficient, It can be said that it is almost impossible to select specific actions / actions of an intruder to be used in a subsequent step.

The object of the present invention is to solve the above-mentioned problems, and it is an object of the present invention to provide a method and apparatus for detecting and tracking an object, It is possible to precisely and sustainably detect and track the object and also to precisely define the motion, behavior or movement pattern of the tracked object by the time series combination of the predefined behavior pattern, The object of the present invention is to provide a system infrastructure that can more effectively implement subsequent utilization according to an action pattern of an object.

Further, the present invention enables automatic recognition of a crime scene by effectively classifying an unacceptable behavior or behavior pattern in a crime scene through definition of a behavior pattern for an object (an object that can be regarded as an intruder or the like) And to provide a system that can automatically implement follow-up actions that are differentiated according to a behavior action or a behavior pattern.

Other objects and advantages of the present invention will become apparent from the following description, and it will be apparent from the description of the embodiments of the present invention. Further, the objects and advantages of the present invention can be realized by a combination of the constitution shown in the claims and the constitution thereof.

According to an aspect of the present invention, there is provided an intelligent security CCTV system including a basic information storage unit for storing information on a plurality of basic behavior patterns; A crime prevention area setting unit for setting information on the crime prevention area; An unauthorized pattern setting unit for setting an unacceptable behavior pattern not allowed in the crime prevention area among the plurality of basic action patterns; A subsequent process setting unit for setting one or more subsequent processes according to the unacceptable behavior pattern; An object information generating unit which generates motion line data of the detected object using the image data of the detection camera and generates motion pattern information of the object using the positional relationship between the motion data and the crime prevention area; And a drive controller for controlling the at least one subsequent processing to be driven when the generated behavior pattern information corresponds to the unacceptable behavior pattern.

Here, the subsequent processing of the present invention may include at least one of image storage of the target object, flashing of a warning light, operation of a siren, warning warning output of audio output means, or warning warning output of display means.

In addition, the drive control unit of the present invention may control the tracking camera capable of PTZF drive when the behavior pattern information corresponds to the unacceptable behavior pattern so that the subsequent processing for photographing the near object image data of the target object is driven .

Preferably, the present invention includes: a reference image DB unit storing reference image data, which is image data of a user permitted to enter and exit; And an image processor for storing the near field image data of the target object only when the near field image data of the target object transmitted from the tracking camera does not correspond to the reference image data.

According to an embodiment of the present invention, there is provided a mobile terminal comprising: a packet storage unit for storing a basic alarm data packet including a subject information area, a place information area, a time information area and a behavior information area; And a controller configured to generate short-range image data for the target object in each of the subject information area, the place information area, the time information area, and the action information area of the basic alarm data packet when the generated behavior pattern information corresponds to the non- And an alarm processor for generating and storing alarm data packets by storing information on the security zone, time information on the unacceptable behavior pattern, and one or more unacceptable behavior pattern information.

When the generated behavior pattern information corresponds to the unacceptable behavior pattern, the image processing unit of the present invention stores the image data of the target object, and stores the tag information of the corresponding unacceptable behavior pattern in the target object In this case, if a video request signal including information on an unacceptable behavior pattern is inputted, one or more video data corresponding to the video request signal is selected using the tag information And the like.

Furthermore, the object information generating unit of the present invention may further include: a receiving unit for receiving image data from n (n is a natural number of 1 or more) detecting cameras; A first detection unit for detecting an object by applying a difference image detection algorithm to image data received from the detection camera and generating first candidate MER information of the detected object; A second detecting unit detecting an object by applying an optical flow algorithm to the image data and generating second candidate MER information of the detected object; A detection determination unit for generating final MER information of the object by applying a hog algorithm to the first and second candidate MER information; A tracking unit for tracking movement of final MER information of the object to generate copper line data of the object; And a behavior pattern determiner for determining a behavior pattern of the object by using a positional relationship between the context aware area as a basis of the behavior pattern analysis and the dynamic line data of the object.

The behavior pattern determination unit of the present invention may further include an event generation unit for generating event information of an object using the relationship between the coordinates of the final MER center point position of the object of each time of the copper line data and the position coordinates of the boundary line of the context- ; And a pattern information generator for generating behavior pattern information for the object using time-series characteristics of the object's event information.

Herein, the event information of the object according to the present invention includes an entry event in which the object moves into the context area, an entry event in which the object is exited from the context area, A moving event that generates a motion at a non-zero moving speed within the same area), a still event that does not move within the context area, an object that is located inside the context area, and the other object Or a combination event in which different objects located in the context aware area located in the context aware area over the reference time generate the same location information or the same displacement movement.

The behavior pattern determination unit of the present invention further includes a pattern information storage unit for storing a plurality of reference behavior pattern information defined by a time series combination of a plurality of event information among the entry, advance, movement, stop, separation, In this case, the pattern information generator of the present invention may be configured to generate behavior pattern information of the object using the reference behavior pattern information corresponding to the copper line data among the plurality of reference behavior pattern information.

The intelligent security CCTV system according to the present invention is configured to be organically combined with processing for detecting, tracking, and generating behavior patterns of an object, and when the object corresponds to a specific behavior pattern, local high-resolution image data is automatically generated And a system capable of accurately recognizing the face of the target object can be implemented using the same.

The intelligent security CCTV system according to the present invention processes object detection algorithms in a binary manner and processes the object results in an organic detection algorithm using a third algorithm that is optimized for intra- By applying it hierarchically, it is possible to adapt to the noise of the external environment, to further improve the probability of object detection, and to increase the computational efficiency.

According to still another embodiment of the present invention, dynamic data of a dynamic object, that is, dynamic characteristics or behavioral characteristics generated by a dynamic object, may be classified into an event of the predefined object and a behavior pattern It is possible to precisely define and quantify or dataize by using information, and it is possible to create an effect of accurately interfacing a behavior pattern of a dynamic object.

Further, according to the present invention, an action / action pattern of an intruder or the like can be typed according to eight basic action patterns, thereby accurately and automatically recognizing an action / action pattern of an intruder or the like, It is possible to provide a system infrastructure that can optimize the organic response according to the crime situation.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate preferred embodiments of the invention and, together with the detailed description of the invention given below, serve to better understand the technical idea of the invention, And shall not be construed as limited to such matters.
1 is a block diagram showing a detailed configuration of an intelligent security CCTV system according to a preferred embodiment of the present invention;
FIG. 2 and FIG. 3 are flowcharts illustrating a processing procedure according to preferred embodiments of the present invention,
Figure 4 illustrates one embodiment of the present invention for an interface environment for establishing a security situation and subsequent actions;
FIG. 5 is a view for explaining one embodiment of history data on a method status in which various information on a crime scene is utilized in connection with the method;
FIG. 6 is a block diagram showing a detailed configuration of the object information generating unit of the present invention shown in FIG. 1;
FIG. 7 is a block diagram showing a detailed configuration of the behavior pattern determining unit shown in FIG. 6;
FIG. 8 and FIG. 9 are flowcharts illustrating a detailed processing procedure of object detection according to a preferred embodiment of the present invention;
FIG. 10 is a flowchart illustrating a detailed process of object tracking according to an exemplary embodiment of the present invention.
FIG. 11 illustrates an embodiment of an image recognition area and a context recognition area according to the present invention.
12 is a view for explaining a characteristic of the HOG algorithm used for object detection,
13 is a view for defining six event information of an object according to an exemplary embodiment of the present invention;
14 is a view for explaining eight behavior pattern information according to a preferred embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary terms, and the inventor should appropriately interpret the concepts of the terms appropriately It should be interpreted in accordance with the meaning and concept consistent with the technical idea of the present invention based on the principle that it can be defined.

Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and are not intended to represent all of the technical ideas of the present invention. Therefore, various equivalents It should be understood that water and variations may be present.

1 is a block diagram showing a detailed configuration of an intelligent crime prevention CCTV system (hereinafter referred to as a 'crime prevention system') 1000 according to a preferred embodiment of the present invention.

1, the security system 1000 of the present invention includes an object information generation unit 100, a basic information storage unit 210, an interface unit 220, a non-permission pattern setting unit 230, An image processing unit 600, an alarm processing unit 700, and a video image processing unit 800. The image processing unit 800 may include an image processing unit 300, a subsequent process setting unit 400, a drive control unit 500,

Prior to the description of the present invention, the crime prevention system 100 of the present invention shown in FIG. 1 and the respective components of the present invention shown in FIGS. 6 and 7 are not physically separated constituent elements, .

That is, since each constitution corresponds to a logical constituent element for realizing the technical idea of the present invention, even if each constituent element is constituted integrally or separately, if the function performed by the logical constitution of the present invention can be realized, It is to be understood that any component that performs the same or similar function should be interpreted as falling within the scope of the present invention regardless of the consistency of the name.

As shown in FIG. 1, the monitoring system 1000 of the present invention can be implemented in a hardware configuration such as a system device, a server, or the like, as well as a sensing camera 50, a zoom camera (a tracking camera / ) 70 and the like, and may be embodied in the form of software for carrying out the processing of the present invention which will be described later.

The surveillance system 1000 of the present invention accurately detects the target object of interest in the image data by raw-data of the image data photographed by one or more detection cameras 50, And motion pattern of the object is tracked to generate motion data or behavior pattern information of the object, and the behavior or behavior pattern of the object (preliminary offender, etc.) And automatically performs subsequent processing according to the recognized situation.

To this end, the crime prevention system 1000 according to the present invention includes information on an area (place) requiring crime prevention, information on an unacceptable behavior / action in the corresponding crime prevention area, and information on unacceptable behavior / The subsequent processing to be performed is automatically set up or is configured to be set through interfacing with an administrator client or the like via the interface unit 220. [

Specifically, the basic information storage unit 210 of the present invention stores information on a plurality of basic behavior patterns for determining whether a behavior / action of a target object is an action requiring attention in relation to a crime.

The plurality of basic action patterns are used for entering, advancing, moving, stopping, separating, and combining using the positional relationship between the motion data of the object and the boundary of the context-aware area as described later with reference to Figs. 13 and 14 And 8 behavior patterns classified as intrusive (appearing), disappearing (escaping), pausing, passing through, organizing, eliminating, inhibiting direction movement, and roaming defined through a time series relationship between these events it means.

In order to implement a user-oriented environment, the crime prevention area setting unit 300 of the present invention divides a plurality of areas (indoor and outdoor) in which a CCTV is installed , Information on the plurality of zones to be a crime prevention target through an interface environment exemplarily shown in FIG. 4 is provided to an administrator client or the like (S200), and information on a crime prevention area to be crime-covered is provided to a manager And may be configured to be set by selection (S210).

In addition, the unauthorized pattern setting unit 230 of the present invention provides a plurality of (preferably eight illustrated above) basic behavior patterns stored in the basic information storage unit 210 through the interface environment illustrated in FIG. 4 , An unacceptable behavior pattern to be monitored is set as an unacceptable behavior / behavior / behavior (hereinafter referred to as 'action') and an action requiring attention in the corresponding crime prevention area (S210).

Through such a configuration, it becomes possible to perform security monitoring differently according to the importance of the area / area (area, section, etc.), the frequency of access, the number of times the intruder appears, and the complementary grade.

When the unauthorized action pattern is generated in the crime prevention area and the corresponding crime prevention area as described above, the subsequent process setting unit 400 of the present invention performs the following process when each unauthorized action pattern occurs through the interface environment, So that the subsequent processing is set (S210).

As illustrated in FIG. 4, the following processing for each action is performed by an audio output means such as a near-field image (zoom image / high-resolution image) photographing by a zoom camera 70, a near field image storage, a warning light 71 driving, A visual warning broadcast through a display means 77, an SMS / MMS transmission to an administrator terminal, a manager emergency call, a police emergency call, and a siren 73, Lt; / RTI >

4, a person is set as a target object when the crime prevention area is Area 3 (research center), a behavior pattern for intrusion, organic and forbidden direction movement is set to an unacceptable behavior pattern, When a behavior pattern is formed, the near-end imaging and storage, the driving of the beep light, and the alarm information transmission to the administrator terminal are set in the subsequent processing.

Through such a complex typed interface environment, a series of setting processes for subsequent processing according to method area, behavior pattern (allowed / not allowed) and unacceptable behavior pattern can be implemented through a more user-oriented automated environment .

When the setting of the method status (S210) is completed, the object information setting unit 100 of the present invention generates the copper line data of the detected object using the video data of the detection camera 50, The action pattern information of the target object is generated using the positional relationship with the crime prevention area described above (S220).

A detailed description of the object information setting unit 100 including the detection, tracking, and pattern determination of the object will be described later. In the following, the basic functions and configuration of the object information setting unit 100 will be described.

The object information setting unit 100 of the present invention detects the target object that is an object of interest in the image data by parsing or analyzing the image data when the image data is received from the detection camera 50 (S221).

 An object of interest is an object of interest or an object of interest, such as a person, an animal, or a vehicle. Various methods such as an image difference algorithm, an optical flow algorithm, and a model matching algorithm may be applied as methods for extracting objects from image data, including binarization, labeling processing, and edge detection.

The method of representing the detection result of the object uses a minimum enclosing rectangle (MER), which is referred to as a minimum adjacent rectangle or a minimum containing rectangle. Specifically, the width of the MER, Height, center point coordinates, density information, and width / height ratio information, and a method of detecting / distinguishing a specific object of interest by comparing the characteristics of the MER and the width and height of the object included in the image data Can be used.

When the detection of the target object is completed, processing for tracking the detected object (S222) proceeds. Tracking of the detected object corresponds to the process of generating path data, behavioral characteristic information, and behavioral pattern information of the object as to which direction the detected object moves to.

When tracking of the object is completed, the behavior pattern information of the object is generated using the mutual positional relationship between the movement data of the created object and the predetermined context-aware area (S223).

As shown in FIG. 11 (a), in the image generation area, which is an area generated by the camera 50, an area that does not need to monitor a human object or the like It is necessary to reduce the amount of computation by excluding the region in which the detection of the object is not required in the image generation region, as shown in Fig. 11 (a).

Furthermore, the area set for the purpose of automatically recognizing the situation of the system (apparatus) as the area included in the image processing area becomes the situation recognition area (FIG. 11 (b)) without user intervention.

When the situation recognition area is set, position coordinates such as the boundary (outline) and center point coordinates of the situation recognition area (image generation area) are generated. Based on the position coordinates of the situation recognition area, The center point coordinate information is compared with the position coordinates of the situation aware area to generate information on the behavior characteristics or behavior patterns of the object.

When the action pattern information for the target object is generated as described above, the drive control unit 500 of the present invention determines whether the generated action pattern information corresponds to the previously set unauthorized action pattern (S230) (Behavior pattern information) corresponds to the unacceptable behavior (unacceptable behavior pattern), control is performed such that at least one subsequent processing set differentially according to the unacceptable behavior pattern is driven (S240).

For example, if the current behavior pattern information of the target object corresponds to the unacceptable behavior pattern, the drive control unit 500 of the present invention can detect the PTZF (Pan, Tilt, Zoom, Focal length) of the tracking camera 70 installed at a predetermined position, And so on, so that high-resolution zoom images (near vision image data) of the target object can be photographed so that evidence for the intruder can be obtained more clearly in the future.

In this regard, as shown in FIG. 3, the present invention includes a reference image DB unit 610 in which reference image data, which is image data containing a face of a user (employee, etc.) (600) for storing the near field image data of the target object or the image data of the detection camera (50) only when the near field image data of the target object transmitted from the controller (70) does not correspond to the reference image data .

Although the actual unaccepted behavior is detected (S310), even if the near vision image data for the target object is captured (S320), the captured near vision image data and the image of the allowed person (reference image data) (S330) only when the target object is a person who is not permitted to enter or leave through the parsing / analysis of the unauthorized action pattern (S340), the case in which the subject of the unauthorized action pattern corresponds to the person allowed to access And the efficiency of data processing, data storage, and the like can be further enhanced.

It is needless to say that the present invention can be configured such that the efficiency of the data analysis and the data processing can be improved by storing the data separately for both the user who is permitted to enter and the user who is not allowed to enter the apparatus according to the embodiment.

In this case, the local image data for the subject information (target object), the image data related to the copper line of the target object (the image data photographed by the detection camera 50), the ratio of the target object It is more preferable that the history data of the form in which the time series information of the allowable behavior pattern is correlated with each other is stored.

More preferably, the packet storage unit 710 of the present invention can be configured to store a basic alarm data packet including a subject information area, a place information area, a time information area, and a behavior information area. The basic alarm data packet (packet) to the basic alarm data set (set) corresponds to a kind of data format according to a predetermined specification.

When the basic alarm data packet is set as described above, the alarm processing unit 700 of the present invention is configured to perform a simple operation of inserting information set or confirmed by the above-described processing into each information area of the basic alarm data packet made of a blank It is possible to effectively generate, store and manage the history data on the crime scene.

Specifically, the alarm processing unit 700 of the present invention inserts or records video data for a target object in a subject area set as a blank as illustrated in FIG. 5, and inserts information set as a security area in a place information area The time information area includes time information in which an unacceptable behavior pattern is formed, and the behavior information area may include one or more unacceptable behavior pattern information in a time series.

By generating individual data sets for each crime scene, it is possible to effectively manage the history data on the crime scene and to maximize the efficiency of the subsequent evidence check process.

When an alarm data packet is generated so that an appropriate syntax scheme (investigation, predicate, etc.) is given to each information area, and is configured to be transmitted by MMS / SMS via a communication network such as an administrator terminal or a police station, user interfacing can be implemented more effectively have.

The alarm data packet thus generated can be configured to be converted into audio data using a text to speech (TTS) engine. The audio output means 75, which is installed in a place where the intruder is located, And can be used for an emergency call with an administrator terminal or a police station communication terminal.

In addition, when the behavior pattern information generated by the above-described method corresponds to the unacceptable behavior pattern, the image processing unit 600 of the present invention may store the image data (the image data photographed by the sensing camera and / And the tag information for the corresponding unacceptable behavior pattern is stored in association with the image data.

When the image data of the target object is stored together with the tag information representing the corresponding unacceptable behavior pattern, the imaging system 800 of the present invention transmits the image request signal including information on the unacceptable behavior pattern, It is possible to more easily and accurately select and provide the classified image data according to the corresponding unacceptable behavior pattern, thereby further improving the subsequent efficiency of the information confirmation.

Hereinafter, the configuration and processing of the object information generation unit 100 of the present invention for performing object detection, object tracking, and behavior pattern information generation of an object, etc. will be described in detail with reference to FIGS. 6 to 9 attached hereto. FIG. 6 is a block diagram showing a detailed configuration of the object information generating unit 100 of the present invention 1000 shown in FIG. 1. FIG. 8 and FIG. 9 are diagrams for explaining the object detecting method according to a preferred embodiment of the present invention. Fig. 3 is a flowchart showing a detailed processing procedure. Fig.

6, the object information generating unit 100 includes a receiving unit 110, a first detecting unit 130, a second detecting unit 140, a detection determining unit 140, (150), a tracking unit (170), and a behavior pattern determination unit (190).

First, as shown in FIG. 6, the receiving unit 110 of the present invention is a receiving unit 110 that receives signals from n detection cameras 50-1, 50-2, ..., 50- The video data is received through a wired or wireless communication network (S400).

Since the object information generating unit 100 is focused on the detection and tracking of the object itself, the detection camera 50 can detect a view angle such that a remote area can be photographed rather than a kind of zoom camera, It is preferable that a wide camera is suitable and that the detection camera 50 is implemented as a plurality of images for each direction so that the image of the omnidirectional region can be captured.

It is needless to say that image data of a specific area generated by a plurality of cameras covering a single or specific direction other than the omnidirectional area may be used according to the embodiment.

If the sensing camera 50 transmits the image data in this way, the first sensing unit 130 of the present invention applies the differential image sensing algorithm robust to the external environmental noise to the transmitted image data, And generates first candidate MER information of the detected object (S420).

The second detection unit 140 detects an object by applying an optical flow algorithm to the transmitted image data independently of the processing of the first detection unit 130, 2 candidate MER information (S430).

As mentioned above, the MER information is information on the minimum rectangular area in which the object is included, and includes width, height, center point coordinates (a, b), density information (Implication) ratio), width / height ratio information, and the like.

Since there may be a plurality of objects in one image data and a plurality of objects estimated as objects in the image data may exist for each algorithm, the first candidate MER information or the second candidate MER information may include a plurality of MERs Lt; / RTI > may be a collection of datasets related to information about < RTI ID = 0.0 >

In this case, when a plurality of image data are input from the plurality of detection cameras 50 in order to more effectively detect an object in a further enlarged area, the foreground image unit 120 of the present invention merges merged to generate panorama image data (S405).

In order to improve the efficiency of the image processing as described above, the image data generated by the camera 50 may be moved (S410) to reduce the image data in consideration of the area where the object can move .

When the panoramic image data is generated as described above, the first detection unit 130 and the second detection unit 140 of the present invention generate first and second candidate MER information for the panorama image data, respectively.

(2) an optical flow algorithm that computes an optical flow velocity vector between a previous image and a current image, and (3) And a method of using a minutiae point information or a method of comparing a direction pattern between pixels in a set search area, and a model matching method of generating and utilizing a model for detection.

The image difference algorithm can be subdivided into a background image difference algorithm, a previous image difference algorithm, or an adaptive background image difference algorithm to the noise change of the external environment. The optical flow algorithm includes a horn & Schunk algorithm, a block matching Matching algorithm, the Lucas & Kanade algorithm, or the Pyramid Lucas & Kanade algorithm.

The model matching method can be subdivided into a shift algorithm, a Surf algorithm, or a Histogram (Hogg of Oriented Gradient) algorithm.

These methods are designed to be optimized for each specific environment and parameters, and thus can be used for various purposes such as hardware resource usage, computation speed, ease of logic implementation, noise removal performance, rapid environmental change adaptability, , There is a considerable variation in the detection performance of each method depending on the sudden change of the illumination environment, the sudden change of the illumination environment, and the shadow effect.

The present invention is configured to maximize detection efficiency by configuring the detection algorithms to be hybridized so as to optimize the disadvantages and advantages of such conventional detection techniques and to organically combine them.

For this purpose, the first and second detection units 130 and 140 of the present invention are adapted to apply detection algorithms corresponding to different categories, respectively. First, in order to increase the probability of object detection, a difference image detection algorithm And generate first candidate MER information using a difference image detection algorithm robust against external environment noise and second candidate MER information using an optical flow algorithm, respectively.

Specifically, the first detection unit 130 of the present invention performs processing S421 of updating / generating a background image every frame using statistical images accumulated for a predetermined time, And generates at least one first candidate MER information through processing (S423).

The second detection unit 140 calculates an optical flow velocity vector between the two images (the previous image and the current image) based on the difference caused by the relative movement of the camera and the object.

Specifically, the second detection unit 140 detects an object by processing an optical flow algorithm distinguished from the image difference algorithm. In order to improve the probability and efficiency of object detection, the second detection unit 140 constructs an image pyramid according to the scale of the optical flow algorithm A pyramid Lucas and Canard algorithm, which is an algorithm for calculating an optical flow velocity vector between a previous image and a current image, is processed (S431) and one or more second candidate MER information is generated using the window matching method (S433) .

When the first candidate MER information and the second candidate MER information are generated by applying the different or biased algorithm in this way, the detection determination unit 150 of the present invention determines the first candidate MER information and the second candidate MER information The final MER information of the object is generated by applying a hog algorithm, which is a third algorithm for object tracking (S440).

Specifically, the detection determination unit 150 of the present invention determines whether or not one or more MER information items belonging to the first candidate MER information item and one or more MER information items belonging to the second candidate MER information item, The MER information in which the deviation is lower than the reference deviation is selected and a hybrid area composed of only the selected MER information is calculated (S441).

When the hybrid region is calculated as described above, the detection determination unit 150 of the present invention performs HOG algorithm processing on the hybrid region (S443) and generates final MER information of the object (S445).

12, a hog operation is performed by dividing an image into blocks, dividing a block into cells, and then using a histogram of feature vectors of each cell, Since the method of detecting an object by calculating a direction pattern between pixels in the area, the search area is limited to the hybrid area, and the calculation efficiency can be further improved.

When the final MER information is generated, the final MER information about the object is compared with the predetermined characteristic information (width, height, center point coordinate, density, width / height ratio, etc.) (S450). The object corresponding to the final MER is classified into a person (S460) or another specific object (vehicle, animal, bag, etc.) according to the result (S470).

In order to further improve the reliability of object detection by more effectively filtering the noise due to the external environment, the second detection unit 140 of the present invention determines an external environmental noise factor as shown in FIG. 9 according to the embodiment (S431 It is determined whether the generated external environmental noise is caused by the shaking of the detection camera (S432).

In this case, the first candidate MER information of the object and the second candidate MER information of the object are used as the difference between the detection camera shake and the second candidate MER information because the difference image detection algorithm is relatively difficult to generate accurate MER information. It is preferable that candidate candidates are calculated (S434) only for the second candidate MER information generated by the optical flow algorithm having a relatively small influence due to the influence of the second candidate MER information.

On the other hand, if it is not the external environment noise due to the shaking of the detection camera or the like, the candidate region is calculated on the basis of the first candidate MER information and the second candidate MER information as described above (S433).

As described above, the present invention utilizes a single algorithm for object detection or an object detection algorithm that is implemented in a different manner, rather than simply using a plurality of algorithms in parallel, in an organic and hierarchical manner The probability of object detection can be further improved, and the computational efficiency can be increased.

Hereinafter, object detection processing (S310) implemented through the configuration of the present invention will be described in detail with reference to FIG. 10, which is a flowchart illustrating a detailed processing process of object tracking according to the present invention.

When the processing for detecting the object is completed by generating the final MER information of the object as described above, the tracking unit 170 of the present invention tracks the movement of the final MER information of the object, Movement path data) (S310).

 The algorithm for tracking an object is to specify a target area of the tracked object, obtain a histogram corresponding to the color distribution characteristic of the designated area, and then calculate the histogram of the area corresponding to the target area size in the continuous input image A Cam-Shift algorithm for tracking an object by actively adjusting the size of a search window to solve problems caused by using a fixed-size search window, an error (P) And a Kalman-filter algorithm for tracking the object by predicting future information (x '& P`) based on the current system measurement value (x) and observation value (z).

These object tracking algorithms have different characteristics according to hardware resource, easiness of logic implementation, and computation amount. However, since MinShitP algorithm does not control the size of search window during object tracking, There is a problem that object tracking is no longer valid since the object is lost.

In the case of the camshift algorithm, the size of the search window is designed to be actively adjusted so that it can adapt to the environment where the size of the object changes. However, since the object can not accurately reflect the moving direction or speed of the object, The object can not be traced.

The Kalman filter algorithm can be applied when there is a stochastic error in the measured value of an object (object), and when the state of the object at a certain point in time has a linear relationship with the state at the previous point of time. And each state vector is an algorithm that assumes a Markov chain that is related only to the vectors of the previous time.

In the case of the Kalman filter algorithm, it is a method of tracking the object by predicting the future information based on the measured value and the observed value, and it can reflect the moving direction or speed of the object during the tracking so that the tracking of the moving object can be optimized However, since the observation value is required for the prediction of the next state, it can be said that there is a difficulty in applying the observation value always.

The present invention uses a Kalman filter algorithm for predicting future information using current measurement data and observation data for the final MER information among the object tracking algorithms as a basic object tracking algorithm to track a moving object, In the Kalman filter algorithm, the observation data required for the next state prediction is calculated by applying the cam shift algorithm to the final MER information, and the process is applied cyclically, And to perform effectively.

That is, as shown in FIG. 10, the tracking unit 170 of the present invention is configured such that the pre-processing (S500), the processing by the Kalman filter algorithm (S510) and the processing by the camshit algorithm (S550) (Step S310). The movement data of the object is generated by tracking the movement of the final MER information of the object by performing the circulating processing.

Specifically, the tracking unit 170 of the present invention first initializes a Kalman filter parameter for performing a Kalman filter algorithm (S501), and sets a color space of a basic window area set as an area corresponding to the final MER information of the object as an HSV color (S502).

The HSV color space corresponds to a color space or color model representing a color based on hue, saturation, and brightness as one of the methods of expressing colors such as an RGB color space and a CMY color space .

After changing or changing the color space, the histogram of the base window area is calculated using the HSV data (S503).

When the pre-processing (S500) is completed, the tracking unit 170 of the present invention updates the parameters (state vector, measured value, state matrix, system noise (Kalman error), measurement noise (Step S511). In step S511, the state vector of the center of the object is predicted.

Thereafter, the search window is set to have the same width and height as the center and base window corresponding to the position of the predicted state vector (S512).

 If the center of the object is searched, the search window is located at the center of the searched object and the region is searched (S552) .

If it is determined that the object to be searched is the object to be searched using the correspondence between the histogram calculated in the searched region and the histogram calculated in the pre-processing (S503) (S553), the result of the calculated histogram The rotation angle and the center coordinates of the object are calculated (S555).

The rotation angle and center coordinate information of the calculated object are used as observation values used for prediction of future information in the Kalman filter.

When the rotation angle of the object and the center coordinate information (observation value information) are calculated as described above, the tracking unit 170 of the present invention updates the state vector using the calculated rotation angle and center coordinate information (S513).

When the state vector is updated, the control vector (object rate) of the Kalman filter algorithm is calculated (S514) and returns to the initial stage of the Kalman filter algorithm to apply the above described state vector prediction step (S511) cyclically.

As described above, according to the present invention, not only a single algorithm is used as an algorithm for tracking an object but a Kalman filter algorithm and a camshit algorithm are combined so as to be organically combined, thereby improving the reliability of object tracking .

According to embodiments of the present invention, the present invention can classify objects according to characteristics of objects such as a person, a vehicle, and an animal, that is, MER characteristic information (ratio of width, height, width height, density, center point coordinates, And an object classifier 160 for classifying the object corresponding to the final MER information by comparing the final MER information with the reference information.

In this case, the tracking unit 170 of the present invention may be configured to generate the moving-line data of the object through the above-described processing only when the object classifying unit 160 classifies the object corresponding to the final MER as a person . The path data generated by the tracking unit 170 and the like can be stored in the DB unit 180 of the present invention together with time information.

Hereinafter, the detailed configuration and processing of the present invention for generating a behavior pattern of an object using the copper line data of the finally generated object will be described.

When the tracking unit 170 of the present invention generates the copper line data of the object as described above, the behavior pattern determining unit 190 of the present invention determines the behavior pattern of the object based on the context- Or the final MER information for each view included in the copper line data to determine the behavior pattern of the object and generate corresponding behavior pattern information at step S320.

Specifically, the behavior pattern determination unit 190 of the present invention includes an event generation unit 191, an event DB unit 192, a pattern information storage unit 193, and a pattern information generation unit 194, as shown in FIG. 7 And the like.

In order to more accurately define the behavior pattern information of the object more accurately and to implement accurate interfacing between the system and the user, the present invention predefines six basic events that can be caused by the object as illustrated in FIG. 13, And a method of determining a behavior pattern of an object using time-series characteristics generated by combining one or more kinds of events in combination.

The six event information of the object illustrated in FIG. 13 and the position information on the boundary of the situation recognition area or the image processing area can be stored and utilized in the event DB unit 192 of the present invention.

The event generation unit 191 of the present invention extracts the coordinates of the center point position of the final MER information of the object of each time of the input copper line data when the copper line data of the object is input from the tracking unit 170, The event information for the object is generated using the relationship between the position coordinates of the boundary line of the situation recognition area stored in the event DB unit 192. [

In this regard, in FIG. 13, an entry event refers to an event that an object comes in from outside the context area, and is determined using the coordinates of the center of the object (final MER information) and the boundary of the context area.

An entry event may be an event that the object has moved out of the context aware area according to the embodiment, the object has moved from the context aware area to the image processing area outside the context aware area, , The case where the boundary of the image and the boundary of the context-aware area are in contact, or the case where the object is moved out of the image in the context-aware area.

The movement event is a motion event in which the object is located in the image processing region (which may be larger than the context-aware region) and there is a change in the center coordinates of the MER or a movement of the center coordinates of the object during the reference time within the context- Pixel) / reference time (s) and the calculated speed is not 0 (px / s).

The stop event corresponds to a case in which the amount of change (moving distance) of the central coordinate during a specific reference time is zero or substantially equal to zero, or when the velocity of the central coordinate is substantially zero.

The separation event is an event corresponding to the case where the first detected object is detected for the first time when another object (person, object, etc.) is detected in the object and the adjacent area when the object is inside the context area, Refers to when the center coordinates of the MERs of the different objects approach each other and when the regions of the MER overlap and then move to the same displacement when the different objects (person vs. person or person vs. object, etc.) are within the context area Event.

When one or more event information is generated as described above, the pattern information generator 194 of the present invention generates behavior pattern information for the object using the time series characteristic of the generated one or more event information.

The behavior pattern determination unit 190 of the present invention may further include a pattern information storage unit 193 which stores six patterns of the objects A plurality of reference behavior pattern information defined by a time series combination of a plurality of event information among events, i.e., entry, advance, movement, stop, separation, and association events can be stored.

In this regard, among the reference behavior pattern information shown in Fig. 14, the " path passing " pattern means a situation in which an object moves beyond a set direction and a set boundary line (an outer boundary line or another inner boundary line) Quot; movement " pattern means a pattern in which a positional variation of an object is generated and moved in a direction opposite to a predetermined direction when a position of an object of a current frame is compared with a position of an object of a previous frame.

When a plurality of reference behavior pattern information is stored in the pattern information storage unit 193 as described above, the pattern information generation unit 194 of the present invention generates the pattern information of the reference MER The reference behavior pattern information corresponding to event information of feature information (center point coordinates, width, height, width / height ratio, density, etc.) of information is selected, Information. ≪ / RTI >

As described above, the reference behavior pattern information defined by the time series combination of a plurality of event information among the six basic events becomes a basic piece of the LEGO block, and as a method of constructing the whole block using each piece, Not only can various behavior patterns be precisely defined, but also by associating these behavior patterns according to the order of time, the complex behavior of the object can be precisely defined.

Therefore, according to the present invention, the behavior pattern information of an object is continuously and accurately generated by an accurate and automated method, so that the situation of the object can be accurately and automatically recognized. Therefore, the generated behavior pattern information can be accurately interfaced to the user, It is possible to provide a system infrastructure in which follow-up or counter-processing optimized for a given situation (behavior pattern) can be induced.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not to be limited to the details thereof and that various changes and modifications will be apparent to those skilled in the art. And various modifications and variations are possible within the scope of the appended claims.

In the above description of the present invention, the first and second modifiers are merely terms of a tool concept used for relatively separating the components of each other, so that they are used to indicate a specific order, priority, etc. It should be interpreted that it is not a terminology.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. It should be understood that various modifications may be made in the ordinary skill in the art.

1000: Intelligent security CCTV system of the present invention
50: index camera 70: tracking camera (zoom camera)
100: Object information generation unit 210: Basic information storage unit
220: interface unit 230: non-allowable pattern setting unit
300: Crime prevention area setting unit 400:
500: drive control unit 600: image processing unit
610: reference image DB unit 700: alarm processing unit
710: Packet storage unit 800: Imaging

Claims (10)

A basic information storage unit for storing information on a plurality of basic behavior patterns;
A crime prevention area setting unit for setting information on the crime prevention area;
An unauthorized pattern setting unit for setting an unacceptable behavior pattern not allowed in the crime prevention area among the plurality of basic action patterns;
A subsequent process setting unit for setting one or more subsequent processes according to the unacceptable behavior pattern;
An object information generating unit which generates motion line data of the detected object using the image data of the detection camera and generates motion pattern information of the object using the positional relationship between the motion data and the crime prevention area; And
And a drive controller for controlling the at least one subsequent processing to be driven when the generated behavior pattern information corresponds to the unacceptable behavior pattern,
Wherein the object information generating unit comprises:
a receiving unit for receiving image data from n detection cameras (n is a natural number of 1 or more);
A first detection unit for detecting an object by applying a difference image detection algorithm to the image data and generating first candidate MER information of the detected object;
A second detecting unit detecting an object by applying an optical flow algorithm to the image data and generating second candidate MER information of the detected object;
And generating a final MER information of the object by applying a hog algorithm to the first and second candidate MER information, wherein the first candidate MER information and the second candidate MER information are hybridized with MER information, Calculating a region and performing a hog operation on the calculated hybrid region to generate final MER information of the object;
A tracking unit for tracking movement of final MER information of the object to generate copper line data of the object; And
An event generation unit for generating event information of an object by using a relationship between coordinates of a final MER center point position of each time-series object of the copper line data and position coordinates of a boundary line of a context-aware area serving as a basis of a behavior pattern analysis; And a pattern information generating unit for generating behavior pattern information for the object using the time series characteristic of the event information of the object,
The event information of the object may include:
An entry event in which the object moves into the context area, an entry event in which the object is exited from the inside of the context area, a movement event in which the object generates movement at a non-zero moving speed within the image processing area, A still event that does not move within the area, another object is detected in the object located inside the context-aware area, and the different object located in the context-aware area or the context-aware area is different Position information, or a combination event in which the same displacement movement occurs. The intelligent security CCTV system according to claim 1, 2. The apparatus according to claim 1,
When the behavior pattern information corresponds to the unacceptable behavior pattern, controls the tracking camera capable of PTZF driving to control the subsequent processing for photographing the near object image data of the target object to be driven. CCTV system. 2. The method of claim 1,
Wherein the automatic security monitoring system comprises at least one of image storage of the target object, flashing of a warning light, operation of a siren, warning output of audio output means, or warning output of a display means. 3. The method of claim 2,
A reference image DB unit for storing reference image data which is image data of a user allowed to enter and exit; And
Further comprising an image processor for storing the local image data of the target object only when the local image data of the target object transmitted from the tracking camera does not correspond to the reference image data. Possible intelligent security CCTV system. 5. The method of claim 4,
A packet storage for storing a basic alarm data packet including a subject information area, a place information area, a time information area and a behavior information area; And
When the generated behavior pattern information corresponds to the unacceptable behavior pattern, the short-range image data for the target object, the short-range image data for the target object, the short- Further comprising an alarm processing unit for generating and storing alarm data packets by storing information on the security zone, time information on the unauthorized behavior pattern, and one or more unacceptable behavior pattern information. Security CCTV system. The image processing apparatus according to claim 4,
Storing the image data of the target object when the generated behavior pattern information corresponds to the unacceptable behavior pattern, associating the tag information of the corresponding unacceptable behavior pattern with the image data of the target object,
Further comprising an image preparation unit for selectively providing at least one image data corresponding to the image request signal using the tag information, when the image request signal including information on the unacceptable behavior pattern is inputted, Intelligent security CCTV system that can detect the situation. delete delete delete delete

Images