KR20210117760A - Method for abnormal behavior detection based on cctv images, and apparatus for the same - Google Patents

Abstract

The described embodiment relates to a technology for detecting an abnormal behavior of a person by analyzing a CCTV image. A method for detecting an abnormal behavior based on a CCTV image includes the steps of: detecting a human object from a CCTV image using a human detection engine; generating a tracking information list based on the detected tracking information of the human object; determining an abnormal behavior event for a person on the tracking information list; and generating an alarm signal for the abnormal behavior event.

Description

CCTV image-based abnormal behavior detection method and device therefor

The present invention relates to a technology for detecting abnormal behavior of a person by analyzing a CCTV image.

CCTVs installed nationwide are already playing a big role in various fields, such as criminal investigations and identification of infection routes in the event of a disease. However, since it is virtually impossible for a person to continuously monitor a number of CCTV images, they are used limitedly as a follow-up role to look back on the incident after it has occurred. In order to overcome this problem, an intelligent CCTV that performs a function of detecting the time when an event occurs by analyzing an image in real time without human assistance has emerged.

However, if false alarms occur too often, the intelligent CCTV requires effort to confirm each time.

In general, the surveillance method of intelligent CCTV targeting people is done by first detecting and tracking people to understand their movements, and then confirming what specific actions of people appear. Although the accuracy of human detection has been greatly improved with the recent development of technologies such as artificial intelligence deep learning, the possibility of false detection and non-detection still exists, which is the main cause of false alarms.

Korean Patent Registration No. 10-1980551 discloses a real-time intelligent CCTV image analysis behavior detection system and method using machine learning object detection. The technology detects an object matching a pre-input reference setting object in an image frame through machine learning, and detects a dangerous behavior object matching the pre-entered object behavior. Even in the case of this technique, there may be errors in object detection, and there is a problem that the possibility of false alarms still exists.

Therefore, there is a need for a detection method to reduce errors in object detection in a monitoring method using an intelligent CCTV, and in particular, a detection method for real-time processing of abnormal behavior monitoring.

An object of the present invention is to provide a CCTV image-based abnormal behavior detection method and apparatus capable of quickly and accurately detecting and tracking a person in a technology for detecting abnormal behavior of a person through CCTV image analysis.

In addition, an object of the present invention is to provide a method for detecting abnormal behavior capable of real-time processing by using a technique that does not require a large amount of computation in detecting and tracking an object and determining an event.

Another object of the present invention is to provide a method for detecting an abnormal behavior capable of accurately detecting a target event and minimizing false alarms by minimizing interference between events to be detected.

A CCTV image-based abnormal behavior detection method according to an embodiment includes: detecting a human object from a CCTV image using a human detection engine; generating a tracking information list based on the detected tracking information of the human object; determining an anomalous behavioral event for a person on the tracking information list; and generating an alarm signal for the abnormal behavior event.

In this case, the step of generating the tracking information list includes: storing the tracking information list of the previous frame of the image as a management list; storing a list of human objects detected in the current frame of the image in a detection list; generating an update list by comparing the management list with the detection list; checking whether the person object is a person based on the moving speed of the person object included in the update list; and comparing the reference image with the current image to determine whether the person has left the screen.

In this case, the generating of the updated list may include: storing a list of human objects included in both the management list and the detection list as an updated list; adding, to the update list, a list of human objects that are not in the management list and are included only in the detection list; performing a person tracking function using a person tracking algorithm for a list of person objects that are not in the detection list and are included only in the management list; and excluding from the update list based on information obtained by performing the person tracking function.

In this case, the step of determining whether the person object is a person may include calculating a movement speed of the person object in the updated list; identifying the person object as a person when the movement speed is within a specified range for the movement speed of the person; and excluding the object, which is not identified as a person, from the update list.

In this case, the step of determining whether the person has left the screen may include: storing a reference image in which no person is included in the image; comparing the reference image with the current image and storing a difference value; determining whether a person in the update list has left the screen based on the difference value; excluding a person who has left the screen from the update list; And it may include the step of storing the updated list excluding the person who deviated from the screen as a tracking information list.

In this case, the step of determining the event may include: determining a wandering intrusion event; determining a collapse event; and determining a fight event.

In this case, the step of determining the intrusion event may include: generating an intrusion event when a person included in the tracking information list enters an intrusion monitoring area; and generating a wandering event when the person enters the wandering monitoring area and stays for a predetermined time or more.

In this case, the step of determining the falling event comprises: calculating the motion intensity in two consecutive time intervals for the person included in the tracking information list; generating a fall event candidate by comparing the two motion intensities; Recognizing the posture of the fall event candidate using a neural network; and generating a falling event when a posture defined as a fall among the recognized postures of the fall event candidate occurs more than a predetermined number of times.

In this case, the determining of the fighting event may include: generating the extracted feature vector by extracting the fighting characteristics of a person included in the tracking information list in a vector form; and generating a fight event from the extracted feature vector using a feature vector classifier.

In addition, the CCTV image-based abnormal behavior detection apparatus according to an embodiment includes a human object detector for detecting a human object from a CCTV image using a human detection engine; a tracking information list generator configured to generate a tracking information list based on the detected tracking information of the human object; an event determination unit for determining an abnormal behavior event for a person in the tracking information list; and an alarm signal generator configured to generate an alarm signal for the event.

According to the present invention, in the technology for detecting abnormal behavior of a person through CCTV image analysis, it is possible to provide a CCTV image-based abnormal behavior detection method and apparatus capable of quickly and accurately detecting and tracking a person.

In addition, according to the present invention, it is possible to provide a method for detecting an abnormal behavior capable of real-time processing by using a technique that does not require a large amount of computation in detecting and tracking an object and determining an event.

In addition, according to the present invention, it is possible to provide an abnormal behavior detection method capable of minimizing false alarms while accurately detecting a target event by minimizing interference between events to be detected.

1 is a block diagram illustrating an example of an apparatus 100 for detecting abnormal behavior based on a CCTV image according to an embodiment.
FIG. 2 is a block diagram illustrating an example of the tracking information list generator 130 shown in FIG. 1 .
FIG. 3 is a block diagram illustrating an example of the update list generator 230 shown in FIG. 2 .
FIG. 4 is a diagram illustrating an example of use of the update list exclusion unit 330 shown in FIG. 3 .
FIG. 5 is a diagram illustrating an example of using the person checking unit 240 shown in FIG. 2 .
FIG. 6 is a block diagram illustrating an example of the event determination unit 140 shown in FIG. 1 .
7 is a block diagram illustrating an example of the collapse event determination unit 620 shown in FIG.
FIG. 8 is a diagram illustrating a time domain for calculating motion intensity in the motion intensity calculator 710 shown in FIG. 7 .
FIG. 9 is a diagram illustrating a posture recognition network, which is a neural network used by the posture recognition unit 730 shown in FIG. 7 .
FIG. 10 is a view showing ten postures used by the posture recognition unit 730 shown in FIG. 7 .
FIG. 11 is a diagram illustrating a feature vector classifier used in the fight event determining unit 630 illustrated in FIG. 6 .
12 is an operation flowchart illustrating an example of a method for detecting abnormal behavior based on a CCTV image according to an embodiment.
13 is an operation flowchart illustrating an example of the step of generating the tracking information list shown in FIG. 12 ( S1230 ).
14 is a diagram showing the configuration of a computer system according to an embodiment.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in a variety of different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the technical field to which the present invention belongs It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

Although "first" or "second" is used to describe various elements, these elements are not limited by the above terms. Such terms may only be used to distinguish one component from another. Accordingly, the first component mentioned below may be the second component within the spirit of the present invention.

The terminology used herein is for the purpose of describing the embodiment and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” or “comprising” implies that the stated component or step does not exclude the presence or addition of one or more other components or steps.

Unless otherwise defined, all terms used herein may be interpreted with meanings commonly understood by those of ordinary skill in the art to which the present invention pertains. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Hereinafter, a CCTV image-based abnormal behavior detection method and an apparatus therefor according to an embodiment will be described in detail with reference to FIGS. 1 to 14 .

1 is a block diagram illustrating an example of an apparatus 100 for detecting abnormal behavior based on a CCTV image according to an embodiment.

Referring to FIG. 1 , the CCTV image-based abnormal behavior detection apparatus 100 according to the embodiment includes a human object detection unit 120 , a tracking information list generation unit 130 , an event determination unit 140 , and an alarm signal generation unit 150 . ) is included.

The human object detection unit 120 detects a human object from the CCTV video stream 110 using a human detection engine. The tracking information list generating unit 130 generates a tracking information list based on the tracking information of the detected human object. The event determination unit 140 determines an abnormal behavior event for a person in the tracking information list. The alarm signal generator 150 generates an alarm signal for the abnormal behavior event.

The human object detection unit 120 and the tracking information list generation unit 130 perform a function of stably tracking a person appearing on the screen from the time of appearance until disappearing out of the screen by receiving the CCTV video stream as an input. Here, the detected and tracked person information is used as information for processing the next frame on the video stream while being managed as a tracking information list. The tracking information list includes information such as the trajectory from the time the person appeared to the present, the motion intensity in each frame, and the like, and the information is transmitted to the event determination unit 140 to determine each event. It can be used as a basis for lowering

The human object detection unit 120 may utilize various human detection engines. The human detection engine can utilize deep learning technology thanks to the recent development of artificial intelligence technology. have. Although the performance of the human detection engine using the deep learning technology as described above has been greatly improved, the probability of erroneous detection (detecting as a human instead of a human) or non-detecting (not detecting a human being) is still not low. Therefore, the present invention proposes a method with a small amount of calculation in order to enable real-time processing while solving the problems of false detection and non-detection. According to an embodiment of the present invention, it is possible to quickly and accurately detect a person based not only on information on a single image but also on information about changes over time.

The tracking information list generating unit 130 stores the tracking information list of the previous frame of the image as a management list, stores a list of human objects detected in the current frame of the image in the detection list, the management list and the detection Compare the list to create an update list. In addition, based on the moving speed of the human object included in the update list, it may be determined whether the human object is a person, and a reference image and a current image may be compared to determine whether the person has departed from the screen.

In order for the tracking information list generating unit 130 to generate the update list, a list of human objects included in both the management list and the detection list is stored as an update list, and the detection list is not in the management list. A list of human objects contained only in . is added to the update list. In addition, the tracking information list generating unit 130 performs a person tracking function using a person tracking algorithm for a list of person objects that are not in the detection list and are included only in the management list, and information obtained by performing the person tracking function An operation of excluding from the update list may be performed based on .

For example, the list of person objects in the detection list, which is the person information list detected by the person detection engine, is displayed as Pd, and the list of person objects in the management list, which is the tracking information list of people managed from the previous frame, is displayed as Po. It will be described in detail below. The management list and the detection list may include not only a list of human objects, but also information about each of the human objects, that is, history information about the trajectory and characteristics of the person object from appearing for the first time in the image to the present. .

'Pd ∩ Po' is a case in which the human object list in the detection list overlaps the human object list in the management list, and it means that the person who has been tracking up to the previous image is detected again in this frame. This can be included in the update list as a log if a person is being tracked and detected normally.

'(Pd ∪ Po) - Po' is a case in which the human object list of the detection list does not overlap the human object list of the management list, meaning that a human object is newly detected in this frame. In this case, a new person object may have appeared in the actual image, or another object may have been erroneously detected. First, it is assumed that the human object has appeared, and a new person list is created and added to the update list.

'(Pd ∪ Po) - Pd' indicates that the human object list of the management list does not overlap the human object list of the detection list, indicating that the human object that was being tracked until the previous frame of the image was not detected in the current frame. it means. In this case, it may be that the human object that was actually tracked went out of the image and deviated, or the human detection engine failed to detect it due to a change in image conditions. Therefore, in this case, a person tracking function is performed using the information of the previous frame to estimate the expected location of the person corresponding to the current frame.

The person tracking function is a very important function for securing the stability of the human object detection unit 120 and the tracking information list generation unit 130 . The human tracking function is a technology that has already been widely studied in the field of computer vision, and many advanced technologies, including particle filtering, have been developed. it is not appropriate In addition, the existing tracking technology generally uses a method of tracking with color information such as the color of a person's clothes. Since the image at night or cloudy weather is almost black and white, the color information disappears and the tracking technology operates properly. won't do it

Therefore, in order to solve the problems of the existing tracking technology as described above, the person tracking function of the present invention first uses a very simple algorithm of the Hill Climbing method called Mean Shift Tracking. In addition, the Histogram Back Projection method suitable for expressing color information is not generally used for the Probability Image, which is the target of the Hill Climbing, but a Background Subtraction Image is used based on the fact that the field of view of the CCTV is fixed. Here, the Probability Image refers to an image in which the area where the tracking target is located is brightly expressed, and the area where the tracking target is not is darkened.

Thereafter, the tracking information list generating unit 130 calculates the moving speed of the person object in the updated list, and when the moving speed is within a range designated as the moving speed of the person, the operation of identifying the person object as a person is performed. can be done In addition, an operation of excluding the person object, which is not identified as a person, from the update list may be performed. That is, the tracking information list generating unit 130 checks whether the human object in the updated list is erroneously created due to erroneous detection.

In general, assuming that the camera is fixed in the CCTV environment, it can be assumed that the person is moving from the time the person first appears on the screen. And the moving speed will be within the range of the speed at which a person can move by walking or running. Accordingly, the tracking information list generator determines whether the detected existence is really a person by evaluating the history of location information of each person object in the updated list.

If the moving speed is very slow, there is almost no movement, and it is a case that a fixed object other than a person is mistakenly detected as a person. It can be said that the object is incorrectly judged as a person.

To evaluate the moving speed, the moving distance D and the moving time T in units of pixels during the last N frames are calculated from the history managed by the specific person pi, and the vertical length H in pixels of the person pi is calculated. In addition, by estimating the average height of the person to be about 1.7 m, the movement speed V of the person may be calculated as in Equation 1 below.

Here, V is the movement speed of a person, D is the movement distance in pixels of a person, H is the vertical length of a person in pixels, and T is the movement time of the person.

When the calculated movement speed is outside the specified range, it is determined that the person pi is not a person, and is removed from the updated list, and this operation is performed for all human objects in the updated list. When all of the above operations are completed, all of the human objects remaining in the update list are identified as people.

Also, the tracking information list generator 130 may store a reference image in which no person is included in the image, compare the reference image with the current image, and store a difference value. In addition, based on the difference value, it may be determined whether the person in the update list leaves the screen, and the person who leaves the screen may be excluded from the update list. Finally, the updated list in which a person who has left the screen is excluded may be stored as a tracking information list.

If the person disappears from the screen, of course, the person detection engine will not be able to detect the person. The person tracking function described above can be performed to check whether a person has moved off the screen, but the problem is that it is not easy to determine whether the person has disappeared or the person detection engine has failed to detect the person.

To overcome this, the tracking information list generator utilizes a background subtraction image. The background subtraction image refers to a difference image between the reference image and the current image. If an object that was not in the original reference image appears in the current image, the difference value in the region increases. Since there is no person in the original reference image, if a person is present in the current image, the difference value is large, and if the person disappears, the difference value is small. Accordingly, if the difference value is less than a predetermined value and the human detection engine fails to detect a person more than a predetermined number of times, the person is deemed to have left the video and the person is removed from the update list.

In this case, since the reference image for performing the background subtraction actually changes over time due to minute movements of the camera or changes in lighting conditions due to changes in weather, it is preferable to update them at regular intervals. However, when updating when there is a person, the reference image can be updated only when the management list (the tracking information list of the previous frame) is empty, because even people are included in the reference image.

As described above, after removing all the deviated persons from the update list, features to be used in the event determination unit 140 are extracted from the persons in the updated list. This characteristic may include a rectangle indicating the current area of the person, the intensity of the Frame Difference Image and the intensity of the Background Subtraction Image within the rectangle.

After the feature extraction is finished, the update list is stored as a tracking information list, and the tracking information list is used as a management list Po when processing the next frame.

The event determination unit 140 may determine a wandering intrusion event, a falling event, and a fighting event indicating an abnormal situation.

In order for the event determination unit 140 to determine a roaming intrusion event, when a person included in the tracking information list enters the intrusion monitoring area, an intrusion event is generated, and when the person enters the wandering monitoring area and stays for a certain period of time or more, a wandering event causes

That is, the event determination unit 140 evaluates whether a person enters a pre-designated area for a person included in the tracking information list. And when the rectangular area representing a person enters the designated intrusion monitoring area, an intrusion event is generated. will be.

The event determination unit 140 calculates the motion intensity in two consecutive time intervals for a person included in the tracking information list in order to determine the fall event, and compares the two motion intensity to be a fall event candidate creates Then, the posture of the fall event candidate is recognized using a neural network, and when the posture defined as a fall among the recognized postures of the fall event candidate occurs more than a predetermined number of times, a fall event is generated.

In order for the event determination unit 140 to determine a case in which a person who appears on the screen suddenly collapses, basically, a region in which the person being tracked suddenly stops moving can be found. However, since the same characteristics may appear in the case of a person walking down the street standing up or squatting, etc., it is better to go through the process of checking the person's posture in the final state and confirming whether a fall event has really occurred. desirable.

First, in order to detect a state in which a person abruptly stops moving, the region from t=t _{-2N to t=t -N} among the last 2N frames of each person's trajectory information is R1, t=t _-N to t=t ₀ area up to Divide by R2 to extract the intensity of motion in two regions.

Assuming that the motion intensity in R1 is m1 and the motion intensity in R2 is m2, if a person who was moving normally _{suddenly collapses and stops moving after t=t -N} , m1 has a large value, and m2 is will have a small value. Therefore, if the condition that m1/m2 is _{greater than the threshold value (th ratio} ), m1 is greater than the threshold value (th _min ), and m2 is less than the threshold value (th _max ) is satisfied, the collapse event at time _{t -N} can be considered to have occurred. The th _ratio , th _min , th _max is a value set by the user or system.

However, in consideration of the time it takes to fall down and the time it takes to transition to a stationary state with no movement after falling, the counter is incremented if the above conditions are satisfied, and if the above conditions are not met, the above By decrementing the counter, it can be determined that a candidate for the falling event has occurred when the counter finally exceeds a predetermined number of times (th _{count ).}

Also, among the fall event candidates, a case in which a person's movement is stopped, such as when a walking person stops on the screen or squats down as in the case of wandering, is included. Therefore, in order to exclude such a case, _{it is necessary to confirm that the final state, that is, the posture of the person at the time point after t = t 0} is a collapsed state, not a standing or sitting state.

In order to recognize the posture of the falling event candidate, the event determination unit 140 uses a posture recognition network composed of a deep neural network. The posture recognition network first uses the pre-trained network of VGG19 for feature extraction, and changes the 4,096 dimension vector, which is the result of feature extraction of VGG19, into a 128 dimension vector through the Fully Connected Layer, and again through the Fully Connected Layer. Finally, it is classified into 10 classes.

The posture recognition network is trained to classify 10 postures using about 50,000 images. The 10 postures are Standing normal, Standing bent, Sitting sofa, Sitting chair, Sitting floor, Sitting squat, Lying face, Lying back, Lying side, Lying crouched. Among them, since the four postures of the Lying face, Lying back, Lying side, and Lying crouched are the postures in the collapsed state, when the four postures are detected more than a predetermined number of times, it can be determined that a collapse event has finally occurred.

In order to determine the fighting event, the event determination unit extracts the fighting characteristics of the person included in the tracking information list in a vector form to generate an extracted characteristic vector, and fights using a characteristic vector classifier from the extracted characteristic vector. events can occur.

In general, a fight is accompanied by intense motion, so a fight can be determined by measuring the intensity of the motion. However, it should be taken into account that similar characteristics may occur in scenes where multiple people move quickly, even if not in a fight. Therefore, after extracting various features accompanying a fight, such as motion intensity, distance between people, and changes in positional relationship, in the form of vectors shown in Table 1 below, a method of detecting a fight event through a feature vector classifier is used. Feature vectors for detecting a fight in Table 1 below are obtained from tracking information of the most recent K frames among tracking information for each person.

characteristic quantity Description of the relationship with the fight event Average Movement When a fight is taking place, the movement of people becomes intense, so naturally, the movement (strength of the difference between frames in the human domain) becomes stronger. Min Aspect Ratio / Max Aspect Ratio In a fighting state, the Aspect Ratio of the rectangle representing a person changes due to violent motions such as extending arms or legs. Therefore, compared to just walking, the difference between Max and Min increases, and this feature tends to decrease. Variance of Aspect Ratio It expresses how much the above Aspect Ratio changes. Min Width / Max Width For the same reason as the Aspect Ratio, the Rectangle that represents a person's position also changes significantly. Min Height / Max Height Same as above Average Speed in Meters The position that represents the person also undergoes a severe change, so the speed increases. Variance of X position In the state of a fight, the position tends to change frequently within a certain range (a change in the situation of sticking with the opponent occurs), and the variance increases. Variance of Y position Same as above Max X position - Min X Position When walking or running, it moves in a certain direction, so it tends to increase or decrease monotonously, whereas in a fight situation it tends to change frequently within a very small range. Max Y Position - Min Y Position Same as above

The feature vector classifier used by the event determination unit may be configured as a shallow neural network having a simple structure. That is, in order to classify whether the feature vector corresponds to a fight or other normal behavior, a shallow neural network composed of an input layer, a hidden layer, and an output layer is simply used.

After determining the roaming intrusion event, the fall event, and the fighting event in the event determination unit as described above, the alarm signal generation unit 150 generates an abnormal behavior event alarm signal 160 according to the event determination result of the event determination unit 140 . ) will be created.

FIG. 2 is a block diagram illustrating an example of the tracking information list generator 130 shown in FIG. 1 .

Referring to FIG. 2 , the tracking information list generation unit 130 includes a management list generation unit 210 , a detection list generation unit 220 , an update list generation unit 230 , a person identification unit 240 , and a screen deviation verification unit. 250 may be included.

The management list generation unit 210 stores the tracking information list of the previous frame of the image as a management list. The detection list generator 220 stores a list of human objects detected in the current frame of the image in the detection list. The update list generator 230 compares the management list with the detection list to generate an update list. The person checking unit 240 checks whether the person object is a person based on the moving speed of the person object included in the update list. The screen departure confirmation unit 250 compares the reference image with the current image to confirm whether the person has departed from the screen.

The update list generator 230 stores a list of human objects included in both the management list and the detection list as an update list, and stores a list of human objects that are not in the management list and are included only in the detection list in the update list. add The update list generator 230 performs a person tracking function using a person tracking algorithm for a list of person objects that are not in the detection list and are included only in the management list, and information obtained by performing the person tracking function Based on this, an operation of excluding from the update list may be performed.

For example, the list of person objects in the detection list, which is the person information list detected by the person detection engine, is displayed as Pd, and the list of person objects in the management list, which is the tracking information list of people managed from the previous frame, is displayed as Po. It will be described in detail below.

'Pd ∩ Po' is a case in which the human object list of the detection list overlaps the human object list of the management list, and it means that the person who has been tracking up to the previous video is detected again in this frame, so it can be included in the update list as it is. '(Pd ∪ Po) - Po' is a case in which the human object list in the detection list does not overlap the human object list in the management list, and it means that a human object is newly detected in this frame. Just create it and add it. And '(Pd ∪ Po) - Pd' is a case in which the human object list of the management list does not overlap the human object list of the detection list, and the human object that was being tracked until the previous frame of the image was not detected in the current frame. Therefore, in this case, the person tracking function is performed using the information of the previous frame to estimate the location where the person corresponding to the current frame is expected.

Here, the person tracking function may use a very simple algorithm of hill climbing method called mean shift tracking. In addition, the background subtraction image can be used by noting that the field of view of the CCTV is fixed without using the Histogram Back Projection method suitable for expressing color information for the Probability Image, which is the target of the Hill Climbing. Here, the Probability Image refers to an image in which the area where the tracking target is located is brightly expressed, and the area where the tracking target is not is darkened.

Thereafter, the person checking unit 240 calculates the moving speed of the person object in the updated list generated by the update list generating unit 230, and if the moving speed is within a range designated as the moving speed of the person, the person You can perform an action that identifies an object as a person. In addition, an operation of excluding the person object, which is not identified as a person, from the update list may be performed. That is, the person checking unit 240 checks whether or not the person object in the update list is erroneously created due to erroneous detection.

In general, assuming that the camera is fixed in the CCTV environment, it can be assumed that the person is moving from the time the person first appears on the screen. And the moving speed will be within the range of the speed at which a person can move by walking or running. Accordingly, the person checking unit 240 determines whether the detected existence is really a person by evaluating the history of location information of each person object in the updated list.

To evaluate the moving speed, the moving distance D and the moving time T in units of pixels for the last N frames are calculated from the history managed by the specific person pi, and the length H in units of pixels of the person pi is calculated. And by estimating that the average height of the person is about 1.7 m, the movement speed V of the person can be calculated as in Equation 1 above.

When the calculated movement speed is outside the specified range, it is determined that the person pi is not a person, and is removed from the updated list, and this operation is performed for all human objects in the updated list. When all of the above operations are completed, all of the human objects remaining in the update list are identified as people.

The screen departure confirmation unit 250 may store a reference image in which no person is included in the image, compare the reference image with the current image, and store a difference value. In addition, based on the difference value, it may be determined whether the person in the update list leaves the screen, and the person who leaves the screen may be excluded from the update list. Finally, the updated list in which a person who has left the screen is excluded may be stored as a tracking information list.

To track the person disappearing from the screen, the person tracker is activated to create a new tracking area. However, since it is not easy to determine whether a person disappears or a person detection engine fails to detect a person, the screen departure confirmation unit uses a background subtraction image to overcome this. The background subtraction image refers to a difference image between the reference image and the current image. If an object that was not in the original reference image appears in the current image, the difference value in the region increases. Since there is no person in the original reference image, if a person is present in the current image, the difference value is large, and if the person disappears, the difference value is small. Accordingly, if the difference value is less than a predetermined value and the human detection engine fails to detect a person more than a predetermined number of times, the person is deemed to have left the video and the person is removed from the update list.

In this case, it is preferable to update the reference image for performing the background subtraction at regular time intervals. However, when updating when there is a person, the reference image can be updated only when the management list (the tracking information list of the previous frame) is empty, because even people are included in the reference image.

After removing all the people who have departed from the update list as described above, features to be used in the event determination unit are extracted from the people in the update list, and the updated list is stored as a tracking information list, and the tracking information list will be used as the management list Po when processing the next frame.

FIG. 3 is a block diagram illustrating an example of the update list generator 230 shown in FIG. 2 .

Referring to FIG. 3 , the update list generation unit 230 may include an update list storage unit 310 , an update list addition unit 320 , and an update list exclusion unit 330 . The update list adding unit may be referred to as a new person creation unit, and the update list exclusion unit may be referred to as a person tracking unit.

The update list storage unit 310 stores a list of human objects included in both the management list and the detection list as an update list. The update list adding unit 320 adds a list of human objects that are not in the management list and are included only in the detection list to the update list. The update list exclusion unit 330 performs a person tracking function using a person tracking algorithm for a list of person objects that are not in the detection list and are included only in the management list, and based on the information obtained by performing the person tracking function to exclude from the update list.

For example, the list of person objects in the detection list, which is the person information list detected by the person detection engine, is displayed as Pd, and the list of person objects in the management list, which is the tracking information list of people managed from the previous frame, is displayed as Po. It will be described in detail below.

'Pd ∩ Po' is a case in which the human object list in the detection list overlaps the human object list in the management list, and it means that the person who has been tracking up to the previous image is detected again in this frame. Therefore, the update list storage unit 310 may consider 'Pd ∩ Po' as a case in which a person is normally tracked and continuously detected, and may be included in the update list as it is.

'(Pd ∪ Po) - Po' is a case in which the human object list in the detection list does not overlap the human object list in the management list, meaning that a human object is newly detected in this frame. In this case, a new human object may have appeared in the actual image, or the human object may have been erroneously detected. Accordingly, the update list adder 320 may create and add a new person list to the update list, assuming that both '(Pd ∪ Po) - Po' have human objects. In case of erroneous detection, it is later removed by the human identification unit.

' (Pd ∪ Po) - Pd' indicates that the human object list of the management list does not overlap the human object list of the detection list, indicating that the human object that was being tracked until the previous frame of the image was not detected in the current frame. it means. In this case, it may be that the human object that was actually tracked went out of the image and deviated, or it could be a case of erroneous detection in which the human detection engine failed to detect due to a change in image conditions. Therefore, the update list exclusion unit 330 performs a person tracking function using the information of the previous frame for all of '(Pd ∪ Po) - Pd' to estimate the location where the person corresponding to the current frame is expected to be. It is determined whether the case is erroneously detected or whether the human object has left the screen, and is excluded from the update list.

The person tracking function of the update list exclusion unit 330 may use advanced technologies such as particle filtering for the person tracking function. Information is tracked based on the information, but since the image becomes almost black and white in an image at night or in cloudy weather, the color information disappears and the image does not operate properly.

Therefore, the person tracking function according to the present invention uses a very simple algorithm of the hill climbing method called Mean Shift Tracking first, and the background subtraction image is obtained by paying attention to the fact that the field of view of the CCTV is fixed for the Probability Image that is the target of the hill climbing. use.

FIG. 4 is a diagram illustrating an example of use of the update list exclusion unit 330 shown in FIG. 3 .

An example in which the person tracking function of the update list exclusion unit 330 is utilized will be described with reference to FIG. 4 .

The person tracking function of the update list exclusion unit 330 uses a very simple algorithm of Hill Climbing called Mean Shift Tracking. In addition, the Histogram Back Projection method suitable for expressing color information is not generally used for the Probability Image, which is the target of the Hill Climbing, but a Background Subtraction Image is used based on the fact that the field of view of the CCTV is fixed. Here, the Probability Image refers to an image in which the area where the tracking target is located is brightly expressed, and the area where the tracking target is not is darkened.

4 shows the contrast between the image 420 in which Histogram Back Projection is performed on the person to be tracked in the original image 410 of CCTV and the image 430 in which the background subtraction is performed in order to show the superiority of the person tracking function according to the present invention. are giving

First, the color characteristics of a person in the original image 410 are not significantly different from the background. Accordingly, in the image 420 on which the histogram back projection has been performed, the human region appears faintly and is almost invisible, whereas in the image 430 on which the back ground subtraction is performed, the human region is displayed much more clearly.

The Mean Shift Tracker iteratively finds a bright area on the Probability Map. Since the human area of the image 430 on which the Back Ground Subtraction has been performed according to the present invention is displayed more clearly and brightly, it is much more stable when used as a Probability Image. tracking becomes possible.

And in the case of a cloudy weather image or a night image, the color characteristics of a person appear almost the same as the background, so the Back Ground Subtraction method is much more effective than the Histogram Back Projection.

FIG. 5 is a diagram illustrating an example of using the person checking unit 240 shown in FIG. 2 .

Referring to FIG. 5 , it will be described that the person checking unit 240 calculates the movement speed of a person and checks whether the person object in the update list is erroneously created due to erroneous detection.

An image in which a person appears is shown in FIG. 5 , and the detected area 510 of the person is indicated by a rectangle. In general, assuming that the camera is fixed in the CCTV environment, it can be assumed that the person is moving from the time the person first appears on the screen. And it can be assumed that the movement speed is generally within the range of the speed that a person can move by walking or running.

The movement speed of a person in the detected human area 510 is calculated by calculating a movement distance D and a movement time T in units of pixels during the last N frames, and calculating the length H of the person pi in units of pixels. And by estimating that the average height of the person is about 1.7 m, the movement speed V of the person can be calculated as in Equation 1 above.

When the calculated movement speed is outside the specified range, it is determined that the person is not a person and is removed from the updated list. In general, when the movement speed is very slow, there is almost no movement, and a fixed object, not a person, is mistakenly detected as a person. This may be a case in which the moved object is incorrectly judged as a person.

FIG. 6 is a block diagram illustrating an example of the event determination unit 140 shown in FIG. 1 .

Referring to FIG. 6 , the event determining unit 140 may include a roaming invasive event determining unit 610 , a falling event determining unit 620 , and a fighting event determining unit 630 .

The roaming intrusion event determining unit 610 generates an intrusion event when a person included in the tracking information list generated by the tracking information list generation unit enters the intrusion monitoring area, and when the person enters the wandering monitoring area and stays for more than a certain time, the person wanders event can be generated.

That is, the roaming intrusion event determining unit 610 evaluates whether a person enters a pre-designated area for a person included in the tracking information list. And when the rectangular area representing a person enters the designated intrusion monitoring area, an intrusion event is generated. will be.

The falling event determination unit 620 calculates the motion intensity in two consecutive time intervals for a person included in the tracking information list generated by the tracking information list generation unit, and compares the two motion intensities to fall Create event candidates. Then, the posture of the fall event candidate is recognized using a neural network, and when the posture defined as a fall among the recognized postures of the fall event candidate occurs more than a predetermined number of times, a fall event is generated.

In order to determine when a person who appears on the screen suddenly collapses, the falling event determining unit 620 may basically find a region where the person being tracked suddenly stops moving. However, since the same characteristics may appear in the case of a person walking down the street standing up or squatting, etc., it is better to go through the process of checking the person's posture in the final state and confirming whether a fall event has really occurred. desirable.

First, in order to detect a state in which a person abruptly stops moving, the region from t=t _{-2N to t=t -N} among the last 2N frames of each person's trajectory information is R1, t=t _-N to t=t ₀ area up to Divide by R2 to extract the intensity of motion in two regions.

Assuming that the motion intensity in R1 is m1 and the motion intensity in R2 is m2, if a person who was moving normally _{suddenly collapses and stops moving after t=t -N} , m1 has a large value, and m2 is will have a small value. Thus, m1 is greater than the threshold value 1 by comparing the value of m2 is determined m1 / m2, m1 is the case which satisfies the condition is greater than the predetermined threshold value 2, m2 is smaller than the predetermined threshold value 3, the time t _-N It can be determined that a collapse event has occurred. However, in consideration of the time it takes to fall down and the time it takes to transition to a stationary state with no movement after falling, the counter is incremented if the above conditions are satisfied, and if the above conditions are not met, the above By decrementing the counter, it can be determined that a candidate for the collapse event has occurred when the counter finally exceeds the predetermined number of times.

Also, among the fall event candidates, a case in which a person's movement is stopped, such as when a walking person stops on the screen or squats down as in the case of wandering, is included. Therefore, in order to exclude such a case, _{it is necessary to confirm that the final state, that is, the posture of the person at the time point after t = t 0} is a collapsed state, not a standing or sitting state.

In order to recognize the posture of the fall event candidate, the fall event determining unit 620 uses a posture recognition network composed of a deep neural network. The posture recognition network first uses the pre-trained network of VGG19 for feature extraction, and changes the 4,096 dimension vector, which is the result of feature extraction of VGG19, into a 128 dimension vector through the Fully Connected Layer, and again through the Fully Connected Layer. Finally, it is classified into 10 classes.

The posture recognition network is trained to classify 10 postures using about 50,000 images. The 10 postures are Standing normal, Standing bent, Sitting sofa, Sitting chair, Sitting floor, Sitting squat, Lying face, Lying back, Lying side, Lying crouched. Among them, since the four postures of the Lying face, Lying back, Lying side, and Lying crouched are the postures in the collapsed state, when the four postures are detected more than a predetermined number of times, it can be determined that a collapse event has finally occurred.

The fight event determining unit 630 generates an extracted feature vector by extracting fighting characteristics in a vector form for a person included in the tracking information list generated by the tracking information list generation unit, and from the extracted characteristic vector A fight event can be generated using a feature vector classifier.

Since a fight entails intense motion, a fight can be judged by measuring the intensity of the motion. However, it should be taken into account that similar characteristics may occur in scenes where multiple people move quickly, even if not in a fight. Therefore, after extracting various features accompanying a fight, such as motion intensity as well as a change in the distance and positional relationship between people, in the vector form of Table 1 above, a method of detecting a fight event through a feature vector classifier is used. The feature vectors of Table 1 are obtained from the tracking information of the latest K frames among the tracking information for each person.

The feature vector classifier used by the fight event determining unit 630 may be configured as a shallow neural network having a simple structure. That is, in order to classify whether the feature vector corresponds to a fight or other normal behavior, a shallow neural network composed of an input layer, a hidden layer, and an output layer is simply used.

As described above, the result of the roaming intrusion event determining unit 610, the falling event determining unit 620, and the fighting event determining unit 630 determining each of the roaming intrusion events, the falling event and the fighting event is sent to the next step, the alarm signal generation unit. will send And the roaming intrusion event determining unit 610, the falling event determining unit 620, and the fighting event determining unit 630 may determine each event at the same time or in reverse order.

7 is a block diagram illustrating an example of the collapse event determination unit 620 shown in FIG.

Referring to FIG. 7 , the falling event determination unit 620 may include a motion intensity calculation unit 710 , a falling event candidate generation unit 720 , a posture recognition unit 730 , and a falling event generation unit 740 . have.

The motion intensity calculation unit 710 calculates the motion intensity in two consecutive time intervals for the person included in the tracking information list. The falling event candidate generating unit 720 generates a falling event candidate by comparing the two motion intensities. The posture recognition unit 730 recognizes the posture of the falling event candidate using a neural network. The fall event generating unit 740 generates a fall event when the posture defined as a fall among the recognized postures of the fall event candidate occurs more than a predetermined number of times.

The fall event determination unit is a part that determines when a person who appears on the screen suddenly collapses. Basically, it finds an area where the person being tracked suddenly stops moving, but the person walking on the road stops in place Since the same characteristics may appear in cases such as squatting or squatting, a process of confirming whether a collapse event has actually occurred by checking the posture of the person in the final state is performed.

First, in order to detect a state in which a person abruptly stops moving, the motion intensity calculation unit 710 calculates a region from t = t _{-2N to t = t -N} among the last 2N frames of each person's trajectory information R1, t =t _-N to t=t ₀ Divide by R2 to extract the intensity of motion in two regions.

Assuming that the motion intensity in R1 is m1 and the motion intensity in R2 is m2, if a person who was moving normally _{suddenly collapses after t=t -N} , m1 will have a large value, and m2 will have a small value. will have Therefore, if the condition that m1/m2 is _{greater than the threshold value (th ratio} ), m1 is greater than the threshold value (th _min ), and m2 is less than the threshold value (th _max ) is satisfied, the collapse event at time _{t -N} can be considered to have occurred. However, taking into account the time it takes to fall down and the time it takes to transition to a motionless state after falling down, if the above conditions are satisfied, the counter is incremented, and if the above conditions are not satisfied, the counter is decremented, Finally, when the counter exceeds a predetermined number of times (th _count ), it is determined that a candidate for the fall event has occurred.

Among the fall event candidates, a case in which a person's movement stops, such as when a walking person stops on the screen or squats down as in the case of wandering, may be included. Therefore, in order to exclude this case, the posture recognition unit 730 needs to confirm that the posture of the person is a collapsed state rather than a standing or sitting state at a time point after _{t = t 0 .}

The posture recognition unit 730 uses a posture recognition network composed of a deep neural network to recognize the posture of the falling event candidate. The posture recognition network first uses the pre-trained network of VGG19 for feature extraction, and changes the 4,096 dimension vector, which is the result of feature extraction of VGG19, into a 128 dimension vector through the Fully Connected Layer, and again through the Fully Connected Layer. Finally, it is classified into 10 classes.

And the posture recognition network is trained to classify 10 postures using about 50,000 images. The 10 postures are Standing normal, Standing bent, Sitting sofa, Sitting chair, Sitting floor, Sitting Squat, Lying face, Lying back, Lying side, Lying crouched.

The fall event generating unit 740 determines that a fall event has occurred, and finally, when four postures of Lying face, Lying back, Lying side, and Lying crouched are detected more than a predetermined number of times among the 10 postures, the fall event is determined can occur

FIG. 8 is a diagram illustrating a time domain for calculating motion intensity in the motion intensity calculator 710 shown in FIG. 7 .

In FIG. 8 , R1 810 denotes _{a region from t=t -2N} to t=t _-N , and R2 820 denotes a region from t=tN to t=t0. Here, m1 is the motion intensity in R1, and m2 is the motion intensity in R2.

If it is _{assumed that a person suddenly collapses and stops moving after t = t -N} , m1 will have a large value, whereas m2 will have a small value. Accordingly, if condition 1 is satisfied that m1/m2 is _{greater than the threshold value (th ratio} ), m1 is greater than the threshold value (th _min ), and m2 is less than the threshold value (th _max ), it collapses at time _{t -N} It can be determined that an event has occurred. The th _ratio , th _min , and th _max may be determined by a user.

However, in consideration of the time it takes to fall down and the time it takes to transition to a stationary state with no movement after falling, the counter is incremented if the above conditions are satisfied, and if the above conditions are not met, the above By decrementing the counter, it can be determined that a candidate for the falling event has occurred when the counter finally exceeds a predetermined number of times (th _{count ).}

FIG. 9 is a diagram illustrating a posture recognition network used by the posture recognition unit 730 illustrated in FIG. 7 .

Referring to FIG. 9 , the posture recognition unit 730 includes a posture recognition network, which includes a VGG19 Pre-trained Network 910 , a first Fully Connected Layer 920 , and a second Fully Connected Layer 930 . It can be composed of a deep neural network including

The pre-trained network 910 of the VGG19 is a network used for feature extraction, and the first Fully Connected Layer 920 changes the 4,096 dimension vector, which is the feature extraction result of the VGG19, into a 128 dimension vector. And the second Fully Connected Layer 930 classifies the 128 dimension into 10 final classes.

The 10 classes refer to 10 postures used in the present invention, which are Standing normal, Standing bent, Sitting sofa, Sitting chair, Sitting floor, Sitting squat, Lying face, Lying back, Lying side, Lying crouched.

FIG. 10 is a view showing ten postures used by the posture recognition unit 730 shown in FIG. 7 .

10, the 10 postures are Standing normal (1000), Standing bent (1010), Sitting sofa (1020), Sitting chair (1030), Sitting floor (1040), Sitting Squat (1050), Lying face (1060), Lying back (1070), Lying side (1080), it may include a Lying crouched (1090). The 10 postures are postures extracted using about 50,000 images.

The posture recognition network composed of the deep neural network of the posture recognition unit 730 is trained to finally classify the postures of the fall event candidates into the 10 postures.

Thereafter, the collapsed event generating unit is the final result of the posture recognition network, Lying face (1060), Lying back (1070), Lying side (1080), Lying crouched (1090) determined to be a collapsed posture When it becomes four postures, It can be considered a fall. Alternatively, in order to be careful in the determination, when the four postures are detected more than a predetermined number of times for the same fall event candidate, it may be determined that the fall event has occurred and the fall event may be generated.

FIG. 11 is a diagram illustrating a feature vector classifier used in the fight event determining unit 630 illustrated in FIG. 6 .

Referring to FIG. 11 , the feature vector classifier for determining the fight event of the fight event determining unit 630 may include an input layer 1120 , a hidden layer 1130 , and an output layer 1140 .

The feature vector classifier receives a feature vector 1110 as shown in Table 1 extracted with respect to a person included in the tracking information list as an input. Then, whether the feature vector 1110 corresponds to a fight or other normal behavior is classified using a shallow neural network.

12 is an operation flowchart illustrating an example of a method for detecting abnormal behavior based on a CCTV image according to an embodiment.

Referring to FIG. 12 , first, a CCTV image is input to the CCTV image-based abnormal behavior detection device (S1210).

The human object detection unit detects a human object from the CCTV video stream 110 using a human detection engine (S1220).

The human object detection unit may utilize various human detection engines. The human detection engine can utilize deep learning technology thanks to the recent development of artificial intelligence technology. have. Although the performance of the human detection engine using the deep learning technology as described above has been greatly improved, the probability of erroneous detection (detecting as a human instead of a human) or non-detecting (not detecting a human being) is still not low. Therefore, the present invention proposes a method with a small amount of calculation in order to enable real-time processing while solving the problems of false detection and non-detection. According to an embodiment of the present invention, it is possible to quickly and accurately detect a person based not only on information on a single image but also on information about changes over time.

The tracking information list generator generates a tracking information list based on the detected tracking information of the human object (S1230).

The tracking information list generation unit stores the tracking information list of the previous frame of the image as a management list, stores a list of human objects detected in the current frame of the image in the detection list, and compares the management list with the detection list. Create an update list. In addition, based on the moving speed of the human object included in the update list, it may be determined whether the human object is a person, and a reference image and a current image may be compared to determine whether the person has departed from the screen.

The tracking information list generating unit stores a list of human objects included in both the management list and the detection list as an update list in order to perform the operation of generating the update list, and a person who is not in the management list but is included in the detection list only Add a list of objects to the update list. And the tracking information list generating unit performs a person tracking function using a person tracking algorithm for a list of person objects that are not in the detection list and are included only in the management list, and based on the information obtained by performing the person tracking function, An operation to exclude from the update list can be performed.

Thereafter, the tracking information list generating unit may calculate the movement speed of the person object in the updated list, and if the movement speed is within a range designated as the movement speed of the person, the operation of identifying the person object as a person may be performed. . In addition, an operation of excluding the person object, which is not identified as a person, from the update list may be performed. That is, the tracking information list generating unit checks whether the human object in the updated list is erroneously created due to erroneous detection.

Also, the tracking information list generator may store a reference image in which no person is included in the image, compare the reference image with the current image, and store a difference value. In addition, based on the difference value, it may be determined whether the person in the update list leaves the screen, and the person who leaves the screen may be excluded from the update list. Finally, the updated list in which a person who has left the screen is excluded may be stored as a tracking information list.

The event determination unit determines an abnormal behavior event for a person in the tracking information list, determines a wandering intrusion event (S1240), determines a fall event (S1250), and determines a fight event (S1260). Each of the steps of determining the roaming intrusion event, the falling event, and the fighting event may be performed simultaneously or in the reverse order.

The event determination unit generates an intrusion event when a person included in the tracking information list enters the intrusion monitoring area in order to perform the step (S1240) of determining the roaming intrusion event, and the person enters the roaming monitoring area for a certain period of time or more If it stays, it triggers a wandering event.

In order to perform the step (S1250) of the event determination unit determining the fall event, the motion intensity is calculated in two consecutive time intervals for the person included in the tracking information list, and the two motion intensities are compared. Create a fall event candidate. Then, the posture of the fall event candidate is recognized using a neural network, and when the posture defined as a fall among the recognized postures of the fall event candidate occurs more than a predetermined number of times, a fall event is generated.

In order to perform the step (S1260) of the event determination unit determining the fighting event, the fighting characteristic is extracted in a vector form for a person included in the tracking information list to generate an extracted characteristic vector, and from the extracted characteristic vector A fight event can be generated using a feature vector classifier.

The alarm signal generation unit generates an alarm signal for the abnormal behavior event determined by the event determination unit (S1270).

13 is an operation flowchart illustrating an example of the step of generating the tracking information list shown in FIG. 12 ( S1230 ).

Referring to FIG. 13 , first, the management list generator and the detection list generator generate each management list and the detection list ( S1310 ).

The management list generating unit stores the tracking information list of the previous frame of the input image as a management list. The detection list generator stores a list of human objects detected in the current frame of the input image in the detection list.

The update list storage unit of the update list generator stores the update list (S1320). That is, the list of human objects included in both the management list and the detection list is stored as an update list.

If the list of human objects in the detection list, which is the person information list detected by the human detection engine, is represented by Pd, and the list of human objects in the management list, which is the tracking information list of people managed from the previous frame, is represented by Po, then 'Pd ∩ Po' is a case in which the human object list in the detection list overlaps the human object list in the management list, and it means that the person who has been tracking up to the previous video has been detected again in this frame, so it can be included in the update list as it is.

The update list adding unit of the update list generating unit creates a new person (S1330). That is, a list of human objects that are not in the management list and are included only in the detection list is added to the update list. For example, '(Pd ∪ Po) - Po' is a case in which the person object list in the detection list does not overlap the person object list in the management list, and it means that a human object is newly detected in this frame, so it is added to the update list. Create and add a new person list.

The update list exclusion unit of the update list generator performs a person tracking function to track a person and excludes it from the update list (S1340). That is, a person tracking function is performed using a person tracking algorithm for a list of person objects that are not in the detection list but only included in the management list, and excluded from the update list based on information obtained by performing the person tracking function action can be performed. For example, '(Pd ∪ Po) - Pd' is a case in which the human object list of the management list does not overlap with the human object list of the detection list. In this case, the person tracking function is performed using the information of the previous frame to estimate the location where the person corresponding to the current frame is expected to be.

When the update list generation unit generates the updated list, the person check unit determines whether the person object in the updated list is a human by calculating a moving speed (S1350). That is, the person identification unit may calculate a movement speed of the person object in the update list, and if the movement speed is within a range designated as the movement speed of the person, the person checker may perform an operation of identifying the person object as a person. In addition, an operation of excluding the person object, which is not identified as a person, from the update list may be performed. That is, the person checking unit checks whether the person object in the update list is erroneously created due to erroneous detection.

If it is determined that the person object is not a person, it is excluded from the update list (S1360), and if it is determined that the person object is a person, the process proceeds to the next step. This operation is repeated for all human objects included in the update list.

And the screen departure unit determines whether a person included in the update list has left the screen (S1370). That is, the screen departure unit may store a reference image in which no person is included in the image, compare the reference image with the current image, and store a difference value. In addition, based on the difference value, it may be determined whether the person in the update list leaves the screen, and the person who leaves the screen may be excluded from the update list. Finally, the updated list in which a person who has left the screen is excluded may be stored as a tracking information list.

If it is determined that the person has departed from the screen, it is excluded from the update list (S1380). If it is determined that the person has not left the screen, the process proceeds to the next step. This operation is also repeated for all persons included in the update list. And the screen exit unit generates a tracking information list by storing the last updated list as a tracking information list (S1390).

14 is a diagram showing the configuration of a computer system according to an embodiment.

The CCTV image-based abnormal behavior detection apparatus according to the embodiment may be implemented in the computer system 1400 such as a computer-readable recording medium.

Computer system 1400 may include one or more processors 1410 , memory 1430 , user interface input device 1440 , user interface output device 1450 , and storage 1460 that communicate with each other via bus 1420 . can In addition, computer system 1400 may further include a network interface 1470 coupled to network 1480 . The processor 1410 may be a central processing unit or a semiconductor device that executes programs or processing instructions stored in the memory 1430 or the storage 1460 . The memory 1430 and the storage 1460 may be storage media including at least one of a volatile medium, a non-volatile medium, a removable medium, a non-removable medium, a communication medium, and an information delivery medium. For example, the memory 1430 may include a ROM 1431 or a RAM 1432 .

According to the embodiment described above, in the technology for detecting abnormal behavior of a person through CCTV image analysis, it is possible to provide a CCTV image-based abnormal behavior detection method and apparatus capable of quickly and accurately detecting and tracking a person. .

In addition, it is possible to provide an abnormal behavior detection method capable of real-time processing by using a technique that does not require much computation in detecting and tracking an object and determining an event.

In addition, it is possible to provide an abnormal behavior detection method capable of minimizing false alarms while accurately detecting a target event by minimizing interference between events to be detected.

Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those of ordinary skill in the art to which the present invention pertains can implement the present invention in other specific forms without changing its technical spirit or essential features. You will understand that there is Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: CCTV image-based abnormal behavior detection device
110: CCTV video stream
120: human object detection unit
130: tracking information list generation unit
140: event judging unit
150: alarm signal generator
160: Abnormal behavior event alarm signal

Claims (1)

detecting a human object from a CCTV image using a human detection engine;
generating a tracking information list based on the detected tracking information of the human object;
determining an anomalous behavioral event for a person on the tracking information list; and
Including the step of generating an alarm signal for the abnormal behavior event, CCTV image-based abnormal behavior detection method.

Images