KR20220057213A - System for detecting abnormal behavior based on artificial intelligence - Google Patents

Abstract

The present invention relates to an artificial intelligence (AI)-based abnormal behavior detection system. The AI-based abnormal behavior detection system may comprise: a video recording device taking a plurality of videos of abnormal behavior; and a monitoring device. The monitoring device receives a plurality of videos captured by the video recording device, generates an algorithm for determining an abnormal behavior on the basis of a result of learning behaviors related to the abnormal behavior in the past to determine a first situation in which the abnormal behavior is to be occurred and a second situation in which the abnormal behavior is occurred step by step on the basis of the abnormal behavior determination algorithm, generates and outputs first notification data when it is determined that the first situation of the abnormal behavior has occurred, and generates and outputs second notification data when it is determined that the second situation of the abnormal behavior has occurred. According to the present invention, the system uses a deep learning-based abnormal behavior determination algorithm to determine and output the first situation in which the abnormal behavior is to be occurred and the second situation in which the abnormal behavior is occurred step by step, thereby enabling a control manager or supervisor to quickly perform an appropriate response according to the situation that the abnormal behavior is occurred.

Description

AI-based abnormal behavior detection system {SYSTEM FOR DETECTING ABNORMAL BEHAVIOR BASED ON ARTIFICIAL INTELLIGENCE}

The present invention relates to an artificial intelligence-based abnormal behavior detection system, and more specifically, to an artificial intelligence-based abnormal behavior detection system capable of detecting abnormal behavior by analyzing a recorded video recorded from CCTV and notifying it to a monitoring device will be.

Recently, corporal punishment or assault in daycare centers or schools has emerged as a social problem. In order to solve the above problems, commercialization of a video surveillance system using CCTV has been recently centered on public institutions.

Existing video surveillance systems are rapidly developing into a network-based intelligent video surveillance system that can automatically recognize/track a characteristic object of an object or person, away from a simple monitoring of the surrounding situation.

However, it is a reality that it is impossible for a small number of observers to check dozens of cameras as a whole with the naked eye. In order to supplement this part, intelligent monitoring methods by computer such as object and human body detection function, and designated tracking function of a specific person or a specific group have been introduced.

Looking at the prior art, as shown in Korean Patent Registration No. 10-1526499, at least one video camera equipped with a communication interface; an image management server that stores and manages image data captured from the video camera, identifies a target object according to condition designation from a selected image, and analyzes an image to determine a driving axis region of the moving object when estimating the moving object; an intelligent image analysis server that collects the image data of the video camera and dozens of collected data from smart devices existing in the area where the video camera is installed and analyzes the image; and a PoE-supported power expansion device for relaying data communication between the video camera and the control center and supplying power to the video camera through an Ethernet cable.

However, in the surveillance system of the related art, in identifying and analyzing objects in image data, the accuracy for precisely detecting violence even in images in which a large number of people appear, or versatility of enabling detection from images with little movement it falls

The present invention has been devised to solve the above problems, and using an image captured by an image capturing device and an algorithm based on deep learning, artificial intelligence that can monitor and notify whether abnormal behavior occurs with high accuracy It aims to provide a system-based abnormal behavior detection system.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

An artificial intelligence-based abnormal behavior detection system according to an embodiment of the present invention receives a plurality of images photographed by an image photographing device and an image photographing device for photographing a plurality of images for abnormal behavior, and acts related to an existing abnormal behavior The abnormal behavior determination algorithm generated based on the result of learning , a monitoring device for generating and outputting first notification data when it is determined that the first situation of the abnormal behavior has occurred, and generating and outputting the second notification data when it is determined that the second situation of the abnormal behavior has occurred there is. Accordingly, the present invention uses an abnormal behavior determination algorithm based on deep learning to classify the first situation, which is the situation in which the abnormal behavior occurs, and the second situation, which is the situation in which the abnormal behavior occurs, and determines and informs the accuracy in stages. It is possible to monitor and notify whether high abnormal behavior occurs, and a control manager or supervisor can make a quick and appropriate response according to the situation of occurrence of abnormal behavior.

The method for detecting abnormal behavior based on artificial intelligence according to an embodiment of the present invention includes the steps of learning the behavior pattern of the abnormal behavior based on an image that can be determined as the existing abnormal behavior, and to be determined as the abnormal behavior by a supervisor. Determining whether image data including possible behaviors and behaviors that cannot be determined as abnormal behaviors are stored Generating and updating a determination algorithm, receiving a plurality of images photographed by an image capturing device, executing an abnormal behavior determination algorithm, using at least one of a YOLO model and a skeleton model for a plurality of images A step of analyzing, determining whether a first situation of the abnormal behavior has occurred, generating and outputting first notification data when it is determined that the first situation has occurred, determining whether a second situation of the abnormal behavior has occurred, the second When it is determined that the second situation has occurred, generating and outputting second notification data and storing an image from the first situation occurrence time to the second situation end time point may include the steps of generating and outputting second notification data.

Details of other embodiments are included in the detailed description and drawings.

In the present invention, the abnormal behavior determination algorithm based on deep learning includes at least one model of the YOLO model and the skeleton model to determine and monitor whether abnormal behavior occurs, so that it is possible to monitor whether abnormal behavior occurs with higher accuracy.

The present invention can prevent the occurrence of abnormal behavior and prevent abnormal behavior by detecting the first situation, which is a situation in which abnormal behavior such as violence, and the second situation, which is a situation in which abnormal behavior such as violence occurs, and detects them in stages. can respond appropriately.

The present invention distinguishes the first situation, which is a situation in which abnormal behavior may occur, and the second situation, which is a situation in which abnormal behavior occurs, and not only responds appropriately to the situation, but also responds to abnormal behavior quickly by notifying the control manager or supervisor .

According to the present invention, objects can be confidential by the YOLO model and the skeleton model, so that the privacy of the individual can be protected.

The effect according to the present invention is not limited by the contents exemplified above, and more various effects are included in the present invention.

1 is a diagram schematically showing the configuration of an artificial intelligence-based abnormal behavior detection system according to an embodiment of the present invention.
2 is a diagram schematically showing a detailed configuration of an image capturing apparatus of an artificial intelligence-based abnormal behavior detection system according to an embodiment of the present invention.
3 is a diagram schematically showing the configuration of a monitoring device of an artificial intelligence-based abnormal behavior detection system according to an embodiment of the present invention.
4A and 4B are diagrams illustrating an example of notifying a danger alert of an artificial intelligence-based abnormal behavior detection system according to an embodiment of the present invention.
5 is a flowchart illustrating an artificial intelligence-based abnormal behavior detection method according to an embodiment of the present invention.

Advantages and features of the present invention, and methods for achieving them, will become apparent with reference to the detailed description in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments presented below, it can be implemented in various different forms, and it is understood to include all transformations, equivalents and substitutes included in the spirit and scope of the present invention. should be The embodiments presented below are provided to complete the disclosure of the present invention, and to fully inform those of ordinary skill in the art to the scope of the invention. In describing the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

Various embodiments of the present invention may be implemented as software (eg, a program) including instructions stored in a machine-readable storage medium (eg, a computer). The device is a device capable of calling a stored command from a storage medium and operating according to the called command, and may include an electronic device (eg, a server) according to the disclosed embodiments. Instructions may include code generated or executed by a compiler or interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not include a signal and is tangible, and does not distinguish that data is semi-permanently or temporarily stored in the storage medium.

According to an example, the method according to various embodiments disclosed in the present invention may be provided as included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product may be distributed in the form of a device-readable storage medium (eg, compact disc read only memory (CD-ROM)) or online through an application store (eg, Play Store™). In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated in a storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

Each of the components (eg, a module or a program) according to various embodiments may be composed of a singular or a plurality of entities, and some sub-components of the aforementioned sub-components may be omitted, or other sub-components may be It may be further included in various embodiments. Alternatively or additionally, some components (eg, a module or a program) may be integrated into a single entity to perform the same or similar functions performed by each corresponding component prior to integration. According to various embodiments, operations performed by a module, program, or other component are sequentially, parallel, repetitively or heuristically executed, or at least some operations are executed in a different order, are omitted, or other operations are added. can be

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram schematically showing the configuration of an artificial intelligence-based abnormal behavior detection system according to an embodiment of the present invention.

1, the artificial intelligence-based abnormal behavior detection system 1000 according to an embodiment of the present invention is a plurality of closed circuit television (Closed Circuit Television: CCTV, hereinafter referred to as 'CCTV., 100-1, 100-2, ..., 100-N) and the monitoring device 200 may be included.

A plurality of CCTVs (100-1, 100-2, ..., 100-N) may record a control target through a camera in real time, and store the captured image for later viewing. In FIG. 1, a plurality of CCTVs (100-1, 100-2, ..., 100-N) have been described as a device for photographing a control target, but the present invention is not limited thereto. 100), for example, it may be a digital video recorder (DVR). In this way, the images captured in real time by the plurality of CCTVs 100-1, 100-2, ..., 100-N are transmitted to the monitoring device 200 .

The monitoring device 200 receives the images captured by a plurality of CCTVs (100-1, 100-2, ..., 100-N) and executes an abnormal behavior determination algorithm to notify the supervisor when it is determined that abnormal behavior has occurred there is. The abnormal behavior determination algorithm may be an algorithm that is created and updated based on the result of learning behavior related to the abnormal behavior. In this case, the abnormal behavior includes acts such as fighting, assault, bullying, bullying, and falling.

The monitoring device 200 analyzes the images taken by the plurality of CCTVs 100-1, 100-2, ..., 100-N, and divides the case determined as the first situation and the case determined as the second situation, Another way is to notify the watcher. Here, the first situation is a case in which a situation in which an abnormal behavior may occur is detected, that is, a situation before the occurrence of the abnormal behavior is detected, and the second situation is a case in which a situation in which the abnormal behavior occurs is detected. The monitoring device 200 distinguishes the case of the first situation and the case of the second situation through the display or the audio output unit of the monitoring device 200 when the first situation or the second situation is detected by the supervisor and displays them in different colors It can be notified as much as possible, or a warning sound can be generated through a speaker or the like.

The abnormal behavior determination algorithm may include at least one of a You Only Look Once (YOLO) model and a Skeleton model. Here, the YOLO model and the skeleton model analyze the images captured by a plurality of CCTVs (100-1, 100-2, ..., 100-N) to extract features of pre-learned behaviors as abnormal behaviors. In addition, the YOLO model and the skeleton model can analyze abnormal behavior by removing specific information that can identify an individual. Here, the removal of specific information refers to confidentiality of the information of the person in the video, for example, mosaic processing of the person in the video. Accordingly, the artificial intelligence-based abnormal behavior detection system 1000 according to an embodiment of the present invention performs an abnormal behavior determination algorithm including at least one of a YOLO model and a skeleton model when detecting the first situation and the second situation. It can be used to determine abnormal behavior, so it is possible to protect personal privacy.

2 is a diagram schematically showing a detailed configuration of an image capturing apparatus of an artificial intelligence-based abnormal behavior detection system according to an embodiment of the present invention.

Referring to FIG. 2 , an image capturing apparatus such as a plurality of CCTVs 100-1, 100-2, ..., 100-N of the artificial intelligence-based abnormal behavior detection system 1000 according to an embodiment of the present invention ( Each of 100) may include a camera 110 , an image transmitter 120 , an image recorder 130 , and a controller 140 .

The camera 110 generates a plurality of images by photographing a space in which it is necessary to monitor an abnormal behavior of a person. In this case, the images generated by the camera 110 may be still images and moving images.

The image transmitter 120 includes an interface circuit for communication with the monitoring device 200 . More specifically, the image transmission unit 120 transmits a plurality of images generated by the camera 110 to the monitoring device 200 .

The image recording unit 130 records still images and moving images captured by the camera 110 . The image recorder 130 may include a volatile memory and/or a non-volatile memory. In order to implement and/or provide the operation and function of the camera 110 , the image recorder 130 may store commands or data related to components, one or more programs and/or software, an operating system, and the like together.

The controller 140 controls the overall operation of the image capturing apparatus 100 . More specifically, the controller 140 transmits the images captured by the image capturing apparatus 100 to the monitoring apparatus 200 through the image transmitting unit 120 in real time, and It is possible to control the images to be recorded and stored in the image recorder 130 .

3 is a diagram schematically showing the configuration of a monitoring device for an artificial intelligence-based abnormal behavior detection system according to an embodiment of the present invention, and FIGS. 4A and 4B are an artificial intelligence-based abnormality according to an embodiment of the present invention. It is a diagram showing an example of notifying a danger alert of a behavior detection system.

Referring to FIG. 3 , the monitoring apparatus 200 of the artificial intelligence-based abnormal behavior detection system 1000 according to an embodiment of the present invention includes an image receiving unit 210 , a database unit 220 , and an image analyzing unit 230 . , an image determination unit 240 , a notification data output unit 250 , and a control unit 260 may be included.

The image receiver 210 includes an interface circuit for communication with the image photographing apparatus 100 . For example, the image receiver 210 may receive a plurality of images captured by the image photographing apparatus 100 .

The database unit 220 may store a plurality of images received through the image receiving unit 210 and an algorithm capable of detecting an abnormal behavior. Here, the algorithm capable of detecting the abnormal behavior may be a program learned based on the previously occurring abnormal behavior image and data. The database unit 220 may include a volatile memory and/or a non-volatile memory. The database unit 220 may store commands or data related to components, one or more programs and/or software, an operating system, etc. in order to implement and/or provide the operations and functions provided by the monitoring device 200 . .

In the database unit 220, after analyzing the image using the YOLO model or the skeleton model based on the plurality of images received through the image receiving unit 210, the first situation and the abnormal behavior, which are situations in which abnormal behavior may occur, are An abnormal behavior determination algorithm capable of determining step by step the second situation, which is the generated situation, may be stored. The abnormal behavior determination algorithm may determine that the second situation has occurred when the first situation continues for a preset time. The abnormal behavior determination algorithm may be an algorithm generated by learning based on an image before the occurrence of the abnormal behavior and an image in which the abnormal behavior occurred.

The abnormal behavior determination algorithm stored in the database 220 may include at least one of a YOLO model and a skeleton model capable of detecting abnormal behavior. More specifically, the YOLO model and the skeleton model are models capable of detecting an object from a plurality of images received through the image receiver 210 . Here, the object may be a person or an action of a person in the image. More specifically, the YOLO model can label objects in one image and label only necessary objects, thereby reducing learning of unnecessary elements. Here, labeling is a technology that can detect abnormal behavior by classifying the characteristic behavior of a person in an image. The skeleton model is also a model that skeletonizes the shape information of the person in the image, and it is possible to reduce the learning of unnecessary elements. As described above, the artificial intelligence-based abnormal behavior detection system according to an embodiment of the present invention uses an abnormal behavior determination algorithm including at least one of a YOLO model and a skeleton model. By detecting abnormal behavior, more clear and precise abnormal behavior detection can be made.

Abnormal behavior determination algorithm is a situation in which abnormal behavior can occur, and pre-learned behaviors, for example, a posture to prepare for a fight, an arm-raising behavior, an arm-stretching act, an arm-stretching act, an act of raising a leg before kicking, an act of holding a tool and swinging Behaviors before a fight, such as trying to wrap, behaviors before falling, such as tilting or losing balance By comparing the stored data with the image received by the image receiving unit 210, it is possible to determine whether an abnormal behavior has occurred.

When it is determined that an abnormal behavior has occurred by the abnormal behavior determination algorithm, the database unit 220 includes a plurality of CCTVs (100-1, 100-2, ..., 100-N) from the occurrence of the abnormal behavior to the end of the abnormal behavior. It is possible to save the video of the corresponding CCTV filmed. That is, the image from the start of the first situation of the abnormal behavior to the end of the second situation of the abnormal behavior may be stored.

The database unit 220 may store an image that can be regarded as an abnormal behavior by the judgment of a supervisor even if it is not a pre-stored behavior that can be determined as the first situation and the second situation of the abnormal behavior. The stored image is regarded as update data, and when the abnormal behavior determination algorithm is updated later, the action of the stored image may be stored as the abnormal behavior behavior.

The database unit 220 is an image determined as the first situation and the second situation of the abnormal behavior by the abnormal behavior determination algorithm, or when it is determined that the abnormal behavior is not abnormal by the monitor, for example, the act of moving with arms crossed; It is possible to store the action image of the intimacy display, such as the action of moving with shoulder-to-shoulder movement. The stored image is also regarded as update data, so that when the abnormal behavior determination algorithm is updated later, the stored behavior is stored as not an abnormal behavior, so that the abnormal behavior can be determined more clearly.

The database unit 220 may store the first notification data generated when the first situation of the abnormal behavior occurs and the second notification data generated when the second situation of the abnormal behavior occurs.

The image analyzer 230 may analyze the image received by the image receiver 210 using a YOLO model or a skeleton model. More specifically, in the case of an algorithm including the YOLO model, multiple people in the image received by the image receiving unit 210 may be labeled, and in the case of an algorithm including a skeleton model, people in the image received by the image receiving unit 210 can be framed.

The image determination unit 240 may determine whether an abnormal behavior occurs in the image by using an abnormal behavior determination algorithm based on the image analyzed by the image analysis unit 230 . The image determination unit 240 may distinguish and determine the first situation in which the abnormal behavior may occur and the second situation in which the abnormal behavior occurs.

The first situation in which the abnormal behavior may occur may be determined based on previously learned contents. For example, the image determination unit 240 determines that the action of the person in the video includes a posture to prepare for a fight, an action to raise an arm, an action to stretch an arm, an action to raise a leg before kicking, an action to hold a tool and swing it. If the behavior before the fight or the behavior before the fall, such as the movement of leaning or losing balance, is included, it can be judged as the first situation in which abnormal behavior can occur.

The second situation in which the abnormal behavior occurs may be determined based on previously learned contents. For example, when physical contact occurs, that is, when a person in the image falls down by extending or swinging a tool, arms, or legs, the image determination unit 240 determines the second abnormal behavior occurs. situation can be judged.

The image determination unit 240 may determine that the second situation has occurred when the first situation continues for a preset time after the first situation of the abnormal behavior occurs. Accordingly, the image determination unit 240 determines that even if the second situation of the abnormal behavior does not occur, if the first situation of the abnormal behavior continues for a specific time, the second situation of the abnormal behavior is highly likely to occur, so that the second situation occurs The occurrence of abnormal behavior such as assault can be prevented by judging it in advance and notifying the supervisor.

The notification data output unit 250 may generate and output notification data for notifying a supervisor when an action in the video is determined to be an abnormal action as a result of the determination by the video determination unit 240 . The notification data output unit 250 may include a display unit 251 and an audio output unit 252 . Accordingly, the notification data output unit 250 may visually or audibly notify the supervisor of whether abnormal behavior is detected or occurred.

As a result of image determination by the image determination unit 240, the notification data output unit 250 determines that the first situation in which an abnormal behavior may occur is determined as the second situation in which the abnormal behavior occurs. Another way is to notify the watcher. For example, as in FIGS. 4A and 4B, if a plurality of CCTVs 100-1, 100-2, ..., 100-N are displayed to the monitor through the display unit 251, the situation in which abnormal behavior occurs If it is determined as the first situation, it may be displayed as shown in FIG. 4A, and if it is determined as the second situation, which is a situation in which an abnormal behavior occurs, it may be displayed as shown in FIG. 4B. That is, the notification data output unit 250 may display different colors in the case of the first situation and the second situation through the display unit 251 to notify the supervisor. In addition, when it is determined that the first situation and the second situation are determined, the notification data output unit 250 may notify the supervisor with different warning sounds through the voice output unit 252 . Accordingly, the artificial intelligence-based abnormal behavior detection system 1000 according to an embodiment of the present invention is determined as a first situation, which is a situation in which abnormal behavior may occur, and a second situation, which is a situation in which abnormal behavior occurs. In this case, it is possible to improve the cognitive convenience of the monitor by notifying the supervisor visually and/or audibly.

The controller 260 may control the overall operation of the monitoring device 200 . The controller 260 may receive a plurality of images received from the image capturing apparatus 100 , and generate and update an abnormal behavior determination algorithm, which is a deep learning-based algorithm, for all of the plurality of images received. The control unit 260 may include an algorithm operation control unit 261 , an image storage control unit 262 , and an abnormal behavior occurrence notification data generation control unit 263 .

The algorithm operation control unit 261 may control the abnormal behavior determination algorithm stored in the database unit 220 to be executed when the image transmitted from the image photographing apparatus 100 is received by the image receiving unit 210 .

The image storage control unit 262 may control the first situation and the second situation of the abnormal behavior to be performed in stages, so that the CCTV image is automatically stored in the database unit 220 from the time the first situation occurs. . In addition, in addition to pre-learned behavior that can be judged as abnormal behavior, the real-time behavior that is included in the abnormal behavior by the observer by the judgment of the observer, or an activity that has been previously learned as abnormal behavior but is not included in the abnormal behavior by the judgment of the observer Images can be saved automatically. In this way, the video that was previously learned as an abnormal behavior or included in the abnormal behavior by the monitor, but determined to be not included in the abnormal behavior by the monitor, can be stored as update data that can be updated later when the abnormal behavior determination algorithm is updated. .

The abnormal behavior occurrence notification data generation control unit 263 generates the first notification data when it is determined that the image action is the first situation of the abnormal behavior as a result of the judgment of the image determination unit 240, and when it is determined that the second situation has occurred The second notification data may be generated and notified to the supervisor. Here, the first notification data and the second notification data may be display data visually displayed by the display unit 251 and alarm sound data that can be audibly notified by the audio output unit 252 . Here, the first notification data and the second notification data may be different data. That is, the abnormal behavior occurrence notification data generation control unit 263 controls to be displayed externally in different colors or messages depending on whether the abnormal behavior is a first situation, a situation in which the abnormal behavior occurs, or a second situation, a situation in which the abnormal behavior occurs. and can be controlled so that different alarm sounds are generated.

As such, the artificial intelligence-based abnormal behavior detection system 1000 according to an embodiment of the present invention includes actions such as assault, fight, and harassment by using a deep learning-based abnormal behavior determination algorithm in the monitoring device 200 . The first situation, which is the situation before the occurrence of the abnormal behavior, and the second situation, which is the situation where the abnormal behavior occurs, are determined, respectively, and the response to the abnormal behavior is appropriately determined by notifying the controller or supervisor who controls the monitoring device 200 . can be done More specifically, when the monitor responds to the first situation of the abnormal behavior, it is before the occurrence of the abnormal behavior, so the occurrence of the abnormal behavior can be prevented. You can get the case resolved.

5 is a flowchart illustrating an artificial intelligence-based abnormal behavior detection method according to an embodiment of the present invention. The operations of FIG. 5 may be performed by the monitoring device 200 .

Referring to FIG. 5 , the monitoring device 200 may first learn a behavior pattern before and after the occurrence of the abnormal behavior based on the abnormal behavior occurrence image ( S501 ). For example, the monitoring device 200 can learn the first situation before the occurrence of the abnormal behavior, that is, the first situation in which the abnormal behavior will occur, through the act of raising the arm, the act of falling, the act of raising a leg, the act of raising a tool, etc. , the second situation can be learned through actions with physical contact.

Thereafter, it is determined whether there is data that can be determined as an abnormal behavior or data that can be determined as an abnormal behavior but not an abnormal behavior by the arbitrary judgment of the observer rather than the learned pattern (S502).

Thereafter, if it is determined that there is data input by the supervisor as a result of the determination, the monitoring apparatus 200 may generate and update an abnormal behavior determination algorithm based on the learning result and data input by the supervisor (S503). On the other hand, if it is determined that there is no data input by the monitor as a result of the determination, an abnormal behavior determination algorithm may be generated based on only the learning result (S504). The generated and updated abnormal behavior determination algorithm may be controlled to be stored in the database unit 220 .

Thereafter, the monitoring device 200 may receive a plurality of images captured by a plurality of CCTVs 100-1, 100-2, ..., 100-N (S505).

Thereafter, the monitoring device 200 executes the abnormal behavior determination algorithm (S506) and analyzes the received plurality of images using the YOLO model or the skeleton model (S507), then executes the abnormal behavior determination algorithm to cause abnormal behavior It is determined whether a first situation, which is a situation, has occurred (S508).

As a result of the determination, when it is determined that the first situation is the situation in which the abnormal behavior will occur, the first notification data is generated and output by the display unit 251 and the voice output unit 252 (S509). On the other hand, if it is not determined as the first situation, which is the situation in which the abnormal behavior will occur, as a result of the determination, it is determined by the supervisor as the abnormal behavior and it is determined whether there is input data (S502).

Thereafter, the monitoring device 200 determines whether a second situation of the abnormal behavior occurs after determining whether a first situation that is a situation in which the abnormal behavior occurs through the abnormal behavior determination algorithm (S510).

As a result of the determination, when it is determined that the second situation is a situation in which the abnormal behavior occurs, second notification data is generated and output by the display unit 251 and the voice output unit 252 (S511). On the other hand, if it is not determined as the second situation, which is the situation in which the abnormal behavior occurred, as a result of the determination, it is determined whether the first situation of the abnormal behavior has continued for a preset time (S512), and the first situation of the abnormal behavior has continued for the preset time If it is determined that the second condition of the abnormal behavior has occurred, the second notification data is generated and output by the display unit 251 and the voice output unit 252 (S511). In addition, if it is determined that the first situation of the abnormal behavior does not last for a preset time as a result of the determination, it is determined by the supervisor as the abnormal behavior and it is determined whether there is input data (S502). In this case, the second notification data may include display data that may be displayed differently from the first notification data and alarm sound data having a different alarm sound.

After that, the monitoring device 200 automatically stores the image from the first situation of the abnormal behavior to the end of the second situation at the time when the occurrence of the abnormal behavior is determined (S513), whether the abnormal behavior occurs with respect to one image The detection and judgment process for .

The artificial intelligence-based abnormal behavior detection system according to various embodiments of the present invention may be described as follows.

An artificial intelligence-based abnormal behavior detection system according to an embodiment of the present invention receives a plurality of images captured by an image photographing apparatus and an image photographing apparatus for photographing a plurality of images of an abnormal behavior, and is related to the existing abnormal behavior After generating an abnormal behavior determination algorithm generated based on the result of learning the behavior, the first situation, which is the situation in which the abnormal behavior occurs, and the second situation, which is the situation in which the abnormal behavior occurs, are determined step by step based on the abnormal behavior determination algorithm. , a monitoring device for generating and outputting first notification data when it is determined that the first situation of the abnormal behavior has occurred, and generating and outputting the second notification data when it is determined that the second situation of the abnormal behavior has occurred there is.

According to another feature of the present invention, the abnormal behavior determination algorithm may include at least one of a YOLO model and a skeleton model.

According to another feature of the present invention, the monitoring device stores an image receiving unit for receiving a plurality of images captured by the image capturing device, an abnormal behavior determination algorithm, and when it is determined that the first situation and the second situation of the abnormal behavior have occurred A database unit that stores images from the occurrence of the first situation to the end of the second situation of the abnormal behavior, and an image analysis unit that analyzes the image received through the image receiving unit using at least one of the YOLO model and the skeleton model , the image judgment unit that determines step by step whether the first situation and the second situation of the abnormal behavior occur on the basis of the abnormal behavior determination algorithm based on the image received through the image receiving unit, and the first situation of the abnormal behavior is determined by the image judgment unit It may include a notification data output unit for outputting the first notification data when the occurrence and second notification data when the second abnormal behavior occurs, and a control unit for controlling the overall operation of the monitoring device.

According to another feature of the present invention, the control unit includes an algorithm operation control unit for controlling an abnormal behavior determination algorithm to be executed when an image is received by the image receiving unit, an abnormal behavior determination algorithm, and a second situation from the time the first situation of the abnormal behavior occurs. An image storage control unit that controls so that images that can be included in the abnormal behavior among the images received in the image receiving unit are stored according to the video up to the end point and the judgment of the monitor, and the image received by the image receiving unit as a result of the judgment of the image determining unit is the image of the abnormal behavior When it is determined as the first situation, it generates first notification data, and when it is determined as the second situation of the abnormal behavior, the abnormal behavior occurrence notification data generation control unit generates second notification data.

According to another feature of the present invention, the first notification data and the second notification data may include different display data and alarm sound data.

According to another feature of the present invention, when the first situation of the abnormal behavior continues for a preset time, the monitoring device may determine that the second situation of the abnormal behavior has occurred, and may generate and output the second notification data.

The method for detecting abnormal behavior based on artificial intelligence according to an embodiment of the present invention includes the steps of learning the behavior pattern of the abnormal behavior based on an image that can be determined as the existing abnormal behavior, and to be determined as the abnormal behavior by a supervisor. Determining whether image data including possible behaviors and behaviors that cannot be determined as abnormal behaviors are stored Generating and updating a determination algorithm, receiving a plurality of images photographed by an image capturing device, executing an abnormal behavior determination algorithm, using at least one of a YOLO model and a skeleton model for a plurality of images A step of analyzing, determining whether a first situation of the abnormal behavior has occurred, generating and outputting first notification data when it is determined that the first situation has occurred, determining whether a second situation of the abnormal behavior has occurred, the second When it is determined that the second situation has occurred, generating and outputting second notification data and storing an image from the first situation occurrence time to the second situation end time point may include the steps of generating and outputting second notification data.

According to another feature of the present invention, the method may further include the step of determining whether the first situation of the abnormal behavior continues for a preset time after the step of determining whether the second situation of the abnormal behavior has occurred.

Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and various modifications may be made within the scope without departing from the technical spirit of the present invention. . Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

1000: AI-based abnormal behavior detection system
100: video recording device
110: camera
120: video transmission unit
130: video recording unit
140: controller
200: monitoring device
210: video receiver
220: database unit
230: image analysis unit
240: image judgment unit
250: notification data output unit
251: display unit
252: audio output unit
260: control unit
261: Algorithm operation control unit
262: video storage control unit
263: Abnormal behavior occurrence notification data generation control unit

Claims (8)

an image photographing device for photographing a plurality of images for abnormal behavior; and
Receive a plurality of images captured by the image photographing device, generate an abnormal behavior determination algorithm generated based on a result of previously learning behavior related to abnormal behavior, and generate abnormal behavior based on the abnormal behavior determination algorithm. After step-by-step determination of a first situation that is a situation and a second situation that is a situation in which the abnormal behavior occurs, if it is determined that the first situation of the abnormal behavior has occurred, first notification data is generated and output, and An artificial intelligence-based abnormal behavior detection system comprising a; when it is determined that the second situation has occurred, a monitoring device for generating and outputting second notification data. According to claim 1,
The abnormal behavior determination algorithm is an artificial intelligence-based abnormal behavior detection system including at least one of a YOLO model and a skeleton model. According to claim 2, wherein the monitoring device,
an image receiving unit for receiving a plurality of images captured by the image capturing apparatus;
A database unit for storing the abnormal behavior determination algorithm, and storing images from the occurrence time of the first situation of the abnormal behavior to the end of the second situation when it is determined that the first situation and the second situation of the abnormal behavior occur ;
an image analysis unit analyzing the image received through the image receiving unit using at least one of the YOLO model and the skeleton model;
an image determination unit for step-by-step determining whether the first situation and the second situation of the abnormal behavior occur on the image received through the image receiving unit based on the abnormal behavior determination algorithm;
a notification data output unit configured to output the first notification data when the first situation of the abnormal behavior occurs as a result of the determination of the image determination unit and the second notification data when the second situation of the abnormal behavior occurs; and
Artificial intelligence-based abnormal behavior detection system comprising a; a control unit for controlling the overall operation of the monitoring device. According to claim 3, wherein the control unit,
an algorithm operation control unit for controlling the abnormal behavior determination algorithm to be executed when an image is received by the image receiving unit;
The abnormal behavior determination algorithm, the image from the time when the first situation of the abnormal behavior occurs to the end of the second situation, and the image that can be included in the abnormal behavior among the images received by the image receiver according to the judgment of the supervisor an image storage control unit controlling to be stored; and
As a result of the determination of the image determination unit, if the image received by the image receiving unit is determined to be the first situation of the abnormal behavior, the first notification data is generated, and if it is determined as the second situation of the abnormal behavior, the second notification data is generated An artificial intelligence-based abnormal behavior detection system comprising a; 5. The method of claim 4,
The first notification data and the second notification data are artificial intelligence-based abnormal behavior detection system including different display data and alarm sound data. According to claim 1,
When the first situation of the abnormal behavior continues for a preset time, the monitoring device determines that the second situation of the abnormal behavior has occurred, and generates and outputs the second notification data. learning the behavior pattern of the abnormal behavior based on an image that can be determined as the existing abnormal behavior;
determining whether image data including an action that can be determined as the abnormal behavior and an activity that cannot be determined as the abnormal behavior by a supervisor is stored;
generating and updating an abnormal behavior determination algorithm based on the stored image data and the learning result when it is determined that the image data input by the supervisor is stored;
receiving a plurality of images captured by an image capturing apparatus;
executing the abnormal behavior determination algorithm;
analyzing the plurality of images using at least one of a YOLO model and a skeleton model;
determining whether a first situation of the abnormal behavior has occurred;
generating and outputting first notification data when it is determined that the first situation has occurred;
determining whether a second situation of the abnormal behavior has occurred;
generating and outputting second notification data when it is determined that the second situation has occurred; and
Storing an image from the first situation occurrence time to the second situation termination time; AI-based abnormal behavior detection method comprising a. 8. The method of claim 7,
After the step of determining whether the second situation of the abnormal behavior has occurred,
An artificial intelligence-based abnormal behavior detection method further comprising the step of determining whether the first situation of the abnormal behavior continues for a preset time.

Images