KR20230112038A - A device that detects specific patterns in traffic conditions using artificial neural network - Google Patents

Abstract

According to an embodiment of the present invention, an apparatus for detecting a traffic condition specific pattern using an artificial neural network is disclosed. The peculiar pattern detection apparatus includes: an object identification unit configured to identify an object from a plurality of area images photographed using a camera, and to generate a plurality of object information about the object corresponding to each of the area images in which the object is identified among the area images; an object pattern management unit generating an object pattern for the object based on the object information; A pattern model management unit that classifies the plurality of object patterns generated during a preset period by preset time intervals and preset days of the week to create a plurality of object pattern groups, and creates or learns a pattern model corresponding to each of the object pattern groups; An object pattern monitoring unit that searches a database for a plurality of the pattern models corresponding to the current time and the current day of the week, creates a reference pattern model based on the searched pattern models, and compares the reference pattern model with a plurality of monitored object patterns, which are the object patterns corresponding to the current time, to determine whether there is an abnormality; and a video monitoring unit for displaying the surveillance object patterns and the region images including the surveillance object patterns when an abnormal state does not occur, and displaying the surveillance object patterns and an abnormal pattern notification when an abnormal state occurs. In addition, each of the object information includes the object type, object location, and identification time of the object, and the object pattern management unit determines an object path, object direction, and object speed for the object based on the object location and identification time of the object information, and generates the object pattern including the object type, the object path, the object direction, and the object speed, and the pattern model management unit first selects object patterns having the same object type and object direction among the object patterns included in the object pattern group. determine as object patterns, determine primary object patterns overlapping at least a part of the object path among the primary object patterns as secondary object patterns, generate a pattern area corresponding to the secondary object patterns based on the object path of the secondary object pattern, and the pattern model management unit, among the plurality of pattern models already created, the secondary object patterns and the object type and the object direction are the same, and the secondary object patterns and the pattern area overlap at least partially, The generated pattern model is searched in the database, and if the already created pattern model is found, the searched already created pattern model is trained using the secondary object patterns, and if the already created pattern model is not found, the pattern model is created using the secondary object patterns, the object pattern monitoring unit determines, among the searched pattern models, the pattern models having the same object type and the same object direction and overlapping at least a part of the pattern area as first pattern models, and determining each of the first pattern models as first pattern models. A sensitivity condition is determined based on secondary object patterns, first pattern models for which the secondary object patterns satisfy the sensitivity condition among the first pattern models are determined as second pattern models, reference pattern models are created by combining the second pattern models, the presence or absence of the abnormality is determined by comparing the monitored object patterns with the reference pattern model, the pattern model management unit generates learning data in which a type of abnormal situation is labeled in the monitored object pattern, and inputs the monitored object pattern using the learning data. Machine learning is performed on the anomaly detection pattern detection model that outputs the type and probability of anomaly when input as a value.

Description

A device that detects a specific pattern for traffic conditions using an artificial neural network

The present invention relates to an apparatus for detecting a specific pattern for a traffic situation using an artificial neural network.

A CCTV is a camera that transmits an image to a specific receiver, and a number of CCTVs are operated for various purposes. In particular, CCTV is frequently used for traffic control.

Conventionally, a method of detecting the occurrence of an abnormal situation such as an emergency patient or a traffic accident by visually identifying an image transmitted by a CCTV is mainly used by a controller who controls traffic conditions.

However, in this method, there may be a case where the occurrence of an abnormal situation cannot be detected due to the carelessness of the controller, and the cost required for control increases as the time to be controlled increases or the size of the area to be controlled increases. A problem is occurring.

In addition, conventional intelligent or AI CCTV monitoring technologies set surveillance areas according to operational goals and prepared settings and monitoring means suitable for each purpose, such as preset object types, situations, and pattern recognition, but for this, each monitoring means must be prepared. As a result, the problem of increasing cost occurred, and operating under conditions set by a single goal, there was a problem that it could not respond to environmental changes in CCTV images that change in real time.

In addition, conventional intelligent or AI CCTV monitoring technologies are based on pattern learning for specific situations by analyzing the movement of objects, which causes a problem in which only preset or pre-learned situations can be recognized. Furthermore, a fixed setting according to a specific time zone without considering the monitoring site conditions that change every holiday, day and night time zone has a problem in that it is difficult to respond to changes in the environment of the site.

An embodiment of the present invention provides an apparatus for detecting a specific pattern for a traffic situation using an artificial neural network.

In addition, the peculiar pattern detecting apparatus includes an object identification unit configured to identify an object from a plurality of area images photographed using a camera, and to generate a plurality of object information about the object corresponding to each of the area images in which the object is identified among the area images; an object pattern management unit generating an object pattern for the object based on the object information; A pattern model management unit that classifies the plurality of object patterns generated during a preset period by preset time intervals and preset days of the week to create a plurality of object pattern groups, and creates or learns a pattern model corresponding to each of the object pattern groups; An object pattern monitoring unit that searches a database for a plurality of the pattern models corresponding to the current time and the current day of the week, creates a basic pattern model based on the searched pattern models, and compares the basic pattern model with a plurality of monitored object patterns, which are the object patterns corresponding to the current time, to determine whether there is an abnormality; and an image monitoring unit for displaying the surveillance object patterns and the area images including the surveillance object patterns when an abnormal state does not occur, and displaying the surveillance object patterns, the surveillance object patterns, and an abnormal pattern notification when an abnormal state occurs.

In addition, each of the object information may include an object type, an object location, and an identification time of the object.

In addition, the object pattern management unit may determine an object path, an object direction, and an object speed for the object based on the object location and the identification time of the object information, and generate the object pattern including the object type, the object path, the object direction, and the object speed.

In addition, the pattern model management unit may determine, among the object patterns included in the object pattern group, object patterns having the same object type and the same object direction as primary object patterns, determine primary object patterns overlapping at least a part of the object path among the primary object patterns as secondary object patterns, and create a pattern area corresponding to each of the secondary object patterns based on the object path of the secondary object pattern.

In addition, the pattern model management unit searches the database for already created pattern models, including pattern areas in which the object type and object direction of the secondary object patterns are the same as the object type and the object direction, and at least partially overlaps with the pattern areas of the secondary object patterns, among the plurality of already created pattern models, and when the already created pattern models are retrieved, the searched already created pattern models are trained using the secondary object patterns, and the already generated pattern models are not found. , The pattern model may be generated using the secondary object patterns.

In addition, the object pattern monitoring unit determines, among the searched pattern models, pattern models having the same object type and the same object direction and overlapping at least a part of the pattern area as first pattern models, determining a sensitivity condition based on the secondary object patterns of each of the first pattern models, determining first pattern models for which the second object patterns among the first pattern models satisfy the sensitivity condition as second pattern models, and generating reference pattern models by combining the second pattern models, It is possible to determine the presence or absence of the abnormality by comparing the model with the reference pattern model.

In addition, the pattern model management unit generates learning data in which the type of anomaly situation is labeled in the monitoring object pattern, and when the monitoring object pattern is input as an input value, the type and probability of the abnormal situation are generated using the learning data. It can perform machine learning on an anomaly detection pattern detection model.

According to an embodiment of the present invention, a traffic situation pattern is automatically generated without the need for a traffic controller to control the traffic situation, and a specific pattern that deviate from the generated pattern is detected. Thus, the cost required for traffic control is reduced, and the efficiency and accuracy of traffic control can be improved.

According to an embodiment of the present invention, since the type and probability of an abnormal situation corresponding to the generated peculiar pattern is calculated without the need for traffic control personnel to control the traffic situation, the cost required for traffic control is reduced and a quick and accurate response to the unusual pattern can be made.

Effects obtainable from the embodiments are not limited to the effects mentioned above, and other effects not mentioned can be clearly derived and understood by those skilled in the art based on the detailed description below.

BRIEF DESCRIPTION OF THE DRAWINGS Included as part of the detailed description to aid understanding of the embodiments, the accompanying drawings provide various embodiments and, together with the detailed description, describe technical features of the various embodiments.
1 is a schematic diagram showing a specific pattern detection system according to an embodiment of the present invention.
2 is a block diagram showing the configuration of a peculiar pattern detection device according to an embodiment of the present invention.
FIG. 3 is a diagram illustrating a process of generating an object pattern by the object pattern management unit according to FIG. 2 by way of example.
4 is a flowchart illustrating a process of generating a pattern model by the pattern model management unit according to FIG. 2 .
FIG. 5 is a flowchart illustrating a process of determining whether or not there is an abnormality based on object patterns included in a region image by the object pattern monitoring unit according to FIG. 2 .
FIG. 6 is a flowchart illustrating a process of monitoring an area image by the image monitoring unit according to FIG. 2 .
FIG. 7 is a flowchart illustrating a process in which the pattern model management unit 104 according to FIG. 2 performs machine learning on an abnormal pattern detection model.
FIG. 8 is a flowchart illustrating a process in which the object pattern monitoring unit 105 according to FIG. 2 identifies the type of abnormal pattern.
FIG. 9 is a block diagram showing the configuration of the singular pattern detection device according to FIG. 2 .

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. Like reference numerals have been used for like elements throughout the description of each figure.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

It should be understood that when an element is referred to as being “connected” or “connected” to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "comprise" or "have" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, but it should be understood that the presence or addition of one or more other features or numbers, steps, operations, components, parts, or combinations thereof is not excluded in advance.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and are not interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic diagram showing a system for detecting a specific pattern according to an embodiment of the present invention.

Referring to FIG. 1 , a singular pattern detection system according to an embodiment of the present invention includes a singular pattern detection device 100 and a camera 200 . The singular pattern detection device 100 may be connected to the plurality of cameras 200 in information communication.

The camera 200 is configured to transmit the captured region image to the device 100 for detecting a peculiar pattern. In one embodiment, the camera 200 may have 1 or more frames per second. That is, the camera 200 may capture at least one region image per second of the region to be photographed and transmit the image to the peculiar pattern detection device 100 . The area image may include at least one area image per second.

Also, the peculiar pattern detecting apparatus 100 may receive an area image photographed by the camera 200 and detect an object included in the area image when a predetermined object is included in the received area image. In one embodiment, the object may be an image corresponding to a moving object such as a person, a four-wheeled vehicle (passenger car, bus, etc.), a two-wheeled vehicle, and the like.

Also, the peculiar pattern detection apparatus 100 may calculate and store object information corresponding to the detected object in a database. In one embodiment, the object information may include the type, size, color, location of the object in the area image, identification time, object ID, object area, and the like of the object. In one embodiment, the object location may be defined by coordinates using the horizontal axis and the vertical axis of the rectangular area image as coordinate systems. Units of the horizontal axis and the vertical axis may be pixels, and the object location may be defined as the location of the center point of the object. In this embodiment, the center point of the object may be the center point of the object area, which is a rectangular area including the object. In one embodiment, the center point of the object may be the center point of the object image corresponding to the object.

Also, the singular pattern detection apparatus 100 may generate an object pattern of an object based on object information of an object created from the point in time when the object is identified in the region image. The object pattern may be generated based on object information from a point in time when an object is identified in the region image to a point in time when the object is not identified. The object pattern may include an object path, an object direction, a movement pattern, and an object speed generated based on object positions and identification times in a plurality of area images in which the object is identified. Also, the object pattern may include a time when an object is first identified and a time when identification of the object is finished.

In addition, the peculiar pattern detection apparatus 100 classifies object patterns according to preset time intervals and preset days of the week for a preset period of time to generate a plurality of object pattern groups, and to generate or learn a pattern model corresponding to each of the object pattern groups.

Also, the peculiar pattern detection apparatus 100 may search for a plurality of pattern models corresponding to the current time and the current day of the week, and generate a reference pattern model based on the searched pattern models. In addition, the peculiar pattern detecting apparatus 100 may compare object patterns corresponding to the current time with a reference pattern model to determine whether there is an abnormality.

In addition, the peculiar pattern detection apparatus 100 may display a region image, object patterns corresponding to the current time, a reference pattern model, and the presence or absence of abnormalities.

In addition, the apparatus 100 for detecting unusual patterns performs machine learning using learning data generated by labeling object patterns determined to be abnormal with actual abnormal situations to generate an abnormal pattern detection model that outputs types and probabilities of abnormal situations.

2 is a block diagram showing the configuration of the singular pattern detection device 100 according to an embodiment of the present invention.

Referring to FIG. 2 , the apparatus 100 for detecting a peculiar pattern includes an image receiving unit 101 that receives a region image photographed by a camera 200 . In one embodiment, the image receiving unit 101 may receive at least one area image per second from the camera 200 . That is, the area image is composed of a plurality of area images captured at preset time intervals, and the preset time is set as a reciprocal of the number of area images received per second.

In addition, the singular pattern detection device 100 includes an object identification unit 102 that identifies objects in the region image.

When a preset object is included in the received region image, the object identification unit 102 may detect an object included in the region image. In one embodiment, the object may be an image corresponding to a moving object such as a person, a four-wheeled vehicle (passenger car, bus, etc.), a two-wheeled vehicle, and the like. In the illustrated embodiment, a plurality of objects corresponding to a person, a plurality of objects corresponding to a car, and a plurality of objects corresponding to a truck are detected in the region image.

When an object is identified in the region image, the object identification unit 102 generates object information corresponding to the object in a plurality of region images from a point in time when the object is identified within the region image to a time point when the object is not identified. In one embodiment, the object information may include the type, size, color, location of the object in the area image, identification time, object ID, object area, and the like of the object. In one embodiment, a pre-learned machine learning model may be used to identify an object. The object identification unit 102 may input the area image into the machine learning model as an input value, and obtain the type of object included in the area image, the color of the object, and the position of the object in the area image from the machine learning model. In one embodiment, the location of the object in the area image may be the center location of the object area including the object. In one embodiment, the object area may be a rectangular area including an object inside. In an embodiment, the object identification unit 102 may use a pre-learned first machine learning model to output a rectangular object region including an object included in the region image when a region image including an object is input as an input value, output an object image included in a region occupied by an object in the partial image when a partial image included in the object region among the region images is input as an input value, and output features corresponding to objects when the object image is input as an input value. In one embodiment, Mask region convolutional neural network (Mask-rcnn) may be used for machine learning of the first machine learning model. In one embodiment, the object identification unit 102 may use a pre-learned second machine learning model to output an object type corresponding to an object when features output from the first machine learning model are input as input values. In one embodiment, random forest, xgboost, and the like may be used for machine learning of the second machine learning model. In one embodiment, the object identification unit 102 may use a pre-learned third machine learning model to output a color corresponding to the object image when the object image output from the first machine learning model is input as an input value. In one embodiment, a convolutional neural network (CNN) may be used for machine learning of the third machine learning model. In one embodiment, the object identification unit 102 may calculate the size of an object based on a partial image or an object image.

In order to identify objects included in the region image and generate object information, various well-known artificial neural networks may be used.

When an object is identified in the area image of any frame included in the area image, the object identification unit 102 assigns an object ID to the identified first object, and determines the type, color, and size of the first object. Also, the object identification unit 102 determines the type, color and size of the second object identified in the area image of the next frame. In addition, when the type and color of the second object are the same as the type and color of the first object, and at least a part of the object area of the second object overlaps the object area of the first object, the object identification unit 102 determines that the second object is the same as the first object and assigns the same object ID to the second object as the first object. Through this, the object identification unit 102 may identify the same object in successive frames. Through this, the object identification unit 102 may obtain object positions having a specific object ID within a plurality of consecutive frames and a plurality of corresponding area images.

Referring back to FIG. 2 , the apparatus 100 for detecting a peculiar pattern includes an object pattern management unit 103 that generates an object pattern of an object based on object information of an object created from the point in time when an object is identified in a region image.

FIG. 3 is a diagram illustrating a process of generating an object pattern by the object pattern management unit 103 according to FIG. 2 by way of example.

The object pattern management unit 103 classifies a plurality of object information generated for a plurality of objects identified from a plurality of consecutive frames and a plurality of corresponding area images based on the object ID.

The object pattern management unit 103 may generate an object pattern of an object based on a plurality of pieces of object information including the same object ID. The object pattern management unit 103 may generate an object path, object direction, movement pattern, and object speed based on a plurality of object locations included in object information including the same object ID.

In one embodiment, the object pattern management unit 103 may create an object path, which is a path created by continuously connecting a plurality of object locations. Also, the object pattern management unit 103 may determine the object direction based on displacement between two consecutive frames and corresponding two object positions. Also, the object pattern management unit 103 may determine a movement pattern of the object based on the object path and the object direction. In one embodiment, the movement pattern may include movement in a specific direction moving in a specific direction as a whole, stop without a change in location, change in location but no specific direction, and loitering within a limited area. In the illustrated embodiment, the object pattern management unit 103 may determine a movement pattern of an object having a specific object ID as "downward movement". In addition, the object pattern management unit 103 may calculate the object instantaneous velocity of an object between two consecutive frames by dividing the distance between two consecutive frames and the corresponding two object positions by the time between the frames. In one embodiment, a unit of the distance between two object positions may be the length of one pixel. In an embodiment, the object pattern management unit 103 may determine an average value of a plurality of object instantaneous velocities calculated among all frames in which the object is identified as the object speed.

The object pattern management unit 103 may create an object pattern including information about object type, object path, object direction, movement pattern, and object speed. The object pattern management unit 103 may generate a plurality of object patterns corresponding to a plurality of objects identified in the region image and given different object IDs.

Referring back to FIG. 2 , the peculiar pattern detection apparatus 100 includes a pattern model management unit 104 that generates a pattern area based on a plurality of object patterns corresponding to a plurality of objects assigned different object IDs.

FIG. 4 is a flowchart illustrating a process of generating a pattern model by the pattern model management unit 104 according to FIG. 2 .

The pattern model management unit 104 classifies a plurality of object patterns collected during a preset period according to preset time intervals and days of the week to create a plurality of object pattern groups (S110).

In one embodiment, the preset period may be a period of 30 days prior to the current time. In one embodiment, the preset time interval may be 10 minutes. In one embodiment, the preset day of the week may be Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday, and public holidays. For example, object patterns collected from Monday 14:00 to Monday 14:10 are included in the same object pattern group.

Also, the pattern model management unit 104 determines object patterns having the same object type and object direction as primary object patterns among a plurality of object patterns included in the object pattern group (S120).

In one embodiment, the object direction may be up, down, left, right, up left, down left, up right, down right, stop, wander, and the like.

In addition, the pattern model management unit 104 determines primary object patterns overlapping at least a part of the object paths of the primary object patterns as secondary object patterns (S130).

In one embodiment, when at least a part of an object path of another object pattern is included in a preset area around an object path of one object pattern, one object pattern and another object pattern may be determined to overlap each other.

Also, the pattern model management unit 104 creates pattern areas corresponding to the secondary object patterns based on the object paths of the secondary object patterns (S140).

In one embodiment, the pattern model management unit 104 may set a region included in a preset range around the object path of each of the secondary object patterns, and set the pattern region by combining the regions of the object paths of the secondary object patterns.

In one embodiment, the pattern model management unit 104 may set a region included within a preset range around the object path of each of the secondary object patterns, and set a portion where the number of overlapping regions is equal to or greater than the preset number as the pattern region.

In addition, the pattern model management unit 104 determines whether there is an already created pattern model having a pattern area in which the object type and object direction of the secondary object patterns and the object type and object direction are the same and at least partially overlaps with the pattern area of the secondary object patterns among the pattern models already created (S150).

In addition, among the already created pattern models, if there is an already created pattern model having a pattern area in which the object type and object direction of the secondary object patterns are the same and the pattern area at least partially overlaps with the pattern area of the secondary object patterns, the pattern model management unit 104 learns the already created pattern model using the secondary object patterns (S170).

In an embodiment, the already created pattern model may be trained based on both existing secondary object patterns and secondary object patterns used to generate the already created pattern model. In one embodiment, the pattern model may include object type, object direction, pattern area, number of secondary object patterns, distribution of secondary object patterns, average speed of secondary object patterns, and the like. In one embodiment, a pattern area of a pattern model already generated through learning through secondary object patterns, the number of secondary object patterns, and a distribution of secondary object patterns may be changed.

In addition, among the already created pattern models, if there is no already created pattern model having a pattern area in which the object type and object direction of the secondary object patterns are the same and the pattern area at least partially overlaps with the pattern area of the secondary object patterns, the pattern model management unit 104 creates a pattern model using the secondary object patterns (S160).

In one embodiment, the pattern model may include object type, object direction, pattern area, number of secondary object patterns, average speed of secondary object patterns, and the like.

The pattern model management unit 103 generates a plurality of pattern models based on object types and object directions.

Referring back to FIG. 2 , the singular pattern detection device 100 includes an object pattern monitoring unit 105 that detects a singular pattern.

FIG. 5 is a flowchart illustrating a process of determining whether or not there is an abnormality based on object patterns included in a region image by the object pattern monitoring unit according to FIG. 2 .

First, the object pattern monitoring unit 105 searches the database for a plurality of pattern models corresponding to the local time and the current day of the week (S210).

In addition, the object pattern monitoring unit 105 determines, among the searched pattern models, pattern models having the same object type and object direction and overlapping at least a part of the pattern area as first pattern models (S200).

In addition, the object pattern monitoring unit 105 determines a sensitivity condition based on each of the secondary object patterns of the first pattern models (S230).

In one embodiment, the sensitivity condition may include the number of secondary object patterns and the average speed of the secondary object patterns.

In an embodiment, the object pattern monitor 105 may determine a range of the number of secondary object patterns of the first pattern models and a range of speeds of the secondary object patterns as sensitivity conditions. For example, the sensitivity condition may include a number range in which the average value obtained by subtracting the standard deviation of the number of secondary object patterns from the average value of the number of secondary object patterns of primary pattern models is the minimum value, and the value obtained by adding the standard deviation of the number of secondary object patterns to the average value of the secondary object patterns is the maximum value. For example, the sensitivity condition may include a speed range in which the average value obtained by subtracting the standard deviation of the average speed of the secondary object patterns from the average value of the average speed of the secondary object patterns of the primary pattern models is the minimum value, and the average value of the average speed of the secondary object patterns plus the standard deviation of the average speed of the secondary object patterns is the maximum value. However, it is not limited thereto, and various number ranges and speed ranges may be included in the sensitivity conditions. For example, sensitivity can be improved by setting the difference between the maximum value and the minimum value smaller.

Also, the object pattern monitoring unit 105 determines, as second pattern models, first pattern models in which secondary object patterns satisfy a sensitivity condition among the first pattern models (S240).

Among the first pattern models, first pattern models in which the number of secondary object patterns is included in the number range and the average speed of the secondary object patterns is included in the speed range are determined as second pattern models.

In addition, the object pattern monitoring unit 105 generates a reference pattern model by combining the second pattern models (S250).

In one embodiment, the object pattern monitoring unit 105 may generate a plurality of reference pattern models. For example, the object pattern monitoring unit 105 may generate a reference pattern model in which the object type is a vehicle and a reference pattern model in which the object type is a person.

In addition, the object pattern monitoring unit 105 compares the object patterns of the region image with the reference pattern model to determine whether there is an abnormality (S260).

In one embodiment, the object pattern monitoring unit 105 compares the object patterns of the region image corresponding to the current time with the reference pattern model to determine whether there is an abnormality.

In one embodiment, the object pattern monitoring unit 105 may compare object patterns recognized during a preset time interval before the current time with a reference pattern model to determine whether there is an abnormality. In one embodiment, the preset time interval may be 10 minutes.

In one embodiment, the object pattern monitoring unit 105 may generate a plurality of monitoring object pattern groups of object patterns based on the pattern regions of the second pattern models included in the reference pattern model. For example, when the object path of any one object pattern is included in the pattern area of any one second pattern model, the object pattern monitoring unit 105 may include the any one body pattern in the monitoring object pattern group corresponding to the any one second pattern model.

It is possible to determine whether there is an abnormality by comparing each of the monitoring object pattern groups with each of the second pattern models. For example, when the number of object patterns or the average speed of the object patterns in the object pattern group to be monitored does not satisfy the sensitivity condition of the second pattern model, the object pattern monitoring unit 105 may determine the object patterns included in the object pattern group to be monitored as abnormal. For example, if the object type or object direction among the object patterns of the monitored object pattern group is not the same as the object type or object direction of the second pattern model, the object pattern monitoring unit 105 may determine the object patterns included in the monitored object pattern group to be in an abnormal state.

Referring back to FIG. 2 , the singular pattern detection device 100 includes an image monitoring unit 106 that displays an area image.

FIG. 6 is a flowchart illustrating a process of monitoring a region image by the image monitoring unit 106 according to FIG. 2 .

The image monitoring unit 106 receives a region image corresponding to the current time, an object pattern of the region image corresponding to the current time, a reference pattern model, and an abnormality from the object pattern monitoring unit 105 .

First, the image monitoring unit 106 determines whether the object patterns of the region image corresponding to the current time are in a normal state (S310).

In an embodiment, the image monitoring unit 106 may determine whether the object patterns of the region image corresponding to the current time are in a normal state based on the abnormalities received from the object pattern monitoring unit 105 .

Also, when the object patterns are in a normal state, the image monitoring unit 106 displays the region image corresponding to the current time, the object patterns of the region image corresponding to the current time, and the reference pattern model through the output interface device (S320).

Also, when the object patterns are in an abnormal state, the image monitoring unit 106 generates an abnormal pattern based on the object patterns of the region image corresponding to the current time (S330).

In one embodiment, the image monitoring unit 106 may generate an abnormality pattern based on object patterns included in a monitoring object pattern group corresponding to a second pattern model in which an abnormality has occurred among second pattern models included in the reference pattern model. The abnormal pattern may include the ratio of object types of object patterns included in the monitoring object pattern group, the number of object patterns, the ratio of object directions of object patterns, the speed of object patterns, the pattern area, and the like.

In addition, the image monitoring unit 106 displays a region image corresponding to the current time, object patterns corresponding to the current time, a reference pattern model, and an abnormal pattern notification through the output interface device (S340).

Through this, the controller controlling the traffic situation can recognize that an abnormal state has occurred.

Also, the video monitoring unit 106 determines whether a deletion request for the generated abnormal pattern is received (S350).

The video monitoring unit 106 may receive a deletion request for the generated abnormal pattern from an operator through an input interface device.

When the deletion request is received, the image monitoring unit 106 determines the generated abnormal pattern as a user-specified pattern model (S360).

The image monitoring unit 106 compares the generated abnormal pattern with a user-designated pattern model, and does not display an abnormal pattern notification when the generated abnormal pattern corresponds to the user-specified pattern model.

FIG. 7 is a flowchart illustrating a process in which the pattern model management unit 104 according to FIG. 2 performs machine learning on an abnormal pattern detection model.

The pattern model management unit 104 generates learning data in which the type of abnormal situation is labeled in the abnormal pattern (S410).

The pattern model management unit 104 may receive an abnormal situation type corresponding to an abnormal pattern from a controller, and may label the abnormal situation type to the abnormal pattern. For example, cases in which a person is hit by a person on a road, a collision between vehicles on a road, a case in which a vehicle drives in reverse on a road, and a case in which a person is knocked down on a walk with few people are examples of abnormal situations.

The pattern model management unit 104 may perform machine learning on an abnormal pattern detection model that outputs the type and probability of an abnormal situation when an abnormal pattern is input as an input value using learning data (S420).

In one embodiment, in machine learning, a multiple regression analysis method can be used in which the ratio of object types of object patterns included in the monitoring object pattern group included in the abnormal pattern, the number of object patterns, the ratio of object directions of object patterns, the speed and pattern area of the object pattern as independent variables and the type of abnormal situation as a dependent variable. In one embodiment, a random forest algorithm, an XGBoost algorithm, or the like may be used for machine learning. However, it is not limited thereto, and various known machine learning methods may be used.

In one embodiment, the abnormal pattern detection model may be generated for each preset time interval and for each day of the week.

FIG. 8 is a flowchart illustrating a process in which the object pattern monitoring unit 105 according to FIG. 2 identifies the type of abnormal pattern.

First, the object pattern monitoring unit 105 inputs the generated abnormal pattern as an input value to the abnormal pattern detection model, and obtains the type and probability of the abnormal situation from the abnormal pattern detection model (S510).

In addition, the object pattern monitoring unit 105 determines whether the obtained probability is greater than or equal to a preset reference probability (S520).

In addition, when the obtained probability is greater than or equal to a preset reference probability, the object pattern monitoring unit 105 provides the obtained abnormal situation type to the image monitoring unit 106, and the image monitoring unit 106 displays the abnormal situation type (S530).

Through this, since the type of abnormal situation occurring in the traffic situation can be known without a resident controller, the operation cost of the traffic control system can be reduced and at the same time, prompt and accurate response to the abnormal situation can be made.

FIG. 9 is a diagram showing the hardware configuration of the singular pattern detection device 100 according to FIG. 1 by way of example.

Referring to FIG. 9, the singular pattern detection apparatus 100 may include at least one processor 110 and a memory for storing instructions instructing the at least one processor 110 to perform at least one operation.

The at least one operation may include the components 101 , 102 , 103 , 104 , 105 , and 106 of the above-described peculiar pattern detection device 100 or other functions or operation methods.

Here, the at least one processor 110 may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. Each of the memory 120 and the storage device 160 may include at least one of a volatile storage medium and a non-volatile storage medium.

For example, the memory 120 may be one of a read only memory (ROM) and a random access memory (RAM), and the storage device 160 may be a flash-memory, a hard disk drive (HDD), a solid state drive (SSD), or various memory cards (eg, a micro SD card).

In addition, the singular pattern detection device 100 may include a transceiver 130 that performs communication through a wireless network. In addition, the singular pattern detection device 100 may further include an input interface device 140 , an output interface device 150 , a storage device 160 , and the like. Each component included in the singular pattern detection device 100 is connected by a bus 170 to communicate with each other.

For example, a communicable desktop computer, a laptop computer, a notebook, a smart phone, a tablet PC, a mobile phone, a smart watch, smart glasses, an e-book reader, a portable multimedia player (PMP), a portable game machine, a navigation device, a digital camera, and a digital multimedia broadcasting (DMB) It may be a player, a digital audio recorder, a digital audio player, a digital video recorder, a digital video player, a personal digital assistant (PDA), and the like.

The methods according to the present invention may be implemented in the form of program instructions that can be executed by various computer means and recorded on a computer readable medium. Computer readable media may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on a computer readable medium may be specially designed and configured for the present invention or may be known and usable to those skilled in computer software.

Examples of computer readable media may include hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions may include not only machine language codes generated by a compiler but also high-level language codes that can be executed by a computer using an interpreter and the like. The hardware device described above may be configured to operate with at least one software module to perform the operations of the present invention, and vice versa.

In addition, the above-described method or device may be implemented by combining all or some of its components or functions, or may be implemented separately.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art can variously modify and change the present invention without departing from the spirit and scope of the present invention described in the claims below. It will be understood that it can be changed.

Claims (1)

In the traffic situation specific pattern detection device using an artificial neural network,
an object identification unit identifying an object from a plurality of area images captured using a camera, and generating a plurality of object information items corresponding to each of the area images in which the object is identified among the area images;
an object pattern management unit generating an object pattern for the object based on the object information;
A pattern model management unit that classifies the plurality of object patterns generated during a preset period by preset time intervals and preset days of the week to create a plurality of object pattern groups, and creates or learns a pattern model corresponding to each of the object pattern groups;
An object pattern monitoring unit that searches a database for a plurality of the pattern models corresponding to the current time and the current day of the week, creates a reference pattern model based on the searched pattern models, and compares the reference pattern model with a plurality of monitored object patterns, which are the object patterns corresponding to the current time, to determine whether there is an abnormality; and
a video monitoring unit for displaying the surveillance object patterns and the region images including the surveillance object patterns when an abnormal state does not occur, and displaying the surveillance object patterns and an abnormal pattern notification when an abnormal state occurs;
Each of the object information includes the object type, object location, and identification time of the object,
The object pattern management unit,
determining an object path, object direction, and object speed for the object based on the object location and the identification time of the object information, and generating the object pattern including the object type, the object path, the object direction, and the object speed;
The pattern model management unit,
Among the object patterns included in the object pattern group, object patterns having the same object type and object direction are determined as primary object patterns;
determining primary object patterns overlapping at least a part of the object path among the primary object patterns as secondary object patterns;
Creating pattern areas corresponding to the secondary object patterns based on the object path of the secondary object patterns;
The pattern model management unit,
Among the plurality of pattern models that have already been created, the already created pattern model, in which the secondary object patterns, the object type and the object direction are the same, and the secondary object patterns and the pattern area overlap at least in part, is searched in the database,
When the already created pattern model is searched for, the searched already created pattern model is learned using the secondary object patterns,
When the already created pattern model is not searched for, generating the pattern model using the secondary object patterns;
The object pattern monitoring unit,
Among the searched pattern models, pattern models having the same object type and the same object direction and overlapping at least a part of the pattern area are determined as first pattern models;
determining a sensitivity condition based on the secondary object patterns of each of the first pattern models;
Among the first pattern models, first pattern models for which the secondary object patterns satisfy the sensitivity condition are determined as second pattern models;
Combining the second pattern models to generate reference pattern models;
Comparing the monitoring object patterns with the reference pattern model to determine the presence or absence of the abnormality;
The pattern model management unit,
Creating learning data labeled with the type of abnormal situation in the monitoring object pattern;
Performing machine learning on an anomaly detection pattern detection model that outputs the type and probability of an anomaly situation when the monitoring object pattern is input as an input value using the learning data,
Unique pattern detection device.