KR20210158037A - Method for tracking multi target in traffic image-monitoring-system - Google Patents

Abstract

An embodiment relates to a method for capturing a location and a motion of a high-speed moving object in an image monitoring system. Specifically, the method collects monitoring images for each camera device installed in each of a plurality of different monitoring areas during image monitoring. Then, feature metadata of the moving object is extracted from the monitoring image collected in this way, and a feature of the moving object is extracted from a preset AI format which extracts a feature value of the image patch according to the feature metadata of the moving object. Therefore, by recognizing the moving object by the preset AI format for estimating the size and location of the entire moving object from the extracted features of the moving object, the moving object is tracked for each moving object. Therefore, through this, the image monitoring system is systematically effective and quickly captures the moving object at high speed.

Description

Position and motion capture method of a high-speed moving object in a video surveillance system {Method for tracking multi target in traffic image-monitoring-system}

The content disclosed in this specification relates to the field of data processing technology, and for example, in an image monitoring system for a vehicle on a road or a person in an offender area, relates to a technology for capturing and tracking a moving object in real time.

Unless otherwise indicated herein, the material described in this section is not prior art to the claims of this application, and inclusion in this section is not an admission that it is prior art.

In general, for example, a traffic signal control system is an important system for increasing safety and communication efficiency by controlling the movement priority of conflicting traffic flows at an intersection. However, incorrect identification of the status quo and signal operation rather worsens communication efficiency, aggravates congestion, and induces drivers' unreasonable driving patterns, which leads to danger.

Currently, traffic signal control systems have been established in 21 metropolitan cities and small and medium-sized cities, including Seoul, but they cannot be operated in a responsive manner due to limitations in the management and operation of the loop detector, a traffic information collection system, and most intersections are operated in the TOD method. have.

In addition, the fixed type plans the signal time based on the approximate traffic volume of the corresponding intersection, and the TOD method is based on the traffic volume determined by conducting a full investigation of the traffic volume during commuting time and during the daytime and early morning hours with constant traffic volume. The signal time is calculated.

Also, in terms of maintenance, a fixed or TOD signaling method is advantageous for an intersection where the traffic pattern is kept constant. In some cases, it is operated inefficiently because it fails to respond to the

Meanwhile, in recent years, the vehicle number recognition system has been used in illegal parking enforcement devices, vehicle speed monitoring devices, vehicle access systems in buildings, and CCTV security systems. The vehicle number recognition system includes a camera, which is a photographing device, and an information processing unit for recognizing a vehicle number by analyzing an image photographed through the camera. The camera can be installed fixedly or mobile.

These license plates are metal or plastic plates that are attached to vehicles for identification purposes. The license plate contains letters, numbers, or a combination thereof, and the shape of the license plate is determined by policy for each country or administrative region.

Conventional methods for detecting license plates of vehicles include a character-based detection method, an edge-based detection method, a color-based detection method, a texture-based detection method, and the like. The character-based detection method is slow in processing speed and requires setting the character area, the edge-based detection method is dependent on empirical variables (eg, aspect ratio, etc.), and the color-based detection method is sensitive to changes in lighting or shadows, The texture-based detection method has a problem of slow processing speed.

And, furthermore, the license plate recognition technology is a core technology in the field of intelligent traffic systems (ITS) used in traffic enforcement systems, parking control systems, and ADAS systems. Recently, the license plate recognition function has been increased through deep learning technology and it is effectively operating in an embedded environment, so it is possible to seek diversification of the license plate recognition technology.

On the other hand, as reported in recent news, when a fire occurs, there are problems such as secondary damage due to vehicles parked near the fire site, and delay in fire suppression time due to vehicles parked near the fire engine access road.

For example, as in the case of the Jecheon Sports Center fire (December 2017), the entry of a fire engine is delayed due to an illegally parked vehicle on the driveway around the building. Here, a fire engine arrived at the scene in 7 minutes at the time of the fire, but a ladder truck could not enter due to an illegally parked vehicle. In addition, even after the fire incident, there were a lot of illegally parked vehicles on nearby roads, and related problems occurred due to the lack of safety awareness of citizens.

In this regard, on the other hand, when damage is caused by typhoons or torrential rain, flooding of vehicles in parking lots in high rivers, which is difficult to manage, continues to occur.

And, on the other hand, there is no notification service for immediate response of nearby vehicle owners in the event of a disaster. In addition, the current emergency disaster text broadcasting (CBS: Cell Broadcasting Service) of the Ministry of Public Administration and Security is a method of collectively transmitting messages to all mobile phones within the radius of a base station in the area, and disaster texts are transmitted to an unspecified number of people.

Therefore, it is necessary to develop an alarm notification system that combines a portable recognition and information transmission terminal using a license plate recognition technology and an administrative information joint use system. In addition, a vehicle number recognition technology suitable for this is also required.

In case of examining the related prior art in this background, the domestic prior art directly related as the related prior art by examining the prior art from the patent document is not searched, but the following patent documents are only available for reference.

(Patent Document 1) KR1020190143548 A

For reference, the technology of Patent Document 1 is improved license plate recognition rate through vehicle number, type, and color learning, and is a technology degree related to a vehicle recognition system capable of real-time learning.

In this regard, in particular, a function of tracking a multi-target is required in a traffic video surveillance system, and in order for this function to operate effectively, it is necessary to secure an accurate location for a plurality of objects and follow it in real time.

On the other hand, recently, various types of surveillance systems using cameras are widely used in industrial fields, and these camera surveillance systems are actively used for security and enforcement purposes.

In the case of some vehicle enforcement cameras installed on the road (e.g., parking enforcement or cut-in enforcement cameras), they are installed to shoot the required area through the camera's posture control (adjustment of shooting direction), but currently, visitors Most security surveillance cameras that monitor intruders are global surveillance cameras that capture the entire surveillance area within a specific distance and angle of view.

The global surveillance camera has a limitation in resolution because it monitors a large area determined by a fixed camera, and for this reason, in most cases, the face of a person in the surveillance area cannot be accurately identified with the image captured by the global surveillance camera.

Accordingly, in recent years, a system in which a camera dynamically tracks a person has been developed.

However, it is difficult to apply the tracking technology to the actual field due to the low level of completion of the tracking technology. Reality.

Accordingly, there is a need for a technology that allows the camera to accurately track and photograph only humans, and when there are several monitoring targets, there is an urgent need to develop a technology for monitoring multiple targets.

Representative methods related to tracking known in the prior art and their technical weaknesses are as follows.

1) Representative method of background and object segmentation (difference image algorithm/background difference technique)

- Difference Picture Algorithm

It is a method of separating an object from a background by comparing two images delayed by a predetermined time, and it is possible to effectively detect a moving object in a fixed background.

- Background Model

This is a method of adding the influence of the current frame to a background image as a reference by examining every frame before the current image, and is a technique mainly used when generating a background image.

ㆍTemporal smoothing method: use the average of pixel values of previous frames

ㆍTemporal median method: use the value displayed in the previous frame in any pixel

- Technical vulnerabilities

If there is little or no movement, it is impossible to detect an object (recognized as a background), and if the boundary of the object is similar to the background, a recognition error occurs. .

2) Detection and tracking of moving objects (feature-based method/area-based method)

- Feature-based method

ㆍMethod using optical flow: A method using the feature that only spatial changes exist for two consecutive frames obtained within a short time and the light energy of the pixel itself is preserved

ㆍMethod using straight lines: Since the number of straight lines in one image is small and the lines themselves have distinct characteristics, the number of potential registration cases can be reduced.

ㆍMethod using Corner Point: Object tracking by characterizing the shape of a moving object with feature flow-based information

- Region-based method: A method of finding an image fragment of a certain region including feature points in one image using correlation with the next image

- Block Matching Method: A method of comparing correlation with the sum of the pixel-based mean square difference and absolute value difference

- Technical vulnerabilities

The calculation process for matching features or blocks is complicated (real-time processing delay), and for object detection, databaseization of object features is required in the system.

3) Stereo based object tracking (optical JTC tracking/BPEJTC tracking)

It is a technology that acquires distance information from the left/right stereo input image to the tracking object and tracks the object by controlling both cameras with coordinates.

- Optical JTC (Joint Transform Correlator) Tracking System: By locating two target video signals on the optical JTC input plane, tracking the movement trajectory of the image based on the measurement of the position change between frames

- BPEJTC (Binary Phase Extraction JTC) tracking system: Uses binary phase type optical JTC with excellent discrimination power for similar images

- Technical vulnerabilities

If there is a lot of background noise, if the backgrounds of the left and right images are different, or if the background changes due to the movement of the camera, tracking is not possible.

As mentioned above, to summarize the weaknesses of the conventional tracking method, first, the conventional tracking method uses a method to detect and track motion, and secondly, a method for tracking motion using the difference between frames is used. is to do

In conclusion, these methods are not easy to implement in real time because they use a mathematical approach or the algorithm is too complex, and when there is no movement, human recognition and tracking are impossible.

Therefore, it is necessary to develop a technology that can accurately identify a target object in real time using a simple tracking calculation formula.

The disclosed subject matter is to provide a method for capturing the position and motion of a high-speed moving object in an image monitoring system that enables the high-speed moving object to be captured systematically and quickly in an image monitoring system.

A method of capturing the position and motion of a high-speed moving object in an image monitoring system according to an embodiment,

During video surveillance, surveillance images are collected for each camera device installed in each of a plurality of different surveillance areas.

Then, the feature metadata of the moving object is extracted from the surveillance image collected in this way, and the feature of the moving object is extracted from the preset AI format that extracts the feature value of the image patch according to the feature metadata of the moving object.

Therefore, by recognizing a moving object by a preset AI format for estimating the size and position of the entire moving object from the extracted features of the moving object, tracking is performed for each moving object.

According to the embodiments, a high-speed moving object is captured systematically and effectively in an image surveillance system.

Additionally, by applying artificial intelligence image analysis technology, image analysis is systematic and reliable, and appropriate image analysis and operation are performed according to changes in the environment.

In addition, it is possible to improve the specific rate of the moving object through objectification by extracting the features of the moving object image.

1 is a conceptual diagram to which a method of capturing a position and a motion of a high-speed moving object in an image monitoring system according to an embodiment is applied; FIG.
2 is a view showing the overall system to which the position and motion capture method of a high-speed moving object in the video surveillance system according to an embodiment is applied;
3 is a block diagram illustrating the configuration of a camera device to which a method of capturing a position and a motion of a high-speed object in an image monitoring system according to an embodiment is applied; FIG.
4 is a flowchart sequentially illustrating a method of capturing a position and a motion of a high-speed moving object in an image monitoring system according to an embodiment;
5 is a diagram for explaining in more detail an example of an object metadata extraction operation according to an embodiment applied to a method for capturing the position and motion of a high-speed moving object in an image monitoring system according to an embodiment;
6 is a diagram for explaining a CNN-based object classification operation according to an embodiment applied to a method for capturing the position and motion of a high-speed moving object in a video surveillance system according to an embodiment
7 and 8 are diagrams for explaining in more detail an abnormality detection operation applied to a method for capturing a position and motion of a high-speed moving object in an image monitoring system according to an embodiment;

1A is a conceptual diagram to which a method of capturing a position and a motion of a high-speed moving object in an image monitoring system according to an embodiment is applied.

As shown in FIG. 1 , the method according to an exemplary embodiment basically uses the camera device 100 installed on each sector when a vehicle travels on a road or when a person passes through a crime zone. Basic information for video surveillance is acquired by collecting surrounding surveillance images and transmitting them to the central control center.

Then, in the method according to an embodiment, the central control center analyzes the image through the surveillance image.

Therefore, based on the results of image analysis, traffic policies related to signal policies in the relevant area are derived, or abnormal traffic signs or crimes are predicted.

The camera device 100 is installed in each of a plurality of different monitoring areas, for example, at major intersections, access roads, and crime areas, and collects monitoring images of its own monitoring area as basic information for video monitoring. will be provided to

FIG. 2 is a view showing the overall system to which a method of capturing a position and a motion of a high-speed moving object in an image monitoring system according to an embodiment is applied.

As shown in FIG. 2, the system to which the method of capturing the position and motion of a high-speed moving object in the image monitoring system according to an embodiment is applied is a camera device 100-1, 100-2, ... 100-m, 100 -n, ... ) and the central control center (200).

The camera devices 100-1, 100-2, ... 100-m, 100-n, ... are, for example, a major intersection, an access road, an intersection, an off-the-cuff traffic area, and an offender for each of a number of different surveillance areas. It is installed in each region, and collects surveillance images for its own surveillance area and provides it to the central control center 200 through its own network as basic information for video surveillance.

The central control center 200 basically collects and analyzes such surveillance images from the camera devices 100-1, 100-2, ... 100-m, 100-n, ... The analysis results, etc. are provided to the disaster prevention management office or an external affiliated organization. At this time, the image analysis is, for example, in the case of a preset dangerous motion by capturing the position and motion of a person in the surveillance image, that is, when a crime is estimated to occur, an external linkage such as a pre-registered police station information processing device This is done by notifying the In this case, the external linkage is a police station information processing device, a transportation information processing device, an emergency rescue information processing device, a fire station information processing device, or the like.

3 is a block diagram illustrating a configuration of a camera device to which a method of capturing a position and a motion of a high-speed object in an image monitoring system according to an exemplary embodiment is applied.

As shown in FIG. 3 , the camera device 100 to which the method of capturing the position and motion of a high-speed moving object in an image monitoring system according to an embodiment is applied includes a camera module 110 , a DSP unit 120 , and a sensor unit. 130 and a PLC controller 140 .

In this case, the camera device 100 according to an embodiment includes an input module for which the PLC controller 140 basically receives a signal from a sensor unit, a CPU module as a central control unit, and an output module for outputting a control signal to a control target. includes

And, the PLC controller 140 is provided with an IOT module that performs an IOT function in the PLC itself at this time, and the CPU module interworks with the input/output module by such an IOT module to perform an IOT function from the corresponding control logic. to do it

The camera module 110 captures an image of a preset monitoring area and transmits it to the DSP unit 120 .

The DSP unit 120 processes the captured image as a digital signal and provides it to the PLC controller 140 .

The sensor unit 130 detects information about the surrounding environment of the monitoring area. The sensor unit 130 includes, for example, an analog sensor that detects fine dust in the monitoring area, a digital switch that detects whether the state of a crosswalk traffic light is changed by the manipulation of a passerby, or various digital sensors.

The PLC controller 140 provides the result processed by the DSP unit 120 to the central control sensor 200 registered in advance, and provides information on the surrounding environment of the monitoring area detected by the sensor unit 130 . Integrated I/O processing is provided to the central control center 200, and it is transmitted to a pre-registered control target. In addition, in this case, the PLC controller 140 has its own IOT module and provides the processing result of the DSP unit provided by the IOT module and the surrounding environment information of the monitoring area to the central control center.

In addition, if we explain further in this regard, modules with various functions are required for the existing PLC system used in various environments, and accordingly, PLC manufacturers provide various modules that satisfy the requirements of users. For example, a module having various functions, such as a digital input/output module, an analog input/output module, and a communication module, is used in a PLC system, and a system desired by a user is built through these various modules.

For example, the technology of patent document KR101778333 Y1 is an invention registered as such a technology, and specifically relates to a PLC system having a diagnostic module for diagnosing whether an output module of the PLC is defective in operation.

The above-described IOT module according to an embodiment uses these points to provide a PLC controller that provides an IOT function from the IOT module, and furthermore, through this, monitoring and control can be easily performed by a user.

Accordingly, according to an embodiment, the PLC controller 140 includes an input module that receives a signal from a sensor unit, a CPU module that is a central control unit, and an output module that outputs a control signal to a control target.

In addition, the PLC controller 140 includes an IOT module that additionally performs an IOT function in the PLC controller itself, so that the CPU module interworks with the input/output module by such an IOT module and provides an IOT function from the corresponding control logic. to perform

For reference, here, the input module receives, for example, analog traffic environment information and digital traffic environment information related to traffic environment information. For example, the input module receives, as analog traffic environment information, the fine dust information around the camera device from the sensor unit, for example, a fine dust sensor. Alternatively, through the D/I module, the sensor unit, for example, the crosswalk traffic light change switch, receives the pedestrian operation information regarding the change of the crosswalk traffic light around the camera device is installed as digital traffic environment information. In this case, the input module is defined as including an I/O card as an input module of the PLC.

In addition, when a signal is input through the input module and a control signal is generated by the CPU module, the output module sends the control signal to, for example, a fine dust reduction device installed around a camera device or a traffic light. to convey

In addition, the IOT module performs an IOT function in the PLC controller itself in this case according to an embodiment. Specifically, the IOT function first performs its own wired/wireless communication when traffic environment information is collected as described above. For example, a LoRa module is provided to provide traffic environment information to an external central control center.

In addition, when a signal is input by the input module, the CPU module processes the input signal according to preset control logic to generate different control signals. And, according to an embodiment, when data is collected by the above-described IOT module, the CPU module processes the collected data according to a preset control logic to generate different control signals, thereby controlling differently according to the IOT data. .

Additionally, in addition to the above, the PLC controller 140 issues an alarm or the like with a voice designated for data collected from the I/O card of the PLC controller.

To this end, the IOT module is equipped with its own TTS engine, and according to the voice information for each data preset for the data collected from the input module of the PLC, for example, the IOT module makes a different voice alarm according to the voice message.

Specifically, the IOT module has its own TTS engine to generate a voice alarm according to voice information corresponding to the surrounding environment information of the monitoring area.

In this case, the voice message is registered in, for example, a flash voice memory provided in the IOT module itself.

So, through this, the data collected from the PLC's I/O card and the like are alarmed with a designated voice.

In addition, the PLC controller 140 additionally performs a noise canceling function for an external voice.

Specifically, the IOT module performs an audio IOT function by receiving an external voice input, performing noise cancellation, and outputting audio.

For example, in this case, an audio IOT function is performed by receiving an external audio input, performing noise cancellation, and outputting audio through an audio amplifier provided in the IOT module itself.

4 is a flowchart sequentially illustrating a method of capturing a position and a motion of a high-speed moving object in an image monitoring system according to an embodiment.

As shown in FIG. 4 , the method for capturing the position and motion of a high-speed moving object in the video surveillance system according to an embodiment is based on the multi-target tracking method of the traffic video surveillance system by the central control center.

First, in the method of capturing the position and motion of a high-speed moving object in an image monitoring system according to an embodiment, a monitoring image is collected for each camera device installed in each of a plurality of different monitoring areas during video monitoring (S401).

Next, the feature metadata of the preset moving object is extracted from the surveillance image collected in this way (S402).

In this case, the feature metadata of the moving object means metadata indicating attributes or characteristics of the vehicle, for example, when the moving object is a vehicle.

Then, the feature of the moving object is extracted from the preset AI format for extracting the feature value of the image patch according to the extracted feature metadata of the moving object (S403).

In this case, the AI format is, for example, a convergence machine learning AI format of GLCM (Grey Level Co-occurence Matrix).

In this case, the GLCM basically has various definitions for a texture image, but represents an image highlighting spatial characteristics included in a target surveillance image.

In this case, the GLCM calculates basic statistics such as the average, contrast, and correlation of the brightness values of the current pixel and its neighboring pixels. So, the calculated value is again assigned as a new brightness value to the central pixel in the kernel and expressed as a partial texture feature for the input monitoring image.

In this case, GLCM includes Homogenity, Contrast, and Dissimiliarity.

The homogenity represents the uniformity of each pixel in the matrix. And, Contrast and Dissimiliarity are concepts for expressing the difference in contrast between pixels, and a high weight is applied to pixels far away from the diagonal. ASM and energy measure the uniformity of light and dark. There is no change in brightness between pixels, and the values of each pixel become similar.

On the other hand, when extracting the features of a moving object, a deep learning cluster is applied.

The deep learning cluster extracts feature information for moving object recognition for recognition from the moving object. The extracted feature information for moving object recognition is stored in the learned model big data storage unit. In the characteristic information for moving object recognition extracted by such a deep learning cluster, for example, an image name (eg, object name, image frame information and time), an image including a deep vehicle number, an example of the type of object For example, a vehicle, etc., and the color of an object may be included.

Next, the moving object is recognized by a preset AI format for estimating the size and position of the entire moving object from the features of the moving object extracted in this way (S404).

In this case, the AI format is, for example, a YOLO format.

More specifically, the YOLO format estimates the size and position of the entire moving object by estimating and combining the total size and position of each moving object for a plurality of different moving object features.

At this time, it is possible to check the image of the moving object by accurately measuring the position of the target, such as a person, in the space of three windows.

To this end, in one embodiment, the deep learning parameters for the object region are calculated in advance through deep learning for 2D object region division from only a plurality of different RGB images, and the recognized by the object 2D region division deep learning model. A moving object is divided into an object 2D area, logical AND with depth information is performed, and a perspective transformation is applied to the depth information of the moving object to perform 3D object area division of the moving object.

So, the multi-target is tracked by tracking each recognized moving object (S405).

Therefore, through this, in the video surveillance system, it is systematically effective and quickly captures the position and motion of a moving object at high speed.

As described above, an embodiment collects monitoring images for each camera device installed in each of a plurality of different monitoring areas during video monitoring.

Then, the feature metadata of the moving object is extracted from the surveillance image collected in this way, and the feature of the moving object is extracted from the preset AI format that extracts the feature value of the image patch according to the feature metadata of the moving object. .

Therefore, by recognizing a moving object according to a preset AI format for estimating the size and position of the entire moving object from the extracted features of the moving object, each moving object is tracked.

Therefore, through this, in the video surveillance system, it is systematically effective and quickly captures the position and motion of a moving object at high speed.

Additionally, at this time, such a capture method is performed in parallel with the traffic analysis operation below, for example.

The specific operation is as follows.

First, image analysis is performed by collecting surveillance images from a camera device installed for each surveillance area.

To this end, firstly, during video surveillance, surveillance images are collected for each camera device installed in each of a plurality of different surveillance areas (S401).

Then, based on the collected surveillance images, for example, traffic environment information is converted into big data for each camera device (S406), and it learns based on preset deep learning (S407).

At this time, unlike the intelligence used in the existing CCTV display and alarm, here, we do not set up virtual areas and lines to satisfy specific conditions within the video, but analyze rules, big data patterns, and movement patterns. do.

Such rule analysis is complexly performed according to moving object information (person, vehicle, etc.), purpose-specific information (child protection, crime prevention, traffic, etc.), location-specific information (specific monitoring area), and time-specific information (schedule).

And, the big data pattern analysis has a lot of information about the moving object, and creates a data distribution using data such as moving object classification (person, car, etc.), area, date, time, etc. among the attribute values of the moving object. And by specifying the activity area of the moving object within the screen, the area where the moving object appears and the area where the moving object does not appear is distinguished, and the user sets whether to monitor the specified data group (normal or abnormal).

In addition, the movement pattern analysis is performed by tracking the moving object and using the movement pattern and speed pattern of the moving object to analyze the usual movement and speed of the moving object, or to set whether to detect it (normal or abnormal).

Therefore, by applying the learned deep learning-based learning result, the image is analyzed according to a number of different environmental conditions, for example, by time period, by weather, by temperature, etc. (S408).

Therefore, by applying artificial intelligence image analysis technology through this, image analysis is systematic and reliable, and appropriate image analysis and operation are performed according to changes in the environment.

5 is a diagram for explaining in more detail an operation of extracting moving object metadata according to an embodiment applied to a method of capturing a position and motion of a high-speed moving object in an image monitoring system according to an embodiment.

As shown in FIG. 5 , in the moving object metadata extraction operation according to an embodiment, first, moving object extraction is performed.

For this purpose, the foreground background is first separated. In more detail, first, a background image is generated by accumulating continuously input images. Then, a pixel with a difference is calculated by comparing the input image with the background image. Then, a region having a difference value equal to or greater than a threshold is output as a foreground image.

Next, noise removal, moving object extraction, and moving object combination are performed. Specifically, noise in the foreground image is removed through a morphology filter. Then, the contour of the moving object feature is extracted from the noise-removed foreground image. Then, the feature of the moving object is extracted by combining the contour of the minimum size or smaller with the adjacent contour and processing it as a single moving object.

On the other hand, in addition to this, multi-moving object extraction and correction are performed.

Specifically, KCF (Kernelized Correlation Filters) constitutes a new tracking framework that uses the characteristics of a circular matrix to improve processing speed. And, in this case, the KCF Tracker tracks the moving object by using the color feature to extract the characteristic of the moving object that is strong against changes in brightness.

Next, the area and position of the moving object are corrected by comparing the moving object extraction result and the KCF Tracker's prediction result.

Then, the moving object is corrected by setting the correction weight to the moving object extraction result and the KCF Tracker prediction result, for example, 4 to 1. For reference, then, the operation according to FIG. 6 below is performed.

6 is a diagram for explaining a CNN-based object classification operation according to an embodiment applied to a method for capturing the position and motion of a high-speed moving object in an image monitoring system according to an embodiment.

As shown in FIG. 6 , the CNN-based object classification operation according to an embodiment is CNN-based with a basic structure of a feature extraction network and a classification network when moving object metadata is extracted. By performing moving object classification of , deep learning-based moving object classification is performed.

Specifically, in the CNN-based moving object classification operation according to an embodiment, when a surveillance image is input as described above, by extracting the moving object metadata from the input surveillance image, deep learning-based image analysis basic data is obtained do.

And, when the moving object metadata is extracted, the CNN-based moving object is classified by a plurality of different moving objects from the extracted moving object metadata, and thus, the moving object is classified according to the moving object based on deep learning.

Additionally, the CNN-based moving object classification operation according to an embodiment classifies an image in which a moving object appears and generates metadata.

And, it uses CNN to classify moving objects, such as vehicles, by constructing and learning with Caffe, a deep learning framework. Then, metadata of each moving object including general information such as color and time of each moving object of the vehicle is generated.

7 and 8 are diagrams for explaining in more detail an anomaly detection operation applied to a method of capturing a position and motion of a high-speed moving object in an image monitoring system according to an embodiment.

Specifically, FIG. 7 is a diagram illustrating a rule analysis operation according to such an abnormality detection operation, and FIG. 8 is a diagram illustrating a big data pattern analysis operation according to such an abnormality detection operation.

7 and 8 , in the anomaly detection operation according to an exemplary embodiment, multi-targets are first tracked, and then image analysis is performed based on a rule, a big data pattern, and a motion pattern.

In other words, unlike the existing intelligent CCTV display and alarm, there is no setting of virtual area, line, etc. to satisfy specific conditions within the video. In the embodiment, the analysis proceeds centering on rules, big data patterns, and movement patterns.

For example, whether the moving object corresponding to the multi-target is abnormal is detected by a preset object abnormal pattern format that detects an abnormal pattern based on an image distribution map for each moving object.

As a specific example, the area of the image in which the moving object appears is divided into 25*25 equal parts (625 block). And, if the moving object corresponds to at least a part of the matching area, the appearance area is accumulated. Then, using the accumulated data, an overlay is displayed centered on a frequently appearing area (heat map). So, when a moving object with similar properties appears in an area other than the accumulated area, it is notified. And, it notifies when the movement of the moving object is not normal. For example, if a car suddenly enters a part where only people have moved, or a part where the pattern in which a person or a car moves is suddenly changed, it will notify you.

This rule analysis is a complex prioritization of moving object information (person, vehicle, etc.), information by purpose (child protection, crime prevention, traffic, etc.), information by location (specific monitoring area), and information by time (schedule). way.

And, in big data pattern analysis, the metadata of the analyzed image has a lot of information about the moving object, and among the attribute values of the moving object, data such as classification (person, car, etc.), area, date, time, etc. Write the distribution. And, by specifying the activity area of the moving object in the screen, the area where the moving object appears and the area where the moving object does not appear is divided, and the user sets whether to monitor the specified data group (normal or abnormal).

In addition, the movement pattern analysis is performed by tracking the moving object in the image and setting whether to detect it (normal or abnormal) by analyzing the movement and speed of the moving object using the unique movement pattern and speed pattern of the moving object.

100: camera device 200: central control center
110: camera module 120: DSP unit
130: sensor unit 140: PLC controller

Claims (10)

In the method of capturing the position and motion of a moving object by a central control center,
A first step of collecting surveillance images for each camera device installed in each of a plurality of different surveillance areas during video surveillance;
a second step of extracting preset feature metadata of a moving object from the collected surveillance image;
a third step of extracting features of a moving object from a preset AI format for extracting feature values of an image patch according to the extracted feature metadata of the moving object;
a fourth step of recognizing a moving object according to a preset AI format for estimating the size and position of the entire moving object from the extracted features of the moving object; and
a fifth step of tracking the position and motion of each recognized moving object; A method for capturing the position and motion of a high-speed moving object in an image surveillance system, comprising: The method according to claim 1,
Between the fourth step and the fifth step,
Object 2D region division of the recognized moving object by the object 2D region division deep learning model obtained by calculating deep learning parameters for the object region through deep learning for 2D object region division from only a plurality of different RGB images in advance performing a logical AND with the depth information, and performing a 3D object region division of the moving object by applying a perspective transformation to the depth information of the moving object; Position and motion capture method of a high-speed moving object in the video surveillance system, characterized in that it further comprises a. The method according to claim 1,
The second step is
a 2-1 step of accumulating images continuously input from the collected surveillance images to generate a background image;
a step 2-2 of comparing the input image with the background image to calculate a pixel having a difference by a preset value;
step 2-3 of outputting a region having the difference value equal to or greater than a threshold value as a foreground image;
a step 2-4 of removing noise from the foreground image by removing it through a morphology filter;
Steps 2-5 of extracting the outline of the moving object feature from the noise-removed foreground image;
Steps 2-6 of combining an outline of a preset minimum size or less with an adjacent outline and processing it as a single object;
a second to seventh step of tracking the single object as a feature of a moving object that is strong against a change in brightness by using a color feature from a KCF tracker composed of a tracking framework using a feature of a circular matrix;
a 2-8 step of compensating the area and position of the moving object by comparing the moving object extraction result with the prediction result of the KCF tracker;
a 2-9 step of correcting the moving object by setting the correction weight to a preset N to 1 result of object extraction and KCF tracker prediction;
Steps 2-10 of classifying a moving object by constructing and learning a deep learning framework using CNN; and
a step 2-11 of extracting by generating and extracting feature metadata in the classified moving object; A method for capturing the position and motion of a high-speed moving object in an image surveillance system, comprising: 4. The method according to claim 3,
Steps 2-10 are
A second method to analyze big data patterns by creating data distribution by metadata of moving objects including classification, area, and time data of the moving object to be controlled, and specifying the activity area of the moving object in the screen and setting whether to monitor it -10-1 step; and
After analyzing the big data pattern, a second to analyze the movement pattern by tracking the moving object in the surveillance image and analyzing the movement and speed of the always moving object according to the movement pattern and speed pattern of the moving object and setting whether to detect it -10-2 steps; A method for capturing the position and motion of a high-speed moving object in an image surveillance system, comprising: The method according to claim 1,
The fourth step is
the AI format is obtained by estimating and combining the overall size and position of each moving object for a plurality of different moving object features, thereby estimating the size and position of the entire moving object; A method of capturing the position and motion of a high-speed moving object in an image surveillance system, characterized in that The method according to claim 1,
After the fifth step,
a sixth step of detecting whether the moving object is abnormal according to a preset moving object abnormal pattern format for detecting an abnormal pattern based on an image distribution diagram for each moving object; Position and motion capture method of a high-speed moving object in the video surveillance system, characterized in that it further comprises a. 7. The method of claim 6,
After the sixth step,
a seventh step of notifying an event by assigning different priorities to each object after the abnormal pattern is detected, taking into account moving object information, information by purpose, information by location, and information by time period; Position and motion capture method of a high-speed moving object in the video surveillance system, characterized in that it further comprises a. 8. The method of claim 7,
the moving object information includes a person, a bicycle, a two-wheeled vehicle, and a vehicle, the information by purpose includes child protection, crime prevention, and traffic, the information for each location is a specific monitoring area, and the information for each time period is a schedule; A method of capturing the position and motion of a high-speed moving object in an image surveillance system, characterized in that The method according to claim 1,
the camera device
a camera module for capturing a surveillance image of a preset surveillance area;
DSP unit for digital signal processing of the captured image;
a sensor unit for sensing surrounding environment information of the monitoring area;
The result processed by the DSP unit is provided to the central control sensor registered in advance, and information about the surrounding environment of the monitoring area sensed by the sensor unit is integrated I/O processing and provided to the central control center, PLC controller that transmits to a pre-registered control target; contains,
The PLC controller is
providing an IOT module of its own to provide the processing result of the DSP provided by the IOT module and information about the surrounding environment of the monitoring area to the central control center; A method of capturing the position and motion of a high-speed moving object in an image surveillance system, characterized in that 10. The method of claim 9,
The IOT module is
having its own TTS engine to provide a voice alarm according to voice information corresponding to each surrounding environment information of the monitoring area; A method of capturing the position and motion of a high-speed moving object in an image surveillance system, characterized in that

Images