KR101793636B1 - Image processing system - Google Patents

Abstract

An integrated intelligent CCTV image processing system according to the present invention comprises: at least one CCTV camera which captures a monitoring area to obtain raw image data; an integrated collecting server which collects and stores the raw image data captured by the CCTV cameras; and a central management center which receives the raw image data via a network, analyzes and processes image data classified by object recorded in the raw image data and obtains an event. The CCTV cameras comprise at least one among a motion detector camera which has a moving object detecting function in the monitoring area, a far infrared camera which senses heats, a high definition RGB camera which captures images of number plates, a camera on which a radar with more reliability is attached, a pan/tilt camera for performing an automatic tracking, and a network camera for performing IP communications. The present invention relates to an integrated intelligent CCTV image processing system which uses an image object detecting function to sense fire, breaking in, object sensing, number plate recognition, loitering and abnormality from raw data of the captured image.

Description

Integrated Intelligent CCTV Image Processing System {IMAGE PROCESSING SYSTEM}

The present invention relates to an integrated intelligent CCTV image processing system capable of detecting fire, intrusion, object detection, car number recognition, wandering and abnormal behavior from raw data of a photographed image using a video object detection function.

In recent years, convergence system of accident detection monitoring has been spotlighted by combining with information and communication technology, and it is spreading throughout the industry. As the installation of CCTV (Closed Circuit Television) increases, interest in intelligent image analysis technology for efficient monitoring is increasing.

Intelligent image analysis technology analyzes the information of images and automatically detects abnormal behavior and sends alarms to the administrator. It helps prevent accidents in advance and promptly responds to accidents to reduce damages. However, the conventional intelligent image analysis technology has focused on one event, and it has been difficult to perform integrated management only for object tracking and license plate recognition. In addition, when the image data captured in the intelligent image analysis is input to the control device, it can not be managed efficiently, and when a plurality of event images for recent event monitoring occur, it has been difficult to perform integrated processing management.

In the conventional CCTV, only the motion detection and tracking corresponding to the one-to-one correspondence to the object tracking are performed from the photographed image, and from the raw data which is the captured image data, the fire, intrusion, object detection, car number recognition, Intelligent analysis was not done. Also, since it is difficult to process the combined image data, it is used as one intelligent analysis server, but the intelligent image analysis can not be performed quickly and efficiently based on the captured low image data of various parameters.

Japanese Patent Application Laid-Open No. 10-2008-0072454 Patent Registration No. 10-1130963 Patent Registration No. 10-1180887 Patent Registration No. 10-169057

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems of the prior art, and it is an object of the present invention to provide an intelligent image analysis technology capable of detecting an abnormal behavior that an intelligent image analysis technology can detect, An object of the present invention is to provide an integrated intelligent CCTV image processing system capable of discriminating and detecting roaming detection.

In addition, the intelligent image analysis system is integrated with the intelligent image analysis system, which is classified automatically according to the change of the intelligent image analysis structure, which can detect the human behavior, the behavior based abnormality detection, the event detection in the event environment, An object of the present invention is to provide an integrated and intelligent integrated CCTV image processing system capable of selectively performing analysis processing by an administrator in the integrated central control center according to the configuration of the priority processing unit.

In addition, it adopts the configuration of object classifier among the configurations in the integrated central control center, and can intelligently classify and manage it in advance, and can use the integrated intelligent CCTV image processing system The goal is to provide.

To provide an integrated intelligent CCTV image processing system capable of distinguishing an abnormal event that can be detected, that is, an event, such as fire detection, flame detection, intrusion detection, object detection, car number detection, It has its purpose.

In addition, in relation to event detection, an integrated intelligent image analysis system that automatically classifies objects according to changes in human behavior recognition, behavior-based abnormal phenomenon detection, event detection in an event environment, And an object of the present invention is to provide an integrated and intelligent integrated CCTV image processing system capable of selectively performing analysis processing by an administrator in the integrated central control center according to the configuration of the priority processing unit when data is input.

The problems to be solved by the present invention are not limited to the above-mentioned problems. Other technical subjects not mentioned will be apparent to those skilled in the art from the following description.

The integrated intelligent CCTV image processing system according to the present invention includes at least one CCTV camera that captures a surveillance region to acquire raw image data, an integrated collection server that collects and stores raw image data captured from the CCTV cameras, And a central control center for receiving the low image data via the central processing unit and analyzing the image data classified by the object recorded in the low image data to acquire an event, A detector camera, an infrared camera with thermal sensation, a high-definition RGB camera for picking up license plate images, a camera with more reliable radar, and a pan tilt camera for performing automatic tracking, and a network camera capable of IP communication Wherein the object is at least one of fire, Vehicle, and the event may be fire detection, intrusion detection, object detection, car number recognition, roaming, and anomaly detection.

The central control center according to the present invention includes a transceiver for receiving low image data picked up from a surveillance area, an object sorter for sorting low image data by objects, an image analyzer And a push notification unit for outputting a push notification signal in accordance with the priority order.

As an embodiment, fire detection is performed by converting an input image into an HSI color space, detecting a primary fire candidate area using a fire setting value based on a color range, a saturation range, and a brightness range that can appear in a fire, A motion vector for a candidate region is obtained and the directionality is analyzed to detect a secondary fire candidate region, a difference image of a secondary fire candidate region and a secondary fire candidate region of a plurality of adjacent frames is generated, A pixel having a pixel value equal to or higher than a reference value in the cumulative image may be detected as a fire region.

According to the present invention, it is possible to distinguish a plurality of regions, and to easily grasp the number of moving objects and movement trajectories in the corresponding region.

1 is an overall schematic diagram of an integrated intelligent CCTV image processing system in accordance with the present invention.
2 is a detailed configuration diagram of a central control center of the integrated intelligent CCTV image processing system according to the present invention.
3 is a detailed configuration diagram of the image analysis unit of the central control center of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the configuration and operation of an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is an overall schematic diagram of an integrated intelligent CCTV image processing system in accordance with the present invention.

The integrated intelligent CCTV image processing system according to the present invention includes a CCTV camera 100, an integrated collection server 200, a network 300, a central control center 400, and a related organization 500.

Referring to the drawings, a CCTV camera 100 captures a surveillance region and acquires raw image data. In this case, the CCTV camera may include a motion detector camera having a moving object detection function in a surveillance area, an infrared camera having a thermal sensation, a high-definition RGB camera imaging a license plate image, a camera with a more reliable radar, And a network camera capable of performing IP communication.

The raw image data picked up from the CCTV camera 100 is transmitted to the integrated collection server 200. The integrated collection server records and stores row image data. In addition, the integrated collection server may separately store and manage feature images by comparing previously stored images with currently captured row image data. The integrated collection server may be installed together with a CCTV camera, or may be installed in a separate management place. For example, in the case of an apartment, it can be installed in a parking room if there is a guard room or a parking facility.

The raw image data recorded in the integrated collection server is transmitted to the central control center 400 via the network 300. The network 300 may be an Internet IP, an Ethernet, a wireless WiFi, or the like, and may be constructed using a wired cable (including HDMI).

As another embodiment, the transmission of the image data can be realized by using a long distance transmission system. The long-distance transmission system transmits the low-loss, high-quality video data to the receiving end even in long-distance transmission by performing image interpolation and time delay correction of mutually loss compensated high-quality video data transmitted using wireless transmission in combination with the existing wired UTP cable, Provide images.

A long-distance transmission system of multi-channel high definition (HD) image data for time and image interpolation for CCTV receives a plurality of digital high definition video signals captured by a plurality of HD cameras, converts the received digital high definition video signals into a plurality of analog video signals, An HD junction box for serially transmitting the plurality of analog video signals according to a time change via a one-line coaxial cable (UTP cable) connection, An image signal receiver for demodulating the image signal to extract a plurality of analog image signals, an HD RX for converting the extracted analog signal into a digital high definition image signal, Quality video signal and interpolating the interpolation and delay time after the comparison and determination; It comprise DVR for storing the digital high definition video signal transducer and an output section for reproducing the digital high definition video signal.

In addition, the HD junction box may further include a second image transmitting unit, and the HD RX may include a second image receiving unit. The second image transmitting unit may receive the converted image signal to reduce the loss of the image signal, And a first gain differential amplifier for amplifying the image gain, and the image signal output from the second image transmitter can be transmitted to the second image receiver via the relay HUB.

The central control center 400 classifies the images according to categories through intelligent image analysis based on the raw image data.

2 is a detailed block diagram of the central control center of the integrated intelligent CCTV image processing system of the present invention.

The central control center includes a transmission / reception unit 410, an object classifier 420, an image analysis unit 430, a priority processing unit 440, an execution unit 450, a push notification unit 460, (470).

The transceiving unit 410 receives the raw image data captured in the monitoring area and transmits a control signal to the CCTV camera to drive the CCTV camera.

The object classifier 420 classifies the row image data by object. As an embodiment, the image stream or the raw image data for a fire, a person, a vehicle, and the like are classified and stored for each object, and classification is performed to quickly compare the classified image data with the accident image. As an embodiment, the object classifier can be conventionally stored in a simple manner, thereby allowing the administrator to classify the images and to automatically classify and record the objects from the captured image data.

The image analyzer 430 performs image analysis classified by objects. The analyzing unit analyzes and detects whether an event is generated for each object. The event may be fire detection, intrusion detection, object detection, car number recognition, wandering and abnormal behavior detection, and the like.

3 is a detailed configuration diagram of the image analysis unit of the central control center of the present invention.

The image analysis unit according to the present invention includes a fire and flame detection analyzer 431, an object detection analyzer 432, a car number recognition analyzer 433, and a roaming and anomaly detection analyzer 434.

The fire and flame detection analyzer 431 of the image analysis unit analyzes the thermal image of the infrared camera when continuous flame is generated in the low image data in the surveillance area. The fire and flame detection analyzer can simultaneously detect and analyze the color gamut of the flame itself and irregular motion vectors.

In the case of fire and flame detection, the first fire candidate region is set based on the color information, the second fire candidate region is set using the motion vector, and the cumulative image in which the difference images are accumulated is generated to detect the fire region can do.

As an embodiment, there is provided an image processing method comprising the steps of: converting an input image into an HSI color space, detecting a primary fire candidate area using a fire setting value based on a color range, a saturation range, A step of detecting a second fire candidate region by obtaining a motion vector for a candidate region and analyzing a directionality; generating a difference image of a second fire candidate region and a second fire candidate region of a plurality of adjacent frames; Accumulating a cumulative image; and detecting a pixel having a pixel value equal to or higher than a reference value in the cumulative image as a fire region.

Here, the fire setting values may be ranges for hue, saturation, and brightness, respectively. The fire setting value can be calculated by Equation (1) below.

≪ Formula 1 >

The step of detecting the secondary fire candidate region may include a step of obtaining a motion vector for the primary fire candidate region, a step of quantizing the motion vector obtained for the primary fire candidate region in the direction of n (natural number) And detecting a primary fire candidate region including a direction whose motion vector direction is not less than a reference value as a secondary fire candidate region using the quantized motion vectors for the fire candidate region.

The method of detecting a second fire candidate region is a method for overcoming the possibility of false detection in the case of detecting a fire candidate region using only color information. The CLG-TV (Combined Local-Global approach with Total Variation) The motion vector used is used.

The motion vector can be calculated by Equation (2) below.

&Quot; (2) "

Here, μ and v denote motion information in the x-axis and y-axis directions, p denotes an image patch, and Ω denotes an image domain. Also,? Represents the coefficient value, and w represents the initial value of the motion vector obtained by the Lucas Kanade method.

The r (μ, ν) represents the residual of the previous input image and the current input image. The residual can be defined as r (μ, ν) = (F (t) - F (t-1) + F , x) and F (t-1, y) represent differentiated images in the x-axis direction and the y-axis direction, respectively.

The step of accumulating the difference images to generate an accumulated image may include generating difference images between a second-order fire candidate region and a second-order fire candidate region of each of a plurality of adjacent frames, respectively, Thereby generating an accumulated image.

The object detection analyzer 432 detects invasion in the surveillance area, movement of the object, disappearance of the object, and the like. As an embodiment, an object is detected by analyzing a movement trajectory of a moving person in a sensing area. In addition, the object detection analyzer patterns the row image data to set an intrusion prevention zone on the basis of the ROI, and detects an intrusion when an object (person) is detected in the area.

The detection of the object is performed based on the type information and the feature information. The tracking of an object can be accomplished by tracking an object by performing a block-based matching algorithm on the basis of the extracted shape control points according to the occurrence of object occlusion, or by tracking a object by a weight-based block matching algorithm or a cyclic update- can do.

Specifically, the method includes generating a background image based on continuously input image frames, updating the background image according to whether the detected object is moving from the image frames, and determining whether there is a difference between the background image and the input image frame A step of extracting a shape control point (SCP) from the detected motion area, and a step of extracting a shape control point based on the extracted shape control points if no object occlusion occurs Based Block Matching Algorithm (CBMA) and periodic update based on block centering algorithm (BMA) based on block centering algorithm And a periodic-update-based Block Matching Algorithm (PBMA). It may include the step of tracking the sun object.

In the step of updating the background image, if the value obtained using Equation (3) below is smaller than a preset threshold value, the background image is updated using Equation (4) below.

&Quot; (3) "

(U, v) denotes horizontal and vertical coordinates, (mx, my) denotes a motion vector, and (p, q) denotes a horizontal and vertical size of an image block.

≪ Equation 4 &

Where B (t) represents the background at time t, I (t) represents the input image at time t, and σ represents the mixing ratio experimentally set in the range of 0 to 1.

This allows tracking of objects with only validated shape control points to be free from potential failure factors such as temporal spatial similarity between object and background, object deformation, occlusion, and so on. According to the present invention, intrusion in a surveillance area, movement of an object, disappearance of an object, and the like can be detected.

The car number recognition analyzer 433 recognizes the car number as an RGB camera, converts it to gray scale, and recognizes the car number through a gradient calculation process.

The license plate can recognize the characters from the license plate detection. The vehicle license plate detection method includes the steps of receiving a video image including a vehicle from an RGB camera, converting the video image into a gray scale, performing a gradient process on the video image converted into the gray scale, And applying the model.

The active contour model uses an energy minimization technique, and the energy function of the active contour model can be calculated according to Equation (5) below.

≪ Eq. 5 &

The coordinates of each vertex are defined as v (Si) = (x (Si), y (Si)), where Eint is the internal energy term and Eimage is the image image energy term And Eprior is the pre-constrained energy term.

In order to precisely detect the license plate, it is possible to detect the license plate by obtaining the extremum of the energy function using the energy function in the input image.

The wandering and anomaly detection analyzer 434 senses irregular wandering and performs pattern analysis when a specific object moves in an unspecified unspecified movement locus for a predetermined period of time within the sensing area.

In the case of detection of a roaming and an abnormal behavior, it is possible to detect an abnormal situation such as a wartime behavior or a roaming behavior that may occur in the monitored area based on the positional change of the moving object detected by the difference between the image frame and the background image, The unmanned surveillance system effectively performs the unmanned surveillance.

As an embodiment, the abnormal behavior detection may include generating a background image corresponding to the current image frame based on an optical flow detected between a current image frame and a previous image frame that temporally precedes the current image frame among a plurality of input image frames, A moving object detecting step of detecting a moving object from the current image frame by a difference between the background image and the current image frame; Detecting an abnormal situation that occurs as the moving object moves within a monitored area corresponding to the current image frame based on a moving trajectory, which is a change in position of the detected moving object, And the movement of the moving object And an abnormal behavior detection step of determining that the moving object is an object roaming in the monitored area if the number of times the trajectory passes exceeds a preset reference number, which is equal to or greater than a preset reference number.

A still object is generated based on a difference between the background image and the current image frame and a difference between the past background image, which is a background image generated in correspondence with the image frame preceding by the reference time preset in the current image frame, And if the position of the stop object is the same as a preset reference number of image frames that are temporally continuous up to the current image frame, the stop object can be estimated as a stop object. Such a neglected object estimates that the stationary object corresponds to an abandoned object or a fainted person based on the invariant moment value of the stationary object.

On the other hand, the row image data may include at least one or more different objects. The priority processing unit according to the present invention selects priorities when events such as fire detection, intrusion detection, object detection, car number recognition, wandering, and abnormal behavior detection are simultaneously detected from the classified raw image data, . The priority may be a preset priority. For example, the priority may be given in order of fire, intrusion, out-of-sight and abnormal behavior, and vehicle number. The priority is a reference for outputting a push notification signal to the relevant authority.

The execution unit 450 of the central control center controls the scratch direction of the camera according to the control of the administrator when intrusion or object detection is detected. As an embodiment, when an object is detected in a region of interest, monitoring may be performed to output a control signal to execute automatic tracking of the camera's imaging direction and the intruding object in real time.

Meanwhile, the central control center according to the present invention further includes a push notification unit 460. The push notification unit adds a content of low image data pattern analysis, an index search, a real-time warning, and an event generation to output a push notification signal to a related organization 500 such as a municipality related department, a police station, In addition, the push notification signal can selectively notify the system administrator via text message, audio notification, and blinking of the light.

The display unit 470 of the central control center according to the present invention displays an image photographed by a CCTV camera.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the present invention is not limited to the disclosed exemplary embodiments, but various changes and modifications may be made by those skilled in the art without departing from the scope of the present invention.

100: CCTV camera 200: Integrated collection server
300: Network 400: Central Control Center
410: Transmitting / receiving unit 420: Object classifier
430: Analyzer 440: Priority processor
450: Executing unit 460: Push notification unit
470: Display unit 500:

Claims (3)

At least one CCTV camera for capturing the surveillance region and acquiring low image data,
An integrated collection server for collecting and storing raw image data captured from the CCTV cameras,
And a central control center for receiving the low image data via the network and analyzing the image data classified by the objects recorded in the low image data to acquire an event,
The CCTV camera includes a motion detector camera having a moving object detection function in a surveillance area, an infrared camera having a thermal sensation, a high-definition RGB camera for sensing license plate images, a camera with more reliable radar, A tilt camera, a network camera capable of performing IP communication,
The object may be a fire, a person, or a vehicle, and the event may include fire detection, intrusion detection, object detection, car number recognition,
In the fire detection, an input image is converted into an HSI color space, and a primary fire candidate region is detected using a fire setting value based on a color range, a saturation range, and a brightness range that can appear in a fire, A second fire candidate region and a second fire candidate region of a plurality of adjacent frames are generated by analyzing a direction vector and a motion vector for the second fire candidate region, And detecting a pixel having a pixel value equal to or higher than a reference value in the cumulative image as a fire area,
In the detection of the secondary fire candidate region, the motion vector for the primary fire candidate region is obtained, and the motion vector obtained for the primary fire candidate region is quantized in the direction of n (natural number) And a primary fire candidate region including a direction in which a motion vector direction is greater than or equal to a reference value is detected as a secondary fire candidate region using the quantized motion vector results.
The method according to claim 1,
The central control center comprises a transceiver for receiving the low image data picked up from the surveillance area, an object classifier for classifying the low image data by objects, an image analyzer for performing an image analysis by classifying the objects, A priority processing unit for assigning a predetermined priority based on the acquired event, and a push notification unit for outputting a push notification signal according to the priority.
The method according to claim 1,
Wherein the motion vector is calculated by the following equation.
<Equation>

Here, μ and v denote motion information in the x-axis and y-axis directions, p denotes an image patch, and Ω denotes an image domain. Also,? Represents the coefficient value, and w represents the initial value of the motion vector obtained by the Lucas Kanade method. And r (mu, v) represents the residual of the previous input image and the current input image. The residual is defined as r (μ, ν) = (F (t) - F (t-1) + F (t, x) ) And F (t-1, y) represent images differentiated in the x-axis direction and the y-axis direction, respectively.

Images