KR20220162351A - Intelligent CCTV And Surveillance System Using It - Google Patents

Abstract

An intelligent CCTV according to an embodiment of the present invention for performing image processing such as object detection in image data captured by a CCTV includes: a photographing part that photographs a target point and generates image data; a processing part that is set in at least one of a face recognition mode, an object recognition mode, a people counting mode, a fire detection mode, a license plate detection mode, and a motion detection mode to process the image data; and a communication part that transmits the processed image data to a control server. The processing part comprises a GPU module detachably coupled to the photographing part.

Description

Intelligent CCTV and Surveillance System Using It {Intelligent CCTV And Surveillance System Using It}

The present invention relates to an intelligent CCTV.

Unless otherwise indicated herein, material described in this section is not prior art to the claims in this application, and it is not admitted that the material described in this section is prior art.

A CCTV surveillance system is applied to various fields for purposes such as facility management, disaster response, method, and traffic management. As of 2013, the number of CCTV cameras installed mainly in public institutions is approximately 65,000 and is continuously increasing. With the increase of CCTV cameras, the number of images that a controller must monitor in a limited space is increasing. In general, about 20 to 50 cameras are suitable for one controller to monitor. In addition, when the controller controls, the monitoring ability decreases by about 45% after 12 minutes and by about 95% after 22 minutes. Therefore, in order to operate an efficient CCTV control system, it is necessary to introduce an intelligent CCTV system based on image understanding and recognition technology.

An intelligent CCTV system requires complex video processing technologies such as object detection, tracking, and event detection. Object detection mainly requires a pedestrian or vehicle detection technique, and depending on the situation, a flame or smoke detection technique is required.

In the case of a conventional intelligent CCTV, image data captured through the CCTV is transmitted to a control server, and image processing is performed in the control server. Accordingly, there is a problem in that a large amount of data processing is required in the control server, which increases server construction cost or maintenance cost.

In addition, in the control server, there was a problem that the operation processing speed was slow as the image processing of multiple CCTVs was performed instead of one CCTV.

An object of the present invention is to provide an intelligent CCTV capable of performing image processing such as object detection in image data captured by the CCTV.

In addition, an object of the present invention is to provide an intelligent CCTV capable of performing image processing in a CCTV stage by mounting a separate NVIDIA GPU module in the CCTV.

An intelligent CCTV according to an embodiment of the present invention includes a photographing unit for generating image data by photographing a target point; and a processing unit set to at least one of a face recognition mode, an object recognition mode, a people counting mode, a fire detection mode, a license plate detection mode, and a motion detection mode to process the image data; It includes a communication unit for transmitting the processed image data to a control server, and the processing unit is characterized in that it consists of a GPU module detachably coupled to the photographing unit.

According to a preferred feature of the intelligent CCTV according to an embodiment of the present invention, the processing unit learns a plurality of face images with a predetermined learning algorithm to generate a face recognition model for extracting facial feature points, and generates a plurality of human images. A people counting model that measures the number of people is created by learning with a predetermined learning algorithm, and a plurality of object images collected for each object including vehicles, people, and animals are learned with a predetermined learning algorithm to extract the characteristics of the object Create an object recognition model, create a fire recognition model by learning a plurality of smoke and fire images with a predetermined learning algorithm, and create a license plate recognition model that extracts a vehicle number by learning a plurality of license plate images with a predetermined learning algorithm and a modeling unit configured to generate a motion detection model by learning a plurality of motion images classified for each motion of the object using a predetermined learning algorithm.

According to a preferred feature of the intelligent CCTV according to an embodiment of the present invention, the face recognition model, the fire recognition model, and the license plate recognition model are learned with the yolo v4 algorithm, the object recognition model is learned with the hog algorithm, and the people counting The model is trained with the cnn algorithm, and the motion detection model is trained with the Mean-Shift and CAMShift algorithms.

According to a preferred feature of the intelligent CCTV according to an embodiment of the present invention, the processing unit, when the target point is a security zone, is set to the face recognition mode, inputs the image data to a face recognition model, and inputs the target point Extracts facial feature point information photographed in and determines whether the extracted feature point information matches a plurality of facial feature point information accessible to the previously stored security zone, and if not matched, bounding the image data with a predetermined color for the person concerned mark the box

According to a preferred feature of the intelligent CCTV according to an embodiment of the present invention, the processing unit stores the facial feature point information of the wanted person and the missing person extracted by inputting the wanted person image and the missing person image into a face recognition model, and the face Image data set in recognition mode and photographed at the target point is input to the face recognition model to extract feature point information of faces included in the image data, and extract feature point information and feature point information of the faces of the wanted person and missing persons. If the match is determined by comparing with the matched personal information, the matching personal information is transmitted to the control server.

According to a preferred feature of the intelligent CCTV according to an embodiment of the present invention, the processing unit, when the target point is a no-parking zone, is set to the license plate detection mode, and inputs the image data to a license plate extraction model to detect the vehicle's The license plate is extracted, and license plate information is generated and transmitted to the control server.

Since the present invention can perform image processing such as object detection in video data captured by CCTV, it is possible to efficiently monitor and supervise without a controller, and reliability is improved because errors due to fatigue of the controller do not occur. and accuracy is improved.

In addition, since the present invention can perform image processing in the CCTV stage by installing a separate NVIDIA GPU module in the CCTV, not only can the construction cost of the control server be lowered, but also the NVIDIA GPU module can be installed in the existing CCTV. It has the effect of reducing the cost of constructing a CCTV system.

In addition, since the image processing is performed in the CCTV stage of the present invention, even if the number of CCTVs is increased, the operation processing speed is not reduced by processing the image data taken by the plurality of CCTVs in the control server as in the prior art. In this case, even if a control server is defective, only the server to receive video data needs to be replaced, so there is an effect of improving the defect recovery speed and server stability.

1 is a diagram showing a monitoring system using an intelligent cctv according to an embodiment of the present invention.
Figure 2 is a diagram showing the configuration of an intelligent CCTV (200) according to an embodiment of the present invention.
3 is a diagram showing the configuration of the processing unit 220 according to an embodiment of the present invention.
4 is an example showing that each model performs image processing on image data.

In this specification, it should be noted that in adding reference numerals to components of each drawing, the same components have the same numbers as much as possible, even if they are displayed on different drawings.

Meanwhile, the meaning of terms described in this specification should be understood as follows. Singular expressions should be understood to include plural expressions unless the context clearly defines otherwise, and terms such as “first” and “second” are used to distinguish one component from another, The scope of rights should not be limited by these terms. It should be understood that terms such as "comprise" or "having" do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

The term “at least one” should be understood to include all possible combinations from one or more related items. For example, "at least one of the first item, the second item, and the third item" means the first item, the second item, and the third item as well as each of the first item, the second item, and the third item. It means a combination of all items that can be presented from two or more of them.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

A surveillance system using an intelligent CCTV according to the present invention can generate image data and perform face recognition, object recognition, people counting, smoke detection, license plate detection, and motion detection on the image data.

In particular, the surveillance system using the intelligent CCTV according to the present invention builds an edge computing environment and performs image data processing performed in the conventional control server on the client side to minimize the computational load of the control server, as well as To minimize construction costs.

1 is a diagram showing a monitoring system using an intelligent cctv according to an embodiment of the present invention. As shown in FIG. 1, the monitoring system 100 using an intelligent CCTV according to the present invention includes an intelligent CCTV 200 and a control server 300.

Intelligent CCTV (200) according to the present invention generates image data, processes the image data and transmits it to the control server (300). Specifically, the intelligent CCTV (200) generates image data by photographing a target point. The intelligent CCTV (200) is set to at least one mode among face recognition mode, object recognition mode, people counting mode, fire detection mode, license plate detection mode, and motion detection mode, and processes the generated image data to the control server 300. send.

Hereinafter, the intelligent CCTV 200 according to the present invention will be described in more detail with reference to FIG. 2 .

Figure 2 is a diagram showing the configuration of an intelligent CCTV (200) according to an embodiment of the present invention. As shown in Figure 2, the intelligent CCTV (200) includes a photographing unit 210, a processing unit 220, and a communication unit (230).

The photographing unit 210 according to the present invention generates image data by photographing a target point.

For example, the photographing unit 210 may be configured as a camera. For example, the photographing unit 210 has an effective pixel of 1920 x 1080 (FHD) or more, a signal-to-noise ratio of 50dB or more, an operating environment of -30°C to +60°C, and a power specification of DC 12V. It could be a camera. However, this is just one example, and the photographing unit 210 may be a camera that does not satisfy at least one of the above or a device for photographing other than a camera, but is not limited thereto.

The photographing unit 210 transfers the generated image data to the processing unit 220 .

The processing unit 220 processes image data generated by the photographing unit 210 .

To this end, the processing unit 220 according to the present invention may be configured as a GPU module. For example, the GPU module may consist of an NVIDIA Jetson GPU module. In particular, in order to build an expensive intelligent CCTV, it costs more than 10 million won per CCTV, but the present invention can drastically reduce the cost by building the processing unit 220 with a separate NVIDIA Jetson GPU module.

In one embodiment, the processing unit 220 may be detachably coupled to the photographing unit 210 . In general, an intelligent CCTV is built after demolishing an existing CCTV, but the present invention may be constructed as an intelligent CCTV 200 by combining an NVIDIA Jetson GPU module composed of a processing unit 220 with an existing general CCTV.

In one embodiment, the NVIDIA Jetson GPU module composed of the processing unit according to the present invention, the GPU has specifications of NVIDIA Volta architecture with 384 NVIDIA CUDA® cores and 48 Tensor cores or more, and the CPU has 6-core NVIDIA Carmel ARM®v8 .2 It has specifications of 64-bit CPU, 6 MB L2 + 4 MB L3 or more, memory has specifications of 8 GB 128-bit LPDDR4x @ 51.2 GB/s or more, and storage can use micro SD.

For example, in the NVIDIA Jetson GPU module, the GPU consists of 512-core Volta GPU with Tensor cores, the CPU consists of 8-core ARM v8.2 64-bit CPU, 8MB L2, 4MB L3, and the memory consists of 32GB 256-Bit It is composed of LPDDR4x/ 137GB/s, and Storage can be composed of 32GB emmc 5.1.

Accordingly, the present invention has an effect of reducing separate removal costs or overall costs.

In one embodiment, the processing unit 220 is set to at least one mode of face recognition mode, object recognition mode, people counting mode, fire detection mode, license plate detection mode, and motion detection mode to process image data. .

When the face recognition mode is set, the processing unit 220 may extract feature point information from a face included in the image data and compare the extracted feature point information with a plurality of previously stored feature point information. The processing unit 220 may transmit information about a person mapped to the corresponding feature point information to the control server 300 when there is minutiae point information identical to the minutiae point information extracted as a result of the comparison.

When set to the object recognition mode, the processing unit 220 may extract feature information of an object from an object included in image data and compare the extracted feature information with a plurality of pieces of pre-stored feature information. The processing unit 220 may transmit the information of the object mapped to the characteristic information to the control server 300 when there is characteristic information identical to the characteristic information extracted as a result of the comparison.

When set to the people counting mode, the processing unit 220 may generate counting information by counting the number of people included in the image data, and transmit the counting information to the control server 300 .

When set to the fire detection mode, the processing unit 220 may determine whether or not smoke and flames included in the image data are included, and transmit fire occurrence information to the control server 300 when smoke and flames are included.

When the license plate detection mode is set, the processing unit 220 may extract the license plate number included in the image data and transfer the license plate number to the control server 300 .

When set to the motion detection mode, the processing unit 220 may determine whether objects included in the image data are moving, generate motion information, and transmit it to the control server 300 .

As such, the processing unit 220 according to the present invention may perform face recognition, object recognition, people counting, smoke detection, license plate detection, and motion detection.

In one embodiment, the processing unit 220 may set a mode according to a target point captured by the photographing unit 210 and may process image data according to the corresponding mode.

For example, the processing unit 210 may set the face recognition mode when the target point is a security zone. In this case, the processing unit 220 may extract facial feature point information from the image data, and determine whether the extracted feature point information matches a plurality of facial feature point information accessible to the previously stored security zone. The processing unit 220 may generate behavioral subject generation information and transmit it to the control server 300 when there is no facial feature point information matching the extracted facial feature point information. Also, the processing unit 220 may display a bounding box of a predetermined color on the corresponding face on the image data.

For example, when the target point is a no-parking zone, the processing unit 210 may be set to a license plate detection mode, generate license plate information by extracting a license plate of a vehicle from image data, and transmit the license plate information to the control server 300. have.

In one embodiment, the processing unit 220 may perform a search for a wanted person or a missing person when receiving a search request for a publicly wanted person or a missing person from the control server 300 . For example, the processing unit 220 is set to the face recognition mode, extracts feature point information of faces included in the image data from image data photographed at a target point, and extracts the extracted feature point information and feature points of the faces of wanted persons and missing persons. It can be compared with the information to determine whether it matches. When matched, the processing unit 220 may transmit matching personal information among wanted person and missing person personal information mapped to facial feature point information and stored to the control server 300 . In this case, the processing unit 220 may store facial feature point information of wanted persons and missing persons extracted from wanted person images and missing person images.

Hereinafter, the processing unit 220 according to the present invention will be described in more detail with reference to FIG. 3 .

3 is a diagram showing the configuration of the processing unit 220 according to an embodiment of the present invention.

The processing unit 220 according to the present invention includes a modeling unit 250, a face recognition model 252, a people counting model 253, an object recognition model 254, a fire recognition model 255, a license plate recognition model 256 , a motion detection model 257, a database 260, an input unit 265, and a mode selection unit 270.

The modeling unit 250 generates a face recognition model 252, a people counting model 253, an object recognition model 254, a fire recognition model 255, a license plate recognition model 256, and a motion detection model 257. can

The modeling unit 250 may generate each model using a predetermined learning algorithm using learning data.

The modeling unit 250 may generate a face recognition model 252 for extracting feature points of a face by learning a plurality of face images using a predetermined learning algorithm. Accordingly, when image data is input, the face recognition model 252 may extract facial feature points included in the image data.

In one embodiment, the face recognition model 252 may display a bounding box of a face on image data. For example, as shown in FIG. 4 , the face recognition model 253 may display a face as a bounding box on image data.

In one embodiment, a learning algorithm for generating the face recognition model 252 may use the YOLO V4 algorithm.

The modeling unit 250 may generate a people counting model 253 for measuring the number of people by learning a plurality of human images using a predetermined learning algorithm. Accordingly, the people counting model 253 may generate information on the number of people included in the image data when image data is input. Here, the number of people information means counting information.

In one embodiment, the people counting model 253 may display a person as a bounding box on image data. For example, as shown in FIG. 4 , the people counting model 253 may display a person as a bounding box on image data.

In one embodiment, a learning algorithm for generating the people counting model 253 may use a CNN algorithm.

The modeling unit 250 may generate an object recognition model 254 by learning a plurality of object images collected for each object including vehicles, people, and animals using a predetermined learning algorithm. Accordingly, the object recognition model 254 may generate object information by classifying vehicles, people, and animals among objects included in the image data. Here, vehicles, people, and animals are exemplified as objects, but this is only one example, and object information may be generated by classifying different objects according to the object image to be learned.

In one embodiment, the object recognition model 254 may create different bounding boxes for each vehicle, person, and animal on image data. For example, as shown in FIG. 4 , the fire recognition model 255 may display objects included in image data as bounding boxes.

In one embodiment, a learning algorithm for generating the object recognition model 254 may use a hog algorithm.

In one embodiment, the modeling unit 250 may generate a fire recognition model 255 by learning a plurality of smoke and flame images using a predetermined learning algorithm. Accordingly, the fire recognition model 255 may generate fire occurrence information by determining whether a fire occurs on the image data.

In one embodiment, the fire recognition model 255 may display smoke and flame as a bounding box on image data. For example, as shown in FIG. 4 , the fire recognition model 255 may display smoke and flame included in the image data as a bounding box.

In one embodiment, the modeling unit 250 may generate a license plate recognition model 256 for extracting a vehicle number by learning a plurality of license plate images using a predetermined learning algorithm. Accordingly, the license plate recognition model 256 may extract the vehicle number from the license plate included in the image data. For example, as shown in FIG. 4 , the license plate recognition model 256 may extract a vehicle number.

In one embodiment, the fire recognition model 255 may display smoke and flame as a bounding box on image data. For example, as shown in FIG. 4 , the fire recognition model 255 may display smoke and flame included in the image data as a bounding box.

In one embodiment, the learning algorithm for generating the fire recognition model 255 may use the YOLO V4 algorithm.

In an embodiment, the modeling unit 250 may generate the motion detection model 256 by learning a plurality of motion images classified for each motion of an object using a predetermined learning algorithm. Accordingly, the motion detection model 256 may generate information about an object whose motion occurs on image data.

In one embodiment, the motion detection model 256 may display an object whose motion is detected on image data as a bounding box. For example, as shown in FIG. 4 , the motion detection model 256 may display an object whose motion is detected as a bounding box on image data.

In one embodiment, a learning algorithm for generating the motion detection model 256 may use a mean-shift and CAMShift algorithm.

The database 260 stores a plurality of learning data. Here, the learning data includes a face image, an object image, a smoke image, a flame image, a license plate image, and an action image. In addition, the database 260 may store facial feature point information of wanted persons and missing persons extracted by inputting wanted person images and missing person images to a face recognition model. In addition, the database 260 may store information on a plurality of facial feature points accessible to the security zone.

The input unit 265 includes a face recognition model 252, a people counting model 253, an object recognition model 254, a fire recognition model 255, a license plate recognition model 256, and a motion detection model 257 according to the set mode. Image data may be input to at least one of the models.

The mode selection unit 270 sets the processing unit 220 to at least one of a face recognition mode, an object recognition mode, a people counting mode, a fire detection mode, a license plate detection mode, and a motion detection mode. As such, in the present invention, one or more modes can be selected and set. For example, the present invention may perform smoke detection or license plate detection while performing face recognition.

In one embodiment, the mode selector 270 may select a mode according to the target point. For example, the mode selector 270 may select a face recognition mode when the target point is a security zone. In addition, the mode selector 270 may select a license plate detection mode when the target point is a no-parking area.

In one embodiment, the mode selector 270 may select a mode according to a request of the control server 300 . For example, the mode selector 270 may select a face recognition mode when a search request for a wanted person or a missing person is received from the control server 300 .

On the other hand, the communication unit 230 performs transmission and reception with the control server 300 .

The communication unit 230 transmits the image data processed by the processing unit 220 to the control server 300 . In addition, the communication unit 230 is the control server 300, personal information, fire occurrence information, behavior award winner information, number of people information, object information, movement information, person information, license plate information, etc. generated by the processing unit 220 Information is transmitted to the control server (300). Also, the communication unit 230 may receive a request for selecting a mode from the control server 300 . For example, the communication unit 230 may receive a search request for a wanted person or a missing person from the control server 300 .

In one embodiment, the communication unit 230 may transmit/receive data with the control server 300 by receiving an external IP address. Accordingly, different IPs may be assigned to each intelligent CCTV 100 in the communication unit 230 .

On the other hand, the control server 300 may receive image data captured and processed from the intelligent cctv (200). In this way, the present invention has an effect that the control server 300 receives the image data processed by the intelligent cctv 200, thereby reducing the load of the server and improving the processing speed.

In addition, the control server 300 is to receive generated information such as personal information, fire occurrence information, behavioral disability generation information, number of people information, object information, movement information, person information, license plate information, etc. from the intelligent cctv (200). can

In addition, the control server 300 may transmit a request for selecting a mode to the intelligent cctv (200).

Those skilled in the art to which the present invention pertains will be able to understand that the above-described present invention may be embodied in other specific forms without changing its technical spirit or essential features. When the present invention is implemented by being divided into a plurality of programs, each program may be loaded on different media.

Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting. The scope of the present invention is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included in the scope of the present invention. do.

Claims (6)

a photographing unit generating image data by photographing a target point;
a processing unit set to at least one of a face recognition mode, an object recognition mode, a people counting mode, a fire detection mode, a license plate detection mode, and a motion detection mode to process the image data; and
A communication unit for transmitting the processed image data to a control server;
The processing unit,
Intelligent CCTV, characterized in that consisting of a GPU module detachably coupled to the photographing unit.
According to claim 1,
The processing unit,
Learning a plurality of face images with a predetermined learning algorithm to create a face recognition model that extracts facial feature points, learning a plurality of human images with a predetermined learning algorithm to create a people counting model that measures the number of people, A plurality of object images collected for each object, including vehicles, people, and animals, are learned with a predetermined learning algorithm to create an object recognition model that extracts object features, and a plurality of smoke and fire images are learned with a predetermined learning algorithm. to create a fire recognition model, create a license plate recognition model that extracts a vehicle number by learning a plurality of license plate images with a predetermined learning algorithm, and learn a plurality of motion images classified by object motion with a predetermined learning algorithm Intelligent CCTV characterized in that it comprises a modeling unit for generating a motion detection model.
According to claim 2,
The face recognition model, fire recognition model, and license plate recognition model are learned with the yolo v4 algorithm,
The object recognition model is learned with a hog algorithm,
The people counting model is learned with the cnn algorithm,
Intelligent CCTV, characterized in that the motion detection model is learned by Mean-Shift and CAMShift algorithms.
According to claim 1,
The processing unit,
When the target point is a security zone, the face recognition mode is set, the image data is input to a face recognition model to extract facial feature point information captured at the target point, and the extracted feature point information and pre-stored security zone An intelligent CCTV characterized in that it determines whether or not it matches with a plurality of accessible facial feature points, and if it does not match, a bounding box of a predetermined color is displayed for the person concerned in the image data.
According to claim 1,
The processing unit,
Public wanted person images and missing person images are input to a face recognition model to store facial feature information of public wanted people and missing people extracted;
The image data set in the face recognition mode and photographed at the target point is input to the face recognition model to extract feature point information of the faces included in the image data, and extract the extracted feature point information and the faces of the wanted person and the missing person. An intelligent CCTV characterized in that it compares feature point information to determine whether they match, and if they match, transmits the matching personal information to the control server.
According to claim 1,
The processing unit,
When the target point is a no-parking zone, the license plate detection mode is set, the license plate of the vehicle is extracted by inputting the image data to the license plate extraction model, and license plate information is generated and transmitted to the control server. Intelligent CCTV.

Images