KR101654267B1

KR101654267B1 - CCTV for recognizing number of vehicles and road security cloud system

Info

Publication number: KR101654267B1
Application number: KR1020160019404A
Authority: KR
Inventors: 최우민; 전민규
Original assignee: 주식회사 아프로시스템즈
Priority date: 2016-02-18
Filing date: 2016-02-18
Publication date: 2016-09-09

Abstract

The present invention relates to a road security cloud system for multiple CCTV number recognition, which comprises: a CCTV including a video photographing unit for photographing a vehicle, and a communication unit for transmitting the photographed video data; an integrated video server for receiving the video data from the communication unit in a remote place, and distributing and displaying the video data by converting the video data into a video image; a number recognition server including a video acquisition unit for receiving the video image from the integrated video server, a video process unit for determining a position of a vehicle number plate by receiving the video image from the video acquisition unit, and a number recognition unit (340) for extracting a number from a vehicle number plate area determined by the video process unit; and a number database for storing information including the number extracted by the number recognition unit.

Description

[0001] The present invention relates to a security cloud system for CCTV,

The present invention relates to a road method system, and more particularly to a system for performing number recognition of a photographed vehicle regardless of the use or model of the camera.

Conventional CCTVs are categorized by function and classified into CCTVs that can recognize numbers and CCTVs that do not have number recognition functions. CCTV used for speed control, illegal parking control, etc. has number recognition function, and road-only (installed outside the speed control on the road) and non-road dedicated CCTV installed in the distance have no number recognition function.

The camera used for overspeed or illegal parking is an essential function that includes the function of recognizing the number plate of the vehicle and extracting the number. However, CCTVs for road use only or non-road use do not have number recognition function, In order to recognize the license plate number, the number on the license plate is grasped by the naked eye while watching the photographed image.

The reason why it is difficult to give the number extraction function to the road-only or non-road-only CCTV is that the controller should be installed around the camera for number recognition and extraction, and the controller installed outside is vulnerable to dust, This is because it is difficult to maintain, and the installation cost is high.

On the other hand, in the case of non-road only, it is difficult to determine the position of the license plate because the environment is varied as well as the location of the vehicle is set as a road, a residential area, and the like.

In addition, in case of road use only, a lot of data processing is necessary to detect all the vehicles as images, and expensive equipments having a level of overspeed control should be provided separately.

However, in order to more effectively utilize CCTVs for road use and non-road use more effectively for crime prevention and criminal enforcement, it is urgently required to give a function of assigning a car number recognition and extraction function to these cameras.

An object of the present invention is to provide a roadside crime prevention system capable of recognizing and extracting a number plate of a vehicle from an image of a camera regardless of the use of the interception camera or the number plate.

According to an aspect of the present invention, there is provided a security cloud system for use as an identification number of a multi-type CCTV, including a vehicle detection unit for detecting entry of a vehicle,

An illumination and lens control unit for receiving a vehicle sensing signal from the vehicle sensing unit and controlling illumination and a lens,

A video photographing unit capable of photographing a vehicle, and

Two or more CCTVs including a communication unit capable of transmitting photographed video data;

An integrated video server for receiving video data from the communication unit at a remote location, converting the video data into a video image, distributing and displaying the video image;

An image acquisition unit for receiving a video image from the integrated video server,

An image processing unit for receiving the image image from the image acquisition unit and determining the position of the license plate;

A number recognition server including a number recognition unit for extracting a number from the license plate area determined by the image processing unit; And

And a number database for storing information including numbers extracted from the number recognizing unit.

The number database may further include at least one of the group consisting of the capturing CCTV, the shooting date and time, and the storage location of the moving image, which essentially include the number of the vehicle extracted by the number recognizing unit.

In addition, the CCTV may be configured as a non-road dedicated CCTV, and the other may be configured as a road-dedicated CCTV.

In addition,

The image sensing unit may receive the vehicle detection signal from the vehicle sensing unit and may include the identification markers to distinguish the image data from the other image data.

In addition,

And the communication unit transmits the image data when the vehicle is detected using the vehicle sensing unit to the integrated image server.

In addition, the number database may further include at least one of the group consisting of the number of the vehicle extracted from the number recognition unit, and the storage location of the photographing CCTV, the photographing date and time, and the moving image.

The roadside crime prevention system includes:

A number search terminal, and when a vehicle number is input to the number search terminal, the vehicle number is searched for from the number database, and a matching vehicle number is searched. If there is a matching vehicle number, the corresponding photographing CCTV, And displaying at least one of the storage locations of the moving picture.

The multi-type road security cloud system capable of recognizing the multi-type CCTV according to the present invention can recognize the number plate of the vehicle from the images photographed from all the cameras and extract the number irrespective of the type and use of the camera.

In addition, vehicle identification is performed by a server located at the center of the control center or the like, so that a field controller is unnecessary, which makes maintenance and repair easy.

In addition, the extracted number is composed of a number database, so that when searching the car number, it is possible to conveniently find the car number related image without manually viewing the image.

FIG. 1 is an overall configuration diagram of a crime prevention cloud system for identification, which is a multi-type CCTV number according to an embodiment of the present invention.
FIG. 2 is a CCTV block diagram of a crime prevention cloud system, which is a multi-type CCTV number according to an exemplary embodiment of the present invention.
FIG. 3 is a block diagram of a number recognition server and a number database of a security crime system for food, which is a number of a multi-type CCTV according to an exemplary embodiment of the present invention.
FIG. 4 is a diagram illustrating an example of performing a grouping of numbers from a collected image according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating an example of extracting a larger area of upper, lower, right, and left sides based on coordinates of a group of numbers according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating a result of performing a number recognition algorithm from a clustered-in-a-number area according to an embodiment of the present invention.
FIG. 7 is a view showing a car license plate image recognition process in an irregular position.
8 is a flowchart of a number recognition method according to the type of CCTV.
9 is a view showing a method of correcting the minutiae points A, B, C and D of the distorted license plate.
FIG. 10 is a view illustrating a result of performing a number search from a number database using a number search terminal of a security crime system for food, which is a number of multiple CCTVs according to an embodiment of the present invention.
FIG. 11 is a CCTV block diagram of a crime prevention cloud system, which is a multi-type CCTV number according to another embodiment of the present invention.
12 is a flowchart of a method for recognizing a number of multiple CCTVs according to an embodiment of the present invention.
FIG. 13 is a flowchart of a number CCTV number recognition step according to an embodiment of the present invention.

Before describing the present invention in detail, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the invention, which is defined solely by the appended claims. shall. All technical and scientific terms used herein have the same meaning as commonly understood to one of ordinary skill in the art unless otherwise stated.

Throughout the description and claims, unless the context requires otherwise, the word "comprise" is intended to include the stated article, step or group of articles, and steps; It is not intended to exclude an object, a step, or a group of objects, or a group of steps.

On the contrary, the various embodiments of the present invention can be combined with any other embodiments as long as there is no clear counterpoint. In particular, any feature that is indicated to be advantageous or advantageous may be combined with any other feature or feature that is indicated to be advantageous or advantageous.

Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is an overall configuration diagram of a crime prevention cloud system for identification, which is a multi-type CCTV number according to an embodiment of the present invention. According to an embodiment of the present invention, a road security cloud system capable of recognizing multiple CCTV numbers includes a CCTV (camera) 100, an integrated video server 200, a number recognition server 300, and a number database 400 .

FIG. 2 is a CCTV (100) block diagram of a crime prevention cloud system for identifying multiple CCTV numbers according to an embodiment of the present invention. According to this, the CCTV 100 can use all the CCTVs capable of capturing and transmitting images regardless of the type or usage. CCTV can be used for CCTVs with number recognition, although CCTVs that do not have their own vehicle identification number or on-site number recognition control are usually available.

The image capturing unit 110 continuously captures a vehicle, a person, and a facility, and transmits the image data to the integrated image server 200 through the communication unit 120. [

The integrated video server 200 receives video data from the communication unit 120 and converts the video data into a video image, and distributes and stores the video image to another server such as a control center and manages the video image to be displayed.

Also, the integrated video server 200 can receive and manage the car number recognized from the number recognition server 300 or the number database 400 to be described later.

Detailed descriptions of various functions for storing and distributing images, which are general functions of the integrated video server 200, will be omitted because they may obscure the gist of the present specification.

FIG. 3 is a block diagram of a number recognition server 300 and a number database 400 of a crime prevention cloud system for identification, which is a multi-type CCTV number according to an embodiment of the present invention. According to this, the number recognition server 300 recognizes the number plate of the photographed vehicle in the image data transmitted from the management server and extracts the number. The number recognition server 300 includes an image acquisition unit 310, an image processing unit 320, an image request unit 330, and a number recognition unit 340.

The image acquisition unit 310 receives necessary image images from the integrated image server 200. The video image acquired by the video acquisition unit 310 is distributed according to a predetermined distribution rule from the integrated video server 200. For example, the video image may be a video image for each set time period . When the image is periodically transmitted, it is preferable that the transmission period is within a range that does not hinder data processing.

Also, the image can be delivered only when a special event occurs. An event is a case where a crime is needed in a specific area, or when it is necessary to acquire an image on the escape route of a suspect.

The image processing unit 320 finds a vehicle entry frame from the image received from the image obtaining unit 310 and performs a community clustering of numbers to secure coordinates at which the corresponding number is located and extracts an image based on the coordinates Algorithm.

At this time, the grouping of numbers can be performed by changing the level to 2, 3 or 4 characters as shown in FIG. In addition, it is preferable to extract a larger area of the upper, lower, right, and left sides of the image with reference to the coordinates as shown in FIG. In case of video taken by general security CCTV, sloped license plates can be taken. In this case, if you extract a larger area, you will not have license plate area that is not recognized. Also, license plate area is enlarged, The probability of existence is increased.

On the other hand, if the image does not conform to the license plate format, the number recognition algorithm is not performed any more.

The number recognizing unit 340 extracts individual characters such as consonants, vowels, and numbers from the license plate area to recognize the numbers. In the case of the old license plate, it is recognized including the area name. The number recognition is applied to both the neural network method and the pattern method. The patterning method is a method of determining a predetermined pattern of individual letters or numbers, determining the degree of matching with the pattern in the image, and recognizing the matching degree as a letter or a number if the degree of matching is higher than a certain level. If a pattern similar to a previously learned letter or number is seen without a pattern, it is recognized as a letter or number.

All the numbers recognized from both are stored in the number database 400 or transmitted to the number manager. The reason why all of the number database 400 extracted from both schemes are used is that even a misidentified number can be used as meaningful data.

Partially recognized numbers are also stored and managed as meaningful data.

For example, as shown in FIG. 6, number recognition is performed in the process of recognizing numbers. However, consonants and vowels can not be recognized at all in the patterning method, and in the neural network method, , And the result of the recognition can be displayed. In this case, when recognition probability is low and recognition probability is low, partial recognition is performed except for unrecognized characters such as 07X1928.

On the other hand, when extracting the license plate area from the images taken by the non-road exclusive CCTV, unlike the road-only CCTV image, it is rare that the entrance of the vehicle and the camera are in the same direction of the same direction, .

FIG. 7 is a view showing a car license plate image recognition process in an irregular position. According to this, when the number recognition algorithm is executed to extract the license plate area, the license plate has a rhombic shape due to the perspective. In this case, it is possible to increase the recognition rate of the license plate by processing the image in a rectangular shape by moving the position of the diamond shape.

Can be described in detail with reference to the algorithm of FIG. (S805). If the type of the CCTV is the road-only CCTV (S810), the front of the license plate is photographed. Therefore, the CCTV information of the video frame (S800) (S815), and performs number recognition (S820).

When the type of the CCTV is the non-road-dedicated CCTV (S825), the sides of the license plate are photographed. Therefore, a grouping of numbers is performed from the received image (S830), and the image is divided into upper, (S835). The vertexes of four edges of the license plate are extracted from the expanded area (S840), and the distorted license plates are corrected using the vertices (S845). The number recognition is performed using the calibrated license plate (S850). After the number recognition is performed, it is determined whether the recognized number is a license plate format (S855). If the license plate format is the license plate format, the data is stored (S860) .

The distorted license plate image described above can be corrected through the following process and will be described with reference to FIG. When the license plate position is determined using the non-road dedicated CCTV in the process of determining the position of the license plate, the number recognizing unit 340 obtains the coordinates of the numerical area by performing the cluster identification of numbers, Extend the image to the left and right peripheral regions and extract the vertices A, B, C, and D of four edges of the license plate from the extracted image.

The rectangle ABCD connecting the four vertices is rotated so that it is at an angle of 90 degrees with the vertical line on the side AB. Then, on the side AB, perform plate number conversion to convert the side CD into a rectangle shape by adjusting the length of the side CD in parallel with side AB and one-to-one correspondence with side AB.

The number database 400 includes the number extracted from the number recognizing unit 340, the photographing CCTV, the image capturing date and time, and the storage location of the corresponding moving image. The vehicle number stored in the number database 400 may be usefully utilized.

The security cloud system for identification, which is the number of multiple CCTVs according to one aspect of the present invention, may further include a number search terminal 500. The number search terminal 500 is connected to the number database 400 and the integrated image server 200. When the vehicle number is inputted as the input means, the vehicle number is searched from the number database 400 and the matching vehicle number is inquired If there is a matching car number, at least one of the photographing CCTV, the photographing date and time, and the storage position of the moving picture can be displayed.

FIG. 10 is a diagram illustrating a result of performing a number search from a number database 400 using a number search terminal 500 of a crime prevention cloud system for identification, which is a number of multiple CCTVs according to an exemplary embodiment of the present invention. According to this, it has been a very time-consuming task to manually view and check all photographed images in order to check whether a specific vehicle has passed through a specific CCTV. However, when the present system is utilized, the number database 400 is searched, , When the video has passed, and if the video has passed, the location where the video is currently stored can be easily identified and recalled.

On the other hand, when the number recognition server 300 is used, i.e., the number recognition is performed at the center rather than at the local area, another problem arises. The video image sent by the number management unit is a video image sent periodically. There may be no vehicle in the video, and there may be no video with the vehicle. In order to prevent this, if the transmission period is reduced, the data capacity becomes large and the processing becomes problematic.

Therefore, it is important to detect the vehicle in order to secure only the image of the image when the vehicle is present. Also, it is necessary to adjust the illumination and lens during shooting to recognize the number after detecting the vehicle.

FIG. 11 is a CCTV (100) block diagram of a security cloud system for identification, which is a multi-type CCTV number according to another embodiment of the present invention. The CCTV (camera) 100 of the road security cloud system capable of recognizing the number of multiple CCTVs according to another embodiment of the present invention further includes a vehicle sensing unit 130, an illumination and a lens control unit 140.

The vehicle sensing unit 130 may be a radar or a laser sensor. The vehicle detection unit 130 may be an image sensor incorporated when the camera is an intelligent CCTV. When the vehicle enters the vehicle, the vehicle sensing unit 130 senses the vehicle and transmits a sensing signal to the illumination and lens control unit 140.

The illumination and lens control unit 140 controls the illumination and the lens according to the vehicle detection signal so as to secure the image in an environment suitable for photographing.

At this time, the shutter speed and white balance of the lens are adjusted to the gray value of the license plate of the vehicle. Recognize the gray value of the incoming license plate and adjust the lens value suitable for the surrounding environment.

By applying such detection means to the CCTV 100, the number recognition unit 340 can receive a clear image including the vehicle.

At this time, if the CCTV 100 is a non-road-dedicated intermittent camera, it is preferable that data is further included in the study. The data processing unit receives the detection signal from the vehicle sensing unit 130 and inserts the identification marker into the image data corresponding to the detected image frame of the vehicle among the image data obtained from the CCTV 100 according to the detection signal. At this time, the identification marker may be included in the image data in various ways. It is also preferable to send the image data by ID to the file name of the image data.

The image data embedded in the image data is sent from the communication unit 120 to the integrated image server 200.

In this case, the integrated image server 200 transmits only the image data including the identification marker to the image acquisition unit 310 of the number recognition server 300. As a result, all of the image images transmitted to the number recognition server 300 are images having clear image quality with the license plate of the vehicle.

On the other hand, when the CCTV (camera) 100 is a road-only intermittent camera, it is not necessary to send all the images to the integrated video server 200. Therefore, in order to reduce the burden on data transmission and processing, It is possible to transmit only the image data in the case where it is detected.

In the case where the CCTV 100 is an intelligent CCTV, the vehicle sensing unit 130 may be applied as being included in the CCTV.

Even in this case, all of the image images transmitted to the number recognition server 300 are images having clear image quality with the license plate of the vehicle. In addition, since the vehicle and the camera can be kept in the inactive state while the vehicle is not detected by the vehicle sensing unit 130, power can be saved.

The road security cloud system capable of recognizing the number of the multi-type CCTV according to the present embodiment also has the advantage of real-time vehicle number recognition. That is, the vehicle is detected and the image is photographed in an environment in which the CCTV 100 is controlled. The photographed image data is transmitted to the number recognition server 300, where the number recognized and recognized is displayed on the monitor of the control center in real time do.

In this case, since the numbers of all the vehicles photographed in the CCTV 100 are confirmed and stored in the number database 400 in real time, they can be used as more useful information. For example, when the number of the target vehicle is registered in the number management unit of the integrated video server 200, when the number of the vehicle is recognized by the number recognition unit 340 and transferred to the integrated video server 200, The corresponding image can be displayed while a warning sound is generated.

Hereinafter, a multi-type CCTV number recognition method according to another aspect of the present invention will be described. 12 is a flowchart of a method for recognizing a number of multiple CCTVs according to an embodiment of the present invention. As shown in FIG. 10, the number recognition method includes a vehicle sensing step, an illumination and lens controlling step, an image data acquiring step, a data converting step, a video image transmitting step, a license plate position determining step, a number extracting step, .

The vehicle detection step S1200 detects whether the vehicle enters the vehicle detection unit including the radar or the laser sensor.

In the illumination and lens control step S1210, when the vehicle detection unit detects the entry of the vehicle, the control unit receives the detection signal and adjusts the illumination and the lens to an environment suitable for photographing.

In the image data acquisition step (S1220), the vehicle is photographed in an environment suitable for photographing adjusted by the illumination and lens control unit, and the photographed image data is transmitted using the communication unit.

The data switching step S1230 receives the image data from the communication unit of the CCTV at a remote location, converts the image data into a video image, and distributes and displays the video image to the integrated video server.

The image image delivery step S1240 is a preparation step for extracting the vehicle license plate position from the data transferred from the image data to the image image.

The car license plate location determination step S1250 is a step of determining the location of the car license plate from the received image image, and performs a community clustering of the numbers to determine coordinates at which the corresponding number is located.

In the number extracting step S1260, a number identification algorithm is performed based on the acquired coordinates by performing a population clustering of numbers to extract numbers.

The number recognizing step S1270 recognizes the number extracted in the number extracting step and judges whether the extracted number matches the license plate format. It stores image data when it conforms to license plate format, and discards image data when it does not conform to license plate format.

The data storing step S1280 is a step of storing the information including the extracted number in the number database, and is stored including the corresponding photographing CCTV, the date and time of photographing, and the storage position of the corresponding moving image.

The features, structures, effects, and the like illustrated in the above-described embodiments can be combined and modified in other embodiments by those skilled in the art to which the embodiments belong. Therefore, it should be understood that the present invention is not limited to these combinations and modifications.

100: CCTV
110:
120:
130:
140: Illumination and lens control unit
200: Integrated video server
300: Number recognition server
310:
320:
330:
340:
400: number database
500: Number Search Terminal

Claims

A vehicle detection unit for detecting the entry of the vehicle,
Receives the vehicle detection signal from the vehicle sensing unit, and controls the illumination and the lens
An illumination and lens controller,
A video photographing unit capable of photographing a vehicle, and
Two or more CCTVs including a CCTV including a communication unit capable of transmitting photographed video data and not equipped with a number recognition function;
An integrated video server located at a remote location from the CCTV and receiving video data from the communication unit and converting the received video data into a video image;
An image acquisition unit for receiving a video image from the integrated video server,
An image processing unit for receiving the image image from the image acquisition unit and determining the position of the license plate;
And a number recognition unit for extracting a number from the license plate area determined by the image processing unit, the number recognition server located at a remote place from the CCTVs; And
And a number database for storing information including numbers extracted from the number recognition unit.

The method according to claim 1,
The first type of the CCTV is a road-dedicated CCTV installed on a road dedicated to an automobile for photographing a car. The second type is a CCTV of a multi-type CCTV including a non-road-dedicated CCTV, A roadside crime prevention system for recognition.

3. The method of claim 2,
When the number recognition unit identifies the license plate position using the video data of the non-dedicated CCTV,
The coordinates of the numerical region are obtained by performing a cluster clustering of numbers,
Extracts an image in the upper, lower, left, and right peripheral regions based on the coordinates,
Extracts vertexes of four edges of the license plate from the extracted image,
Connecting the neighboring vertices to create a rectangle,
The left side of the rectangle is rotated so as to be an angle of 90 with the vertical line,
The security cloud system for edible roads is a number of various CCTVs that performs plate number correction by matching the left side and the right side one by one by adjusting the right side of the rectangle to the left side and the length same as the left side so that the rectangle is rectangular.

The method according to claim 1,
Wherein the number database essentially includes a number of the vehicle extracted by the number recognition unit and is at least one of a group consisting of a photographing CCTV, a photographing date and time, and a storage location of the moving picture, Security cloud system.

3. The method of claim 2,
The non-
Wherein the CCTV number is a number of a multi-type CCTV that receives an identification signal from the vehicle sensing unit and inserts an identification marker so as to be distinguished from other video data in the video data including the vehicle.

3. The method of claim 2,
The road-
And the communication unit transmits the video data when the vehicle is detected using the vehicle sensing unit to the integrated video server.

delete

The method according to claim 1,
The roadside crime prevention system includes:
A number search terminal, and when a vehicle number is input to the number search terminal, the vehicle number is searched for from the number database, and a matching vehicle number is searched. If there is a matching vehicle number, the corresponding photographing CCTV, A crime prevention cloud system for use as a CCTV number for displaying at least one of video storage locations.

A vehicle sensing step of sensing an entrance of the vehicle at the vehicle sensing unit;
An illumination and lens control step of receiving a detection signal from the vehicle sensing unit and controlling illumination and a lens in an illumination and lens control step;
An image data acquisition step of photographing a vehicle with two or more CCTVs including a CCTV not having a number recognition function and transmitting photographed image data from the CCTV to an integrated image server located at a remote place;
A data conversion step of receiving image data from the CCTV and converting the image data into a video image and distributing the video image according to a predetermined rule in the integrated video server;
A video image delivery step of delivering the video image to an image acquisition unit;
A vehicle license plate position determination step of receiving the image image from the image processing unit and determining the position of the license plate;
A correction step of correcting the license plate extracted from the identified license plate area;
A number recognition step of recognizing a number from the corrected number plate; And
And a data storing step of storing information including the extracted numbers in a number database.

10. The method of claim 9,
The first type of the CCTV is a road-dedicated CCTV installed on a road dedicated to an automobile for photographing a car. The second type is a CCTV of a multi-type CCTV including a non-road-dedicated CCTV, Recognition method.

11. The method of claim 10,
Wherein the number recognition step comprises:
When the position of the license plate is determined using the image data of the non-dedicated CCTV,
A numerical region coordinate acquiring step of acquiring coordinates of a numerical region by performing a cluster-finding of a numeral;
Extracting an image by expanding an image into upper, lower, left, and right peripheral regions based on the obtained coordinates;
A vertex extraction step of extracting vertexes of four edges of the license plate from the extracted image;
A rotation step of connecting the extracted vertices to form a rectangle and rotating the left side of the rectangle so as to have an angle of 90 with a vertical line; And
A rectangle length adjustment step of adjusting the right side of the rectangle so as to be a rectangle so that the rotated rectangle is parallel to the left side and the same length as the left side so that the left side and the right side of the rectangle correspond one to one, How to recognize multiple CCTV numbers.