KR102051829B1 - Bigdata Previewport Multi-resolution video transmission CCTV using License plate detection and Bigdata Previewport Multi-resolution video transmission traffic control method using License plate detection - Google Patents

Abstract

The present invention relates to a big data preview port multi-resolution video transmission-based smart traffic information collection device using license plate number detection which is capable of reducing a bandwidth traffic amount and minimizing a transmission delay speed and a traffic control method using the same. According to the present invention, the smart traffic information collection device comprises: a pan/tilt housing camera unit (110); a memory unit (120); a first data transmission/reception unit (130); a big data analysis unit (200) receiving a high-resolution CCTV video in a bitstream and analyzing the video through a big data view port prediction technique; a multi-resolution processing unit (300) processing video data of a vehicle with a high traffic law violation probability among pieces of vehicle information analyzed in the big data analysis unit (200) at a high resolution through a set preview port (27) and processing the remaining background video and objects excluding the preview port (27) at a low resolution to wirelessly transmit processing results to a central control center (500) in real-time; a vehicle license plate recognition unit (400) recognizing a vehicle license plate corresponding to a high-resolution priority view port (12) processed in the multi-resolution processing unit (300) to transmit a recognition result to the multi-resolution processing unit (300); and a CPU (160).

Description

Big data prediction viewport based on car number recognition Smartport information collection device based on multi-resolution video transmission and traffic control method using the device {Bigdata Previewport Multi-resolution video transmission CCTV using License plate detection and Bigdata Previewport Multi-resolution video transmission traffic control method using License plate detection}

The present invention relates to a smart traffic control method and traffic information collecting device based on big data prediction viewport multi-resolution image transmission through vehicle number recognition, and more particularly, to vehicle image information and a central control center photographed from a traffic information collecting device. By comparing the stored video data with the violation of the big data road traffic law, if the high-visibility vehicle image information exceeds the set threshold probability value, a certain area is set as the prediction viewport centered on the vehicle image information and transmitted first in high quality. The present invention relates to a smart traffic information collection device based on big data prediction viewport multi-resolution image transmission through vehicle number recognition that can reduce the amount and minimize the transmission delay speed, and a traffic control method using the device.

Recently, as the area of technology using big data is expanded, the influence on the integrated vehicle traffic control management system is also affected. In particular, the CCTV, a traffic information collection device for integrated traffic control, is equipped with a built-in industrial computer to integrate various technologies such as crackdown, security, crime prevention, parking management, search, lighting control, real-time communication, mobile linkage, and IP tracking. Smart traffic control service is provided through more advanced system configuration.

Among them, vehicle tracking technology through license plate recognition, which is widely used in areas such as crackdown, security, and crime prevention, has been developed very rapidly. Currently, a system for detecting vehicle information in an advanced manner using various algorithms has been applied.

As a prior art to which such a system is applied, an automatic illegal parking control system and method using a fixed camera, which is registered as Korean Patent No. 10-0785757, has been devised.

The prior art divides the surrounding stop areas such as one-way, round-trip lane, three-way, four-way into several detection areas, and then periodically monitors the camera using high-speed pan tilt characteristics of the camera. The present invention relates to a system and a method for more effectively preventing and cracking down illegal parking vehicles by automatically detecting and violating a vehicle that violates the parking rules, and operating an on-site system and a center system. The camera repeatedly monitors a plurality of detection areas periodically using a camera, and when detecting an entry vehicle in each detection area, recognizes a vehicle number from an image including a vehicle license plate and transmits parking stop violation information to the center system. , The center system is based on the parking violation facts information In addition, the vehicle system issues a notice of a parking violation through the vehicle inquiry, and the field system determines a plurality of areas according to the shooting distance of the camera, that is, the distance from which the camera is properly focused to obtain a clear image in multiple directions. It is possible to monitor whether there are illegal parking violations in the detection areas divided by

However, a large amount of capacity is required to transmit the processed image to the central control center. In particular, in order to perform real-time data transmission in a wireless manner, it is necessary to increase traffic of a communication network, and the cost burden is inevitably increased.

An image transmission method and apparatus for recognizing a vehicle number as disclosed in the prior art, which has been devised to solve the above problems, and a multi-resolution image processing apparatus and image processing as disclosed in Korean Patent No. 10-1704775. A method is disclosed.

An image transmission method and apparatus for recognizing a vehicle number in the vehicle number recognition method and apparatus therein, the image transmission method for recognizing a vehicle number, to obtain a first image captured by the target vehicle from a camera installed on the road Extracting a second image corresponding to a vehicle license plate from the first image, reducing the capacity of the first image to generate a third image, and the second image and the third image This method provides an image transmission method for vehicle number recognition including matching and transmitting the same to the control server, thereby pre-processing and transmitting the captured image of the camera, thereby eliminating the overload problem of the control server and miniaturizing the equipment in the field. To make it work.

Also, in the multi-resolution image processing apparatus and the image processing method, which is registered Patent No. 10-1704775, a dynamic object identification unit for identifying a dynamic object by extracting a motion vector from an input image, and extracting feature points from the dynamic object. Extracts and includes a region of interest detector configured to set at least a portion of the dynamic object as the region of interest and a variable compressor configured to compress the image so that the compression rate for the region of interest and the compression rate for the dynamic object are different from each other. For the region, high resolution ensures high compression efficiency and minimizes compression.

However, in the case of the prior arts as described above, the algorithm does not analyze image information by using big data, but lacks a decision factor of performing high resolution and low resolution image processing based on an allowable range of wireless bandwidth. There remains a problem of practically low practicality and specificity.

In addition, by employing a simple feature extraction method rather than setting and arranging prediction viewports for each tile of image information, the result does not minimize the actual transmission delay speed.

Patent Publication No. 10-2015-0115067 Patent Registration No. 10-0785757 Patent Registration No. 10-1704775

Accordingly, the present invention was devised to solve the above problems, and the vehicle image information having a high probability of violation by comparing the vehicle image information photographed from the traffic information collecting device with the vehicle image information of the big data road traffic law violation stored in the central control center. Exceeds the set threshold probability value, it sets a certain area around the vehicle image information as a prediction viewport and transmits it in high quality first to reduce the amount of bandwidth traffic and minimize the transmission delay speed. The focus is on smart traffic control method and traffic information collection device based on viewport multi-resolution image transmission.

In accordance with another aspect of the present invention, there is provided a smart traffic information collecting device based on big data prediction viewport multi-resolution image transmission through vehicle number recognition, and a pan tilt housing camera unit for collecting image input information of the video. 110); A memory unit 120 that periodically receives and classifies and stores road traffic law violation vehicle image information 10 classified and stored in the database 520 of the central control center 500; A first data transmission / reception unit 130 provided to periodically update the road traffic law violation vehicle image information 10 classified and stored in the database 520 of the central control center 500; Big data analysis unit 200 for receiving a high-resolution CCTV image in the bitstream to analyze the big data viewport prediction technology; Of the vehicle information analyzed by the big data analysis unit 200, the image data of the vehicle having a high possibility of violating the road traffic law is processed in high quality through the set prediction viewport 27, and the remaining background image except the prediction viewport 27 is processed. And a multi-resolution processor 300 for real-time transmission of the objects to the central control center 500 by processing the objects with low quality. And a CPU 160 embedded in the traffic information collecting device A to perform calculation and control for data processing of the big data analyzer 200 and the multi-resolution processor 300, wherein the big data analyzer The tile setting unit 210 and the tile setting unit 210 may be configured to divide and set the current vehicle image information 11 collected from the camera unit 110 into tiles 16, which are units of a predetermined frame. Based on the set tile 16, the road traffic law violation vehicle image information 10 stored in the memory unit 120 and the current vehicle image information 11 are compared to the compared vehicle image information value through a big data based algorithm. The viewport analyzer 220 and the viewport analyzer 220 determining whether the similarity value 28 of the violated vehicle image information 10 corresponding to the specific vehicle S is equal to or greater than a threshold probability value T set. Predictive viewport (27) is given by setting the viewport for the determined threshold probability value (T) And a multiple viewport setting unit 230, and the multi-resolution processing unit 300 receives the predicted viewports 27 arranged in the viewport analysis process S122 in real time, and allows a range of bandwidths based on the current bandwidth. If the bandwidth exceeding determination unit 310 to determine whether exceeds the current bandwidth determined by the bandwidth exceeding determination unit 310 exceeds the allowable range, the prediction viewport 27 is set as the priority viewport 12 An image rendering unit that renders and generates image image data through a fusion of image quality processed by the resolution setting unit 320 and the viewports processed by the resolution setting unit 320 at low quality. 330, an encoding unit 340 for receiving and encoding image data generated by the image rendering unit 330, and high-quality image information encoded by the encoding unit 340. It consists of a communication unit 350 to transmit the real-time wireless pre-loading, the image rendering unit 330 by recognizing the vehicle number through the current vehicle image information 11 of the image quality processed by the viewport in the resolution setting unit 320 And a vehicle number recognition unit 400 for transmitting to the vehicle, and the vehicle number recognition unit 400 converts the input RGB color vehicle image into a gray scale image and converts the license plate area through Kenny edge detection and LSD algorithm. The character area is detected through local edge-based binarization preprocessing, vertical filtering, CLNF, and blob analysis of the license plate candidate area detection unit 410 and the license plate candidate area detection area 410 detected by the license plate candidate area detection unit 410. A license plate character for character recognition using image normalization, pixel matrix, and DBN through the character area detection unit 420 to detect and the character area detected by the character area detection unit 420 It characterized by consisting of sikbu 430.

delete

In addition, the present invention, the central control center 500 is to determine the traffic situation of the road in real time and to manage and control the traffic flow, the integrated control server 510 for receiving and processing the traffic situation as real-time image information And a database 520 for collecting and storing the data processed by the integrated control server 510, receiving image information on a real-time traffic situation, and transmitting and receiving data for transmitting a control command to the traffic information collecting device A. FIG. A decoding unit 540 which decodes encoded image information by processing in high quality through a big data viewport prediction technique, an output unit 550 which displays the decoded image information on a screen, and CCTV control unit 560 for generating a control command to be transmitted through the data transmission and reception unit 530, and a high-definition image of a specific vehicle (S) that is likely to violate the road traffic law recognized by the output unit 550 Based on the information characterized by comprising a warning unit 570 for real-time alert to the specified vehicle (S).

delete

The present invention includes a series of built-in camera unit 110 to receive a control command from the CCTV control unit 560, and according to the control command to adjust the angle of the camera unit 110, image magnification, image stop. It characterized in that the configuration of the control command receiving unit 111 provided to perform the commands.

delete

The present invention provides a smart traffic control method based on big data prediction viewport multi-resolution image transmission through vehicle number recognition, comprising: collecting image input information from a video (S110); Tile the collected image input information and compare the road traffic law violation vehicle image information with the current vehicle image information through a big data based algorithm to predict the viewport 27 for a specific vehicle S having a high possibility of violation of the road traffic law. Big data analysis step (S120) to set; Multi-resolution for receiving the prediction viewport 27 set in the big data analysis step (S120) and processing the set prediction viewport 27 as the priority viewport 12 as a high quality image and processing the remaining viewports 13 as low quality. Processing step 130; And receiving, decoding, and outputting the image information processed through the multi-resolution processing step (S130) at the central control center 500 (S150), wherein the big data analysis step (S120) includes the current vehicle. Violation of the road traffic law stored in the memory unit 120 through the big data-based algorithm based on the tiled process (S121) and the tile set in the tiled process (S121). A viewport analysis process (S122) for comparing the vehicle image information 10 and the current vehicle image information 11, and determining whether the compared similarity value 28 is equal to or greater than a set threshold probability value T, and the viewport analysis process ( It is composed of a prediction viewport setting process (S123) of granting and arranging the prediction viewport (27) by setting the viewport for the threshold probability value (T) determined in step S122, and the multi-resolution processing step (S130), the viewport analysis Course (S A bandwidth exceeding determination step (S131) of judging whether the prediction viewport 27 arranged in step 122 is exceeded in real time on the basis of the current bandwidth, and the current bandwidth determined in the bandwidth exceeding determination step (S131). In the case of exceeding the allowable range, the resolution setting process (S132) of performing high quality processing on the prediction viewport 27 as the priority viewport 12 and lowering the remaining viewports 13, and in the resolution setting process (S132). An image rendering process (S133) for rendering and generating image image data through fusion of the image quality processed for each viewport, a process of receiving and encoding the image data generated in the image rendering process (S133) and the encoding. It is composed of the step (S135) of real-time wireless preloading and transmitting the high-definition video information encoded in the step (S134) to the central control center, the multi-resolution After the step S130, the vehicle number recognition step S140 is further provided, and the vehicle number recognition step S140 includes edge detection in the current vehicle image information 11 of the viewport received from the multi-resolution processor 300. Extracting the license plate area through the line propagation process (S141), and removing the miscellaneous except the character area by blob detection on the license plate area extracted in the license plate area extraction process (S141) and labeling the numbers and the character areas. And a character recognition process using a backpropagation neural network algorithm based on the character region separated through the process of separating (S142) and the process of separating the number and the character region (S142). It is done.

delete

In addition, the present invention, after the process of performing the character recognition (S143), the process of extracting the feature vector by performing a prior learning with a limited Boltzmann machine (RBM) (S144), and deep learning model deep neural network (DBN) algorithm Characterized in that further comprising the step (S145) for performing the final character recognition through a fine adjustment through.

delete

According to the present invention as described above, deep learning self-learning can be utilized through analysis of vehicle image information using big data-based algorithms through image data of constantly accumulating road traffic violation vehicles, thereby significantly reducing false recognition rates.

In addition, by setting the allowable range of the wireless bandwidth, the image quality can be flexibly determined based on the current real-time bandwidth through the factor of performing high resolution and low resolution image processing, and the vehicle image information can be transmitted with the optimized traffic amount.

In addition, the forecasting viewport is set and sorted according to the other columns of the image information, and the new information stored in the database of the central control center is updated at the traffic collecting device at a regular interval and stored in the memory unit. By directly comparing and analyzing, the transmission delay can be minimized by minimizing the time required for arithmetic processing.

1 is a schematic configuration diagram of a traffic information collecting device and a central control center according to the present invention.
2 is an exemplary view illustrating a traffic information collecting device having a housing unit according to an embodiment of the present invention.
3 is an exemplary view illustrating a traffic information collecting device equipped with a security lamp according to an exemplary embodiment of the present invention.
4 is a schematic exemplary view of image information of a current vehicle captured by a camera unit according to an exemplary embodiment of the present invention.
5 to 12b are exemplary views showing a traffic information collecting device showing the operation of the big data analyzer components according to an embodiment of the present invention.
13 to 17 are exemplary views illustrating a traffic information collecting device showing the operation of the multi-resolution processor according to an embodiment of the present invention.
18 to 28b are exemplary views illustrating a traffic information collecting device including a vehicle number recognition unit according to an embodiment of the present invention.
29 is an exemplary view of an output unit according to an embodiment of the present invention.
30A is an exemplary view of a warning unit according to an embodiment of the present invention.
30B is an exemplary view of an integrated traffic information collection system according to an embodiment of the present invention.
31 to 35 are exemplary views illustrating a smart traffic control method according to an embodiment of the present invention.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The present invention compares the vehicle image information photographed from the traffic information collecting device and the vehicle image information of the big data road traffic law violation stored in the central control center 500, when the vehicle image information having a high probability of violation exceeds the set threshold probability value T. Big data prediction viewport multi-resolution image through vehicle number recognition, which can reduce the amount of bandwidth traffic and minimize the transmission delay speed by setting a certain area as the prediction viewport 27 and transmitting it in high quality first based on the vehicle image information Referring to FIG. 1 as a transmission-based traffic information collecting device, the traffic information collecting device A includes a camera unit 110, a memory 120, a first data transmitting and receiving unit 130, a housing unit 140, Security 150, CPU 160, big data analysis unit 200, a multi-resolution processing unit 300, the vehicle number recognition unit 400 is made.

The traffic information collection device (A) is a CCTV device for monitoring a vehicle, which is generally present. Image cameras, detection sensors (Loop, Laser, Radar), built-in computers, communication modules for detecting license plates and vehicle number recognition from moving images Collect real-time vehicle image information, including, and after the image analysis and processing in the big data analysis unit 200 and the multi-resolution processing unit 300 to be described later, and transmits the processed image information to the central control center 500 Device.

The traffic information collecting device (A) is not implemented as a single collection device but implemented as a plurality of collecting devices to constitute a main component of the integrated traffic information collecting system 600 which will be described later.

The camera unit 110 is a high-resolution CCTV image photographing apparatus for photographing vehicles on the road, which can be rotated 360 degrees, and pan tilt to move the camera 110 up and down (tilt) left and right (pan) remotely. Type camera device, comprising a control command receiving unit 111 that can receive a control command to enable the operation in the central control center (500).

The camera unit 110 is preferably equipped with a HD CABLE SLIP RING for 360-degree continuous rotation to enable accurate variable speed control and position control by applying a BLDC MMOTOR, MEGA PIXEL (Full HD) camera for high resolution image acquisition It consists of, and transmits the current vehicle image information 11 to be taken in real time to the big data analysis unit 200 to be described later.

Referring to FIG. 4, the current vehicle image information 11 is image information of the current vehicle that the camera unit 110 is photographing in real time.

The memory unit 120 is a data storage device of a conventional traffic CCTV embedded computer, such as a computer code input / output, a software operating system, a program, an application, a user interface function, a processor function, etc. which the traffic information collecting device A performs. Various device firmware can be stored.

The memory unit 120 is added to the road traffic law violation vehicle image information 10 stored in the database 520 of the central control center 500 through the first data transmission and reception unit 130 to be described later and the database 520. The new image information may be periodically updated to be downloaded and stored in advance, and in addition to the memory mounted in the traffic information collecting device A, it may be used in conjunction with an external memory.

As described above, the memory unit 120 accumulates new image information downloaded and updated to form back data, and forms a DBN system capable of continuously self-learning based on existing image information data. Can be.

In addition, the memory unit 120 is configured to provide the stored road traffic law violation vehicle image information 10 to the big data analysis unit 200, which will be described later, to compare and analyze the current vehicle image information 11 in real time.

Referring to FIG. 1, the first data transmitter / receiver 130 includes a first wireless communication module 131.

The first wireless communication module 131 is a communication module that can wirelessly communicate with the data transmitter 530 of the central control center 500, and the image information of the traffic information collecting device (A) within the allowable bandwidth Real-time transmission wirelessly, and may periodically receive new image information added to the database 520 of the central control center 500.

Based on the allowable range of the bandwidth, the multi-resolution processing unit 300 to be described later transmits the entire current vehicle image information 11 in high quality using a bandwidth determination algorithm, or preferentially transmits only the ordered priority viewport 12 in high quality. Determine whether or not.

The first wireless communication module 131 is preferably implemented through 5G GiGa-Wifi, but is not limited thereto and includes all communication means capable of transmitting and receiving data wirelessly.

Referring to FIG. 2, the housing part 140 constitutes an outer portion of the traffic information collecting device A, and is an all-weather case device having a durable durability for outdoor use to be resistant to vibration, dust, toxic gas, and high temperature.

The housing unit 140 is designed to protect the camera unit 110 from high temperature and radiant heat so that a fan, a heater, and a temperature control bimetal are mounted on an inner holder, and other brackets such as a pan tilt and an angle adjuster are disposed below the housing unit 140. It is configured to be mounted.

The housing 140 is preferably made of stainless steel, and is configured to cope with various weather changes such as a compressor method, an electric method, and a high pressure air injection method, and is not limited thereto.

Referring to FIG. 3, a security lamp 150 is provided in a pair on both outer sides of the camera unit 110 to provide illumination according to the interlocking view of the camera unit 110 at night.

The security light 150 has a function to turn on and off the lights in response to the ambient brightness based on high brightness, high reliability power LED, preferably provide excellent cooling effect through the heat dissipation aluminum cooling fin design. Can be.

The security light 150 may receive a warning signal for a specific vehicle S having a high possibility of violating a road traffic law from the warning unit 570 of the central control center 500, which will be described later, and recognize the specific vehicle S. Can be blinked.

CPU 160 is a central processing unit embedded in the traffic information collecting device (A), as shown in Figure 1, a big data analysis unit 200, a multi-resolution processing unit 300, a vehicle number recognition unit (to be described later) The data processing operation of 400 can be calculated and controlled.

The big data analysis unit 200 may store the road traffic law violation vehicle image information 10 stored in the memory unit 120 through a big data based algorithm based on the current vehicle image information 11 photographed by the camera unit 110. Compares the current vehicle image information 11 with the threshold probability value T set by the similarity value 28 of the road traffic law violation vehicle information R corresponding to the specific vehicle S according to the compared vehicle image information value. As described above, as the apparatus for assigning and arranging the prediction viewport 27, the apparatus includes a tile setting unit 210, a viewport analysis unit 220, and a viewport setting unit 230 as illustrated in FIG. 1.

The tile setting unit 210 receives the current vehicle image information 11 from the camera unit 110 in real time and divides and sets the tile 16 into tiles 16, which are fixed frame units, as shown in FIG. 5.

Various settings such as the range, number, and length of the tile 16 may be controlled using the icons 17 as shown in FIG. 5 in the central control center 500.

Referring to FIG. 6, the viewport analyzer 220 may compare the road traffic law violation vehicle image information 10 stored in the memory unit 120 through a big data based algorithm based on the tiles set by the tile setting unit 210. Comparing the current vehicle image information 11, if the similarity value 28 to be described later of the compared vehicle image information is equal to or more than the threshold probability value set, the prediction viewport 27 is arranged to arrange the device, big data algorithm analysis unit ( 221, wherein the big data algorithm analysis unit 221 includes an image recognition unit 222, an image retrieval unit 223, a parallel processing unit 224, a similarity analysis unit 225, and a critical probability value analysis unit 226. It consists of.

The image recognition unit 222 recognizes and classifies an image through the teaching teacher learning model 18 or the non-teacher learning model 19 based on the tile 16 image information set by the tile setting unit 210. To perform.

Referring to FIG. 7A, the tutor learning model 18 extracts various features such as color histogram, SIFT, and HOG of the set tile 16 image information, and based on the features, teacher learning such as SVM, neural network, and GMM. The algorithms recognize and classify the images.

Referring to FIG. 7B, the comparative history learning model 19 is a learning model for recognizing and classifying an object using a deep learning technology. The plurality of layers h which are image information of the tile 16 set by the tile setting unit 210 are shown. It's a multi-layer network.

The upper layers h ¹ of the comparative learning model 19 receive the output of the previous layer h ² and generate the features W ₁ of the upper level to recognize and classify the image information of the tile 16. do.

The image retrieval unit 223 is a large-capacity image content retrieval device for retrieving the road traffic law violation vehicle image information 10 classified and stored in the memory unit 120 based on the image information recognized by the image recognition unit 222.

The image retrieval unit 223 is a device capable of retrieving image information similar to the current vehicle image information 11 among the road traffic law violation vehicle image information 10 classified and stored in the memory unit 120. The sample image candidate group 20 for the threshold probability value T to be set at 225 is extracted.

The sample image candidate group 20 is a sample of image information most similar to the current vehicle image information 11 among the road traffic law violation vehicle image information 10 classified and stored in the memory unit 120.

Similar image search technology 21 is used as a method for selecting the sample image candidate group 20.

Referring to FIG. 8, the similar image retrieval technology 21 is a technique for retrieving an image similar to the current vehicle image information 11 from the memory unit 120. The parallel image retrieval unit 224 may be configured based on feature vector clustering. The sample image candidate group 20 is detected through a fast pseudo-vector search technique.

Referring to FIG. 9A, the parallel processor 224 is a framework device that provides an open and extensible library for various applications in a system for large-scale image retrieval. The feature extractor 224a and the unit model learner 224b. ), The statistical machine learning section (224c).

The parallel processor 224 classifies the image images stored in the memory unit 120 through the feature vector clustering technique to calculate the effective image 24 through feature extraction, unit model learning, and statistical machine learning.

Referring to FIG. 9A, the feature extractor 224a receives the current vehicle image information 11 from the image searcher 223 based on object information such as a vehicle, an artificial facility, a natural object, a road lane, and relative location information. Set a number of features.

Referring to FIG. 9B, the feature extractor 224a displays the object information based on the image information (learning data) of the tile 16 calculated by the image recognition unit 222 through the teaching teacher learning model 18. Learning classifiers such as SVM (Support Vector Machine) and GMM (Gaussian Mixutre Model) using global features such as histogram, color moment, color correlogram, and local features such as scale-invariant feature conversion, local binarization pattern, gradient histogram, etc. Through the unit model learning unit 224b.

The unit model learning unit 224b configures the unit models 22 based on the setting features, and sets the base model 23 by constructing a core outline using the unit models 22. do.

The base model 23 is an outline ensemble model in which weight values for each element are calculated in consideration of the relative position, size, range, etc. of objects based on key setting characteristics of the current vehicle image information 11.

Referring to FIG. 9C, the unit model learning unit 224b generates a unit model pool and sets the current vehicle image information 11 as a reference image based on the core outline information in the generated unit model pool. The unit model H most similar to the reference image is set, and the optimal base model 23 is detected through the ensemble classifier.

Referring to FIG. 9A, the statistical machine learning unit 224c sets an image image corresponding to the base model 23 in the memory unit 120 as an effective image 24 based on the base model 23. Multiple valid images 24 are extracted.

The similarity analyzer 225 calculates a sample image candidate group 20 based on the extracted plurality of valid images 24 and sets a threshold probability value T for determining the similarity value 28 to be described later. And a fuse reducer 225a.

The fuse reducer 225a is a device for extracting and removing an error regarding misrecognition, shadowing, and error of a plurality of valid images 24 generated by the statistical machine learning unit 224c. The sample image candidate group 20 is calculated through an error elimination method used by the blob analysis unit 423 of 420.

An error elimination method used by the blob analysis unit 423 will be described in detail later.

The threshold probability value T is an initial threshold value set in the similarity analysis unit 225 and is set based on a hierarchical matching technique used in an automatic threshold value determination method using a conventional cumulative similarity measure. The threshold probability value T ) May vary according to various setting methods, weighting factors, digitization stages, subjective perception, etc., and the numerical values may be input by an administrator of the central control center 500.

That is, the threshold probability value T is a value which is a criterion for determining the similarity value 28 which will be described later. As described above, the threshold probability value T may be set freely by the administrator of the central control center 500, but as mentioned above. It is desirable to calculate the optimal threshold value through the hierarchical matching technique used in the automatic threshold determination technique using conventional cumulative similarity measurement.

The threshold probability value analysis unit 226 digitizes the sample image candidate group 20 based on the threshold probability value T set in the similarity analysis unit 225 to determine whether the threshold probability value T is exceeded.

Referring to FIG. 10, the process of digitizing the sample image candidate group 20 based on the threshold probability value T set in the similarity analyzer 225 may include the sample image candidate group 20 based on the base model 23. The similarity value is normalized by measuring the similarity value of, and calculated by using a polynomial used in the similarity value determination technique using the cumulative similarity measurement.

The process of calculating and calculating by the polynomial is preferably performed through the algorithm of the above-described teaching teacher learning model 18 or the comparative learning model 19.

That is, the calculation process weights the set features extracted by the feature extractor 224a through the algorithm of the supervised teacher learning model 18 or the non-teacher learning model 19, as shown in FIG. 10. The similarity value 28 of the sample image candidate group 20 normalized through the Gaussian mixture model G is calculated by assigning a high weight value to important setting features and a low weighting value to relatively unnecessary setting features.

The weight value may be set by the administrator of the central control center 500 as a flexible calculation factor according to various environmental variables.

The Gaussian mixture model G is given a smaller weight as it moves away in two dimensions to generate a similarity value 28 in the form of a normal distribution according to the standard deviation in a circularly symmetrical structure.

When the calculated similarity value 28 is compared with the set threshold probability value T, the sample image is extracted as the residual image 25 when the threshold probability value T is exceeded.

That is, according to FIG. 10, which is a preferred embodiment of the present invention, assuming that the set threshold probability value T is 80, the threshold probability value T is 81% of the similarity values 28 of the sample image candidate groups 20 calculated. The sample image 28b exceeding is extracted as the residual image 25.

 Referring to FIG. 11, when the residual image 25 exists, the threshold probability value analysis unit 226 transmits the residual image 25 and the viewport setting command 26 to the viewport setting unit 230, and the residual image. If 25 does not exist, no command is sent to the viewport setting unit 230.

Referring to FIG. 12A, when the residual image 25 occurs in the fuse reducer 225a, the viewport setting unit 230 may determine the threshold probability value based on the road traffic law violation vehicle information R of the residual image 25. Corresponding to the violation vehicle information (R) according to the calculation basis of the similarity value (28) 81% calculated by the analysis unit (226) (process of calculating the similarity value of the Gaussian mixed model (G) by reflecting the weight value of the set feature) The prediction viewport 27 of the specific vehicle S for the tiled current vehicle image information 11 is set to be transmitted to the multi-resolution processor 300 to be described later.

That is, the prediction viewport 27 is connected to the specific vehicle S of the current vehicle image information 11 having a similar appearance to the road traffic law violation traffic information R of the residual image 25 as shown in FIG. 12B. It is an area of extracting and setting the Korean tile group 28, and it is a technique of presetting an area of interest for a specific vehicle S that has a similar behavior to the road traffic violation vehicle information R. FIG.

The viewport setting unit 230 transmits the tiled current vehicle image information 11 including the set prediction viewport 27 to the multi-resolution processing unit 300.

The multi-resolution processing unit 300 determines whether the current bandwidth of the traffic information collecting device (A) exceeds the allowable range of the set bandwidth, and if it exceeds, recognizes the set prediction viewport (27) as the priority viewport (12) A device that encodes first in high definition and transmits the remaining viewports 13 in low quality. As shown in FIG. 1, the bandwidth exceeding determination unit 310, the resolution setting unit 320, the image rendering unit 330, The encoding unit 340 and the communication unit 350 is configured to include.

The bandwidth over-determination unit 310 is a device for determining whether the current bandwidth exceeds the allowable range of the set bandwidth. If the current bandwidth exceeds the allowable range of the set bandwidth, the priority viewport 12 is transmitted to the resolution setting unit 320 to be described later. If not, the setting command is transmitted, and if not exceeded, the current vehicle image information 11 is processed as it is of high quality and transmitted to the encoding unit 340.

The current bandwidth is the amount of traffic transmitted in real time from the traffic information collecting device (A) to the central control center 500 by using a general traffic measuring device (automatic).

The allowable range of the bandwidth is the total amount of traffic allocated to the traffic information collecting device A by the wireless communication network, and the performance of various built-in computer hardware and software such as the CPU, processor, memory, and communication module of the traffic information collecting device A. It depends on the type of product model and the communication network, and the traffic information collection device (A) itself is set or controlled to be controlled remotely from the central control center (500).

Referring to FIG. 13, the resolution setting unit 320 receives a determination result of whether the current bandwidth exceeds the allowable range of the set bandwidth in real time from the bandwidth exceeding determination unit 310.

When the resolution setting unit 320 receives a determination result from the bandwidth exceeding determination unit 310 that the current bandwidth exceeds the allowable range of the set bandwidth (E1), the prediction view port 27 received from the viewport setting unit 230 Based on the tiled current vehicle image information 11 including), the prediction viewport 27 is set as the priority viewport 12 and processed in high quality, and the remaining background images and objects except the prediction viewport 27 are processed. The remaining viewports 13 corresponding to the tile information are processed with low quality and transmitted to the image rendering unit 330.

That is, when the current bandwidth exceeds the allowable range of the set bandwidth through the above process, only the prediction viewport 27 is set as the priority viewport 12 so as to be transmitted in high quality through the communication unit 350 to be described later. Therefore, the amount of traffic can be flexibly adjusted according to the bandwidth.

On the other hand, when the resolution setting unit 320 receives the determination result that the current bandwidth does not exceed the allowable range of the set bandwidth from the bandwidth excess determination unit 310 (E2), the priority viewport 12 and the remaining viewport ( The entire viewport 36 including all 13) is processed in high quality and transmitted to the image rendering unit 330.

That is, since the amount of traffic does not need to be adjusted when the current bandwidth does not exceed the allowable range of the set bandwidth, the entire viewport 36 is transmitted in high quality.

Meanwhile, the above-described operations simply compare the current bandwidth with the allowable range of the set bandwidth, and a series of processes are performed according to the determination result. However, in another embodiment of the present invention, the allowance of the current bandwidth and the set bandwidth is determined. By measuring the difference in detail, it is possible to perform various modifications such as widening the range of the prediction viewport 27 by the amount of traffic corresponding to the difference, or correcting the image quality of the prediction viewport 27 to ultra high quality. It will be obvious from the standpoint of an ordinary engineer working in the industry.

Hereinafter, the operation of each component when the resolution setting unit 320 receives a determination result from the bandwidth exceeding determination unit 310 that exceeds the allowable range of the set bandwidth (E1) will be described. .

As illustrated in FIG. 14, the resolution setting unit 320 sets the prediction viewport 27 as the priority viewport 12 to include both the base layer 29 and the enhancement layer 30 of image encoding. The remaining viewports 13 include only the base layer 29 to differentially correct the image quality.

The base layer 29 refers to a low quality layer which is not subjected to any image quality correction process in the scalable image encoding technology.

The low quality layer of the base layer 29 has a small amount of data so that a separate compression encryption process is not necessary, and thus has little effect on the amount of traffic when transmitting image information using wireless communication.

Since the enhancement layer 30 occupies a large amount of data as a separate layer generated as a high quality layer through the image quality correction process on the layer of the base layer 29, the encoding unit 340, which will be described later, undergoes a compression encryption process to perform the center. The preload is transmitted to the control center 500.

In this case, since the tile numbers of the priority viewport 12 including both the enhancement layer 30 and the base layer 29 are consecutive, the tile start and end numbers, coordinate information, and intra-tile coding without all the numbers being sent. It can be effectively compressed by using the unit (CU) number list and the tile number as an expression. (Not shown)

The image rendering unit 330 is a device for rendering and generating image image data through fusion of image quality processed by the viewport in the resolution setting unit 320. The image layer 330 and the base layer 29 processed in the resolution setting unit 320 The enhancement layer 30 receives the layer and synthesizes the entire image 31 to be transmitted to the encoding unit 340.

The image rendering unit 330 receives the final recognition character information 44 and 45 which are vehicle number information recognized by the vehicle number recognition unit 400 to be described later, according to another embodiment of the present invention. The vehicle number insertion image 32 can be configured.

The operation of configuring the vehicle number insertion image 32 based on the vehicle number information recognized by the vehicle number recognition unit 400 will be described later in more detail.

The encoding unit 340 is a device for compressing and encrypting the entire image 31 synthesized by the image rendering unit 330 and is a video compression module having a video encoding program that is widely commercially available.

Referring to FIG. 15, the encoding unit 340 transmits the basic layer 29 of the entire image 31 received from the image rendering unit 330 to the communication unit 350 as it is, and transmits the priority viewport 12 to the priority viewport 12. The enhancement layer 30 including the corresponding base layer 29 is generally compressed and transmitted to the communication unit 350 through a widely commercialized video encoding program.

The communication unit 350 transmits and receives the second data transmission / reception unit 530 of the central control center 500 to describe the high quality compressed image information 34 encoded and encrypted by the encoding unit 340 and the low quality image information 33 which is not compressed and encrypted. As a communication device for transmitting to), and includes a second wireless communication module 351.

Referring to FIG. 15, the communication unit 350 prioritizes the high-definition compressed image information 34 to the second data transmission / reception unit 530 of the central control center 500 in real time through the second wireless communication module 351. The low quality video information 33 is subordinated in the order received from the encoding unit 340 by using the remaining bandwidth.

As described above, the operation of each component in the case where the resolution setting unit 320 receives a determination result from the bandwidth exceeding determination unit 310 that exceeds the allowable range of the set bandwidth (E1) has been described.

Hereinafter, the operation of each component in the case where the resolution setting unit 320 receives a determination result from the bandwidth exceeding determination unit 310 that the current bandwidth does not exceed the allowable range of the set bandwidth (E2) will be described. In addition, the resolution setting unit 320 configures the basic layer 29 and the enhancement layer 30 for the entire tiled current vehicle image information 10 as the entire viewport 36 so as to have high quality as shown in FIG. 16. Process.

The image rendering unit 330 receives the entire viewport 36 layer processed by the resolution setting unit 320 and processes the entire viewport 36 layer composed of only high quality images to be transmitted to the encoding unit 340.

Referring to FIG. 17, the encoding unit 340 compresses and encrypts the entire image 35 transmitted from the image rendering unit 340 to the communication unit 350 as described above, and the communication unit 350 is of high quality. The compressed image information 36 is transmitted to the second data transmitter / receiver 530 of the central control center 500.

In summary, when the resolution setting unit 320 receives a determination result from the bandwidth exceeding determination unit 310 that the current bandwidth exceeds the allowable range of the set bandwidth (E1), as shown in FIG. While the image information 34 is first transmitted to the central control center 500 and the low quality image information 33 is transmitted subordinated based on the remaining bandwidth, the resolution setting unit 320 is currently present from the bandwidth over decision unit 310. In the case of receiving the determination result not exceeding the allowable range of the set bandwidth (E2), as shown in FIG. 17, the central control center 500 displays the entire image 35 including only the high quality compressed image information 36. To send.

Through the above method, when the resolution setting unit 320 receives a determination result from the bandwidth exceeding determination unit 310 that the current bandwidth exceeds the allowable range of the set bandwidth (E1), there is a high possibility of a violation of the road traffic law. The high quality compressed image information 34 corresponding to the priority viewport 12 of the specific vehicle S is inserted in the central control center 500 and corresponds to the remaining viewports 13 of the object and background image having a relatively low importance. By transmitting the lower quality video information 33 based on the remaining bandwidth, the traffic situation can be controlled without delay in real time within a limited amount of traffic.

Hereinafter, the operation of each component of the traffic information collecting device A including the vehicle number recognition unit 400 according to an embodiment of the present invention will be described in detail.

In a preferred configuration of the traffic information collection device (A) including the vehicle number recognition unit 400 according to an embodiment of the present invention, the resolution setting unit 320 is the current bandwidth is set from the bandwidth over determination unit 310 In case of receiving the determination result exceeding the allowable range of the bandwidth (E1) and the resolution setting unit 320 receives the determination result that the current bandwidth does not exceed the allowable range of the set bandwidth from the bandwidth over-determination unit 310 Actions according to case (E2) should be distinguished.

 Referring to FIG. 18, the vehicle number recognition unit 400 receives a determination result from the resolution setting unit 320 that the current bandwidth exceeds the allowable range of the set bandwidth (E1). If only the priority viewport 12 is input, and the resolution setting unit 320 receives a determination result from the bandwidth exceeding determination unit 310 that the current bandwidth does not exceed the allowable range of the set bandwidth (E2) The viewport 36 is input.

Although the vehicle number recognition unit 400 is the same for a specific configuration action even if the input target is different from the priority viewport 12 or the entire viewport 36 as described above, the resolution setting unit 320 exceeds the bandwidth When the determination result of the current bandwidth exceeds the allowable range of the set bandwidth from the determination unit 310 (E1) and the resolution setting unit 320 allows the bandwidth of the current bandwidth is set from the bandwidth exceeding determination unit 310 When receiving the result of judgment not exceeding the scope (E2), it shall be described without distinguishing.

As described above, the vehicle number recognition unit 400 receives the priority viewport 12 or the entire viewport 36 from the resolution setting unit 320 of the multi-resolution processing unit 300, and the vehicle of the vehicle existing in the viewport. A device for recognizing a number and transmitting image information added with the recognized information to the image rendering unit 330 of the multi-resolution processing unit 300, as shown in FIG. 1, the license plate candidate area detection unit 410, The text area detection unit 420 and the character recognition unit 430 are configured.

The license plate candidate area detecting unit 410 converts the input RGB color vehicle image into a gray scale image and detects the license plate area through the Kenny edge detection technique and the LSD algorithm. As shown in FIG. And an image converter 412, an edge extractor 413, and a line propagator 414.

The vehicle detector 411 classifies the vehicle through a background removal method, an average background method, a car image method, a feature-based method, etc., which is a method of detecting a vehicle by dividing a background image and a foreground image from the vehicle image information of the input viewport. Detect.

The background removal method is a road image without a vehicle as a background image, and a method of removing and distinguishing a difference between the background image and the currently input image is composed of an average background method and a Gaussian Mixture Model.

The average background method is a method of generating a background using an average and a standard deviation of pixels of an input frame, and thus can generate a high quality background image through continuous background modeling.

The Gaussian mixture model method probably distinguishes pixels corresponding to a background by calculating pixel value data distribution of image information. In other words, a new probability distribution is represented through the sum of M Gaussian probability distributions to distinguish and generate a background.

20A and 20B, the moving or disappearing vehicles except the stationary objects in the background image are gradually faded out and absorbed as the background image by using the Gaussian mixed model in FIG. 20A, the input image. An image such as 20b is output.

The difference image method is a method of extracting a vehicle by subtracting an image to be used as a background and a current image. In other words, the difference between the background image and the current input image is calculated by calculating the difference between the pixel value of the background image and the current input image, and generating a binary foreground image based on the threshold that sets the absolute value. .

The feature-based method extracts and analyzes various features such as the contours and corners of the vehicle from the input image, and can identify and recognize objects using occulusion handling for overlapping vehicles.

The image converting unit 412 converts the input RGB color vehicle image into a gray scale image, and extracts the license plate using an RGB color model, an HSI color model, a YCbCr color model, etc. using color information of the license plate.

In other words, the gray scale image is converted into a gray scale image by considering the aspect ratio of the license plate area, the aspect ratio of the background and the character area, the change of the character width and contrast, the character density, and the relative character position information.

In addition, by using Adaboost, the characteristics of color, shape, and texture of license plate information can be trained by various techniques such as Haar-like feature, LBP (Local Binary Pattern), and MCT, and composed of Cascade classifier structure. Detection can be as shown in FIG. 20C.

The edge extractor 413 detects the license plate area by applying a Hough transform algorithm through various edge operator filters based on edge information of the license plate using the Kenny edge detection method 37.

The Kenny edge detection method 37 is a representative algorithm used to extract regions within an image by using outer boundary lines and edge information, and is generated when elements such as depth, shadow, and surface color of objects in the image exhibit changes and discontinuities. The edges of the objects are extracted and detected through the difference in the brightness variation using an edge, which is a set boundary of pixels having a constant brightness value distribution.

Referring to FIG. 21A, the Kenny edge detection method 37 includes a smoothing order S37a, a finding gradients order S37b, a non-maximum suppression order S37c, a double thresholding order S37d, and an edge tracking by hysteresis order. S37e).

The smoothing order S37a is a preprocessing order of blurring a video image to remove noise and removing noise on the image, using filtering such as median filtering, sharpening, gasussian filtering, and the like.

Finding gradients order (S37b) is an order using the various edge masks such as line edge mask, Sobel mask, prewitt mask, Roberts mask in order to find the part where the pixel value changes rapidly by using the gradient of the filtered image image Detects an edge and sets the remaining pixel values to zero.

As shown in FIG. 21B, the non-maximum suppression order S37c is an order of thinning the detected edges using the mask. If it is large, leave it as it is or remove it.

In the double thresholding sequence S37d, two threshold values T-low and T-high are used to distinguish portions of the image processed through the non-maximum suppression sequence S37c, which are not noise or edges. Apply to determine the edge.

 In the edge tracking by hysteresis order (S37e), a pixel whose size is larger than the T-high value set in the double thresholding order (S37d) is designated as an edge, and the second high size pixel adjacent to the specified pixel is reset to the edge to make the second value. The edge tracking by hysteresis sequence S37e is repeated for the pixel set as the edge.

The line propagation unit 414 is an apparatus for extracting lines including edge points through the Kenny edge detection method 37 and Hough transform using the line segment of the LSD algorithm 38.

Referring to Fig. 22A, the LSD algorithm 38 includes a similar angle forming point grouping and image segmentation order S38a; Optimal Line Segment Extraction Order (S38b). It consists of line segment individual authentication order S38c.

 The similarity angle forming point grouping and image dividing order S38a divides the formed groups through the line region in order to connect the points forming similar angles within the allowed range and form the group.

The optimal line segment extraction order S38b extracts an optimal line segment in an area composed of the respective lines in the image divided in the similar angle forming point grouping and the image segmentation order S38a as shown in FIG. 22B. do.

In the line segment individual authentication sequence S38c, the line segment extracted in the optimal line segment extraction sequence S38b is individually authenticated and detected.

Through the above-described procedure of the LSD algorithm 38, the license plate candidate area 39 is extracted as shown in FIG. 22C by vertical line segment detection and line propagation processing.

Referring to FIG. 19, the character area detection unit 420 includes a binarization unit 421, a CLNF unit 422, and a blob analysis unit 423.

The binarization unit 421 shortens and converts the image including the license plate candidate area 39 detected by the license plate candidate area detection unit 410 into the foreground and the background according to an application range based on a threshold. Binarization is performed through global and local techniques.

The method calculated through the global technique uses Ostu binarization.

The Ostu binarization method calculates a threshold value so that variance in the same group is minimized, and thus can be quickly binarized by applying a single threshold value to the entire image.

On the other hand, the method of calculating through the local technique uses a local edge-based binarization (40).

Referring to FIG. 23, the regional edge-based binarization method 40 includes an edge image generation order S40a, an intensity edge removal order S40b below a reference range, a duplicate edge image removal order S40c, and a sub image confirmation order S40d. , Average pixel value calculation and threshold setting order S40e, and setting threshold based window area binarization order S40f.

In the edge image generation procedure S40a, an Sobel edge detector is applied to generate an edge image for the gray scale image.

The intensity edge removal order (S40b) below the reference range includes within the upper 5-10% range of the edges generated in the edge image generation sequence (S40a) and removes the remaining small intensity edges.

In the overlapped edge image removal order S40c, the overlapped edge image is removed in the intensity edge removal order S40b below the reference range and the duplicate edge image is removed through a W × W size window.

In the sub image checking order (S40d), it is checked whether there are any edges in the selected sub image among the images de-duplicated in the duplicate edge image removing order (S40c).

In the average pixel value calculation and threshold setting sequence S40e, an average pixel value for the grayscale image is calculated based on a reference boundary value on the pixel-based original image of the images checked in the sub-image checking sequence S40d. Set the gray scale thresholds for the windows individually.

In the set threshold based window area binarization order (S40f), the binarization of each window area is performed by using the threshold value set in the average pixel value calculation and the threshold setting order (S40e) to generate the character recognition area as shown in FIG. Detect.

The CLNF unit 422 is a device for performing the CLNF operation 41 through the blob detection algorithm in a structure in which the CCLUF and the NFPP are mixed.

Referring to FIG. 25, the CLNF operation 41 is based on the binarization image information (binary foreground mask) processed by the binarization unit 421. An eight-period pixel check procedure S41a, a foreground mask correction order S41b, and 1 are performed. The moving object list 42 is determined through the difference labeling and union-find structure construction sequence S41c, the final labeling determination sequence S41d, and the rectangular region calculation sequence S41e.

At this time, the foreground mask correction order (S41b) replaces the morphological operation preprocessing performed by CCLUF to shorten the operation processing time, and the primary labeling and union-find structure construction order (S41c) is a four-way neighboring pixel or It is formed through connection element labeling, an 8-way neighboring pixel concatenation technique, and NFPP-based foreground pixel correction.

The blob analysis unit 423 is a device for removing a meaningless area in the license plate to determine the character area. The moving object list determined by the CLNF unit 422 using logical information such as horizontal, vertical size, and number of pixels. 42) blob merge and filter.

Referring to FIG. 26, the process of blob-merge and filtering the determined moving object list 42 includes a vertical histogram, an aspect ratio, a rectangular shape using vertical and horizontal edges, and a character of a license plate using a vertical edge density of an area. The final character area 43 is detected by removing the remaining area except the area "67ver 4455".

On the other hand, a method of extracting and removing errors of the plurality of valid images 24 generated by the statistical machine learning unit 224c, which is the action of the fuse reducer 225a described above, is also performed through the blob merging and filtering process. The sample image candidate group 20 may be calculated through the error elimination method.

Referring to FIG. 19, the character recognition unit 430 includes a mesh vector extractor 431, a thin cross-cross vector extractor 432, and a backpropagation analyzer 433.

The mesh vector extractor 431 is a device for normalizing the image of the final character region 42 detected by the character region detector 420 to a predetermined size. The mesh vector extractor 431 is a sum of the number of black pixels of a certain region. Calculate and determine the feature value of the subregion.

Referring to FIG. 27A, the final text area 43 image is normalized, divided into 16 sub-blocks, and a frequency of 0 for each sub-region is calculated and expressed as a binary value.

The thin cross-section vector extractor 432 is a pixel matrix device using the intersection point as a feature vector using horizontal and vertical detection lines based on the binary image generated by the mesh vector extractor 431.

Referring to FIG. 27B, the thinning cross vector extractor 432 processes an object surface brightness value of a binary value image by a parallel processing method by applying a thinning algorithm, centering on a pixel whose brightness value is 0. By removing the outermost pixels by 1 pixel and extracting them into a structure having a thickness of 1, 16 × 16 normalized images have 16 rows in the X direction and 16 rows in the Y direction. It has a value from 0 to a maximum of 14.

Backpropagation analysis unit 433 is a device for reducing the error by comparing the target data and the actual data output value of the training data through the teacher teaching model 18, the backpropagation neural learning through forward and backward analysis Perform.

The omnidirectional analysis is an analysis process in which the values calculated by the input function and the active function at each node pass through the hidden layer and output the value at the output layer when an input pattern is input to the input layer to derive an arbitrary target value. .

The backward analysis is an analysis process of calculating the error between the target value and the actual data output value and propagating the error value from the output layer to the input layer and adjusting the connection strength of the weights and biases between the nodes in a direction to minimize the error. .

The backpropagation analysis unit 433 repeats forward analysis and backward analysis until convergence to an error reference value, and performs final character recognition when a feature vector, which is an optimal value, is extracted.

As described above, the component-specific operation of the traffic information collecting device A including the vehicle number recognition unit 400 according to an embodiment of the present invention is performed as described above.

Hereinafter, an operation of receiving and processing the character information finally recognized by the backpropagation analysis unit 433 by the image rendering unit 330 of the multi-resolution processing unit 300 will be described.

The image rendering unit 330 receives the final recognition character information 44 finally analyzed by the backpropagation analysis unit 433, and the resolution setting unit 320 has a current bandwidth from the bandwidth exceeding determination unit 310. When receiving a determination result exceeding the allowable range of the set bandwidth (E1), as shown in Figure 28a, the final character recognition information 44 corresponding to the priority viewport 12, the resolution setting unit 320 ) Receives the determination result that the current bandwidth does not exceed the allowable range of the set bandwidth from the bandwidth exceeding determination unit 310 (E2), as shown in FIG. 28B, the final character corresponding to the entire viewport 36. Recognition information 45 is included in each viewport for synthesis and image processing.

The images rendered by the image rendering unit 330 are transmitted to the encoding unit 340 to be encoded and transmitted to the second data transmission and reception unit 530 of the central control center 500 through the second wireless communication module 351. Since the process is as described above, a detailed description thereof will be omitted.

Referring to FIG. 1, the central control center 500 is a mechanism for comprehensively grasping and managing traffic conditions by receiving image information from the traffic information collecting device A, and as an integrated control server 510 and a database 520. ), A second data transmission / reception unit 530, a decoding unit 540, an output unit 550, a CCTV control unit 560, and a warning unit 570.

The integrated control server 510 is a management server used in a conventional traffic control method and includes a processor that can process and control data transmitted from various traffic collection devices.

The database 520 is a collective storage of all backup data managed by the integrated control server 510 and accumulates and stores big data of the road traffic law violation vehicle image information 10 according to an embodiment of the present invention.

The database 520 may classify and store all image information transmitted from the traffic information collecting device A through an algorithm according to violations of road traffic law such as signal violation, speeding, and parking violation.

As described above, the database 520 may periodically update and wirelessly transmit the road traffic law violation vehicle image information 10 newly added to the memory unit 120 of the traffic information collecting device A. FIG.

The update radio transmission may be manually controlled by an administrator or set to be automatically performed.

The second data transmission / reception unit 530 receives the data transmitted from the traffic information collecting device A and transmits the second wireless communication module 531 to transmit the data wirelessly to the traffic information collecting device A. FIG. It is a communication device included.

As described above, the second data transmission / reception unit 530 receives the encoded high quality compressed image information 34 from the communication unit 350 of the traffic information collecting device A, and transmits it to the decoding unit 540 to be described later. The low-quality video information 33 which is not encoded is received and output to the output unit 550 which will be described later.

The decoding unit 540 is a decompression decoding apparatus corresponding to the encoding unit 340 of the traffic information collecting device A. In general, the decoding unit 540 is an image compression module having a widely commercially available image decoding program.

The decoding unit 540 decodes and encodes the high quality compressed image information 34 encoded and encoded by the encoding unit 340 of the traffic information collecting device A in real time and outputs it to the output unit 550.

Referring to FIG. 29, the output unit 550 is a screen device that outputs the decoded high quality compressed image information 34 and the low quality image information 33 in real time, and includes a screen screen provided in a general traffic control center.

As described above, the CCTV control unit 560 transmits a control command to the control command receiving unit 111 included in the camera unit 110 of the traffic information collecting device A to rotate and image the camera unit 110. It is a remote control device equipped with various control activities such as enlargement, stop, text display, and status check.

The CCTV control unit 560 according to a preferred embodiment of the present invention may configure various embodiments through the action of expanding or stopping high quality image information transmitted to the priority viewport 12.

The CCTV control unit 560 may be provided in a controller control method such as AC, DC pan tilt control, lens control, built-in monitor, 8-channel distribution, all-weather type, Ethernet control, and the like is not limited to the listed types.

Referring to FIG. 30A, the warning unit 570 utilizes a vehicle tracking system based on license plate information of a specific vehicle S of the high-definition compressed image information 34 output from the output unit 550. The warning command may be transmitted through the navigation system N of the specific vehicle S, and the warning lamp 150 may be transmitted by transmitting a warning command to the traffic information collecting device A provided with the security lamp 150. can do.

The navigation system N includes all vehicle-mounted smart devices that can be accessed over the web, and interlocks with the integrated traffic information collection system 600 to receive various received information from the central control center 500. It means a system and apparatus that can be.

Referring to FIG. 30B, the integrated traffic information collecting system 600 accumulates in the database 320 through interaction with a plurality of the traffic information collecting device A and the central control center 500, and the road traffic law violation vehicle image. As a system capable of transmitting and storing the information 10, based on the image information collected through a plurality of traffic information collecting devices (A) as described above using a big data algorithm to classify according to the road traffic law violations Self-learning can be performed.

In other words, deep learning self-learning can be utilized through analysis of vehicle image information using the big data-based algorithm as described above, thereby continuously accumulating image data, thereby significantly reducing false recognition rates.

On the other hand, the smart traffic control method based on the multi-resolution image transmission according to the present invention, as shown in Figure 31, in the smart traffic control method based on big data prediction viewport multi-resolution image transmission through vehicle number recognition, Collecting image input information (S110); Big data analysis that tiles the collected image input information and sets a prediction viewport for a specific vehicle (S) having a high probability of violation by comparing the road traffic law violation vehicle image information and the current vehicle image information through a big data based algorithm. Step S120; A multi-resolution processing step of receiving the prediction viewport set in the big data analysis step (S120) and processing the set prediction viewport in high quality and processing the remaining viewports in low quality; And receiving, decoding and outputting the image information processed through the multi-resolution processing step (S130) at the central control center 500 (S150).

At this time, the big data analysis step (S120), as shown in FIG. 32, in the tiled process (S121) for setting and classifying the current vehicle image information into a tile of a predetermined frame unit, and in the tiled process (S121) Based on the set tiles, the road traffic law violation vehicle image information 10 and the current vehicle image information 11 stored in the memory unit 120 are compared through the big data based algorithm, and the threshold probability value set by the similarity value ( T) Predictive viewport setting process of assigning and arranging the prediction viewport 27 through the viewport analysis process (S122) for determining whether the image is abnormal or not and the viewport setting for the critical probability value (T) determined in the viewport analysis process (S122) ( S123).

In the multi-resolution processing step (S130), as shown in FIG. 33, the prediction viewports 27 arranged in the viewport analysis process (S122) are received in real time and exceed the allowable range of the bandwidth based on the current bandwidth. If the bandwidth exceeding determination process (S131) and the current bandwidth determined in the bandwidth exceeding determination process (S131) exceeds the allowable range, the priority viewport 12 is processed in high quality and the remaining viewports 13 are of low quality. An image rendering process (S133) for rendering and generating image image data through fusion of image quality processed for each viewport in the resolution setting process (S132), and the image rendering process (S133). Receiving and encoding the image data generated in step (S134) and the high-quality image information encoded in the encoding step (S134) of the central control center It may be composed of a process (S135) for real-time wireless transmission prefetching.

That is, as shown in FIG. 34, it is determined whether the similarity value 28 exceeds the threshold probability value T by using the current vehicle image information 11, and if it exceeds, the real-time current bandwidth is within the allowable range of the set bandwidth. If it is determined to exceed, if it exceeds, the priority prioritized viewport 12 is transmitted in high quality, and the remaining viewports 13 are subordinated to low quality according to the remaining bandwidth, thereby transmitting vehicle image information with an optimized traffic amount. The transmission delay can be minimized.

In addition, after the multi-resolution processing step (S130), further includes a vehicle number recognition step (S140), the vehicle number recognition step (S140), as shown in Figure 35, the vehicle image taken through the camera module Extracting the license plate area through edge detection and line propagation process (S141), and separating the number and the character area by labeling the license plate area by removing blobs except the character area through blob detection (S142). In addition, a process of performing character recognition using the deep learning model Deep Belief Network (DBN) algorithm on the separated character region (S143), and the preliminary learning with a limited Boltzmann machine (RBM), which is a comparative company learning method, is performed. And a process of extracting feature vectors (S144), and performing a final character recognition through fine adjustment through a backpropagation neural network, which is a teacher learning method (S145).

A process of extracting feature vectors by performing character learning using the deep learning model deep trust neural network (DBN) algorithm (S143) and limited Boltzmann machine (RBM), which is a comparative company learning method. The remaining steps and processes except for (S144) are the specific configuration and operation of the big data analysis unit 200, the multi-resolution processing unit 300, the vehicle number recognition unit 400, which are the main components of the traffic information collecting device (A) Since it is as described above in the description is omitted.

In the process of performing character recognition on the separated character region using the deep learning model deep neural network (DBN) algorithm (S143), a hop generated based on the EBM encoding the correlation between energy model variables of machine learning based on probability modeling The deep learning model Deep Belief Network (DBN) algorithm, which is a combinatorial optimization algorithm for field neural networks, recognizes characters in the separated text domain.

The deep learning model deep neural network (DBN) algorithm is preferably composed of SBN (Sigmond Belief Network) using the SBN as an activation function of the feature vector to derive a non-cyclic graph with a direction consisting of random variables.

The process of extracting the feature vector by performing prior learning with a limited Boltzmann machine (RBM), which is a non-comparative learning method (S144), is performed between the visible layer and the hidden layer array through the Boltzmann distribution, which is a partition function of the canonical ensemble based on statistical physics. Prior learning is performed using a limited Boltzmann machine (RBM), an algorithm that performs data learning over a connection.

On the other hand, after the reception and decoding after output step (S150), as shown in Figure 31, the integrated vehicle information collection system 600 through the plurality of traffic information collecting device (A) in violation of the vehicle image information ( 10) receiving and storing the accumulated data in the database 320 and classifying the violation according to the violation contents in the database 320 to perform deep learning self-learning (S160); And, based on the violation vehicle information received, decoded and output from the central control center 500 may further comprise the step (S170) to perform automatic vehicle inquiry, penalty charges, bill issuing on-line.

That is, the fine bill online transmission step (S170) includes not only sending a bill online, but also sending an SMS through a text message using a personal information according to the vehicle tracking, sending a text using an SNS account, and the like. .

Through the smart traffic control method and traffic information collection device based on big data prediction viewport multi-resolution image transmission through vehicle number recognition according to the present invention, it is possible to reduce the amount of bandwidth traffic by dynamically determining image quality based on real-time bandwidth. The transmission delay speed can be minimized.

In addition, minimizing the time required for arithmetic processing by directly comparing and analyzing the violation vehicle image information 10 and the current vehicle image information 11 stored in the back data of the memory with the traffic information collecting device A is performed. By doing so, transmission delay can be minimized.

In addition, a specific vehicle (S) having a high probability of violating the road traffic law detected by using the prediction viewport 27 technology may be alerted and notified through lighting, voice, SMS, and a navigation system (N). Extension is possible.

The present invention described above has been described with reference to one embodiment shown in the drawings, but this is merely exemplary and various modifications and equivalent other embodiments are possible to those skilled in the art. Able to know. Therefore, the true technical protection scope of the present invention should be interpreted by the appended claims, and all technical ideas within the equivalent scope should be construed as being included in the scope of the present invention.

A: traffic information collection device 230: viewport setting unit
110: camera unit 300: multi-resolution processing unit
120: memory 310: bandwidth exceeded determination unit
130: first data transmission and reception unit 320: resolution setting unit
140: housing 330: image rendering unit
150: security light 340: encoding part
160: CPU 350: communication unit
200: big data analysis unit 400: vehicle number recognition unit
210: tile setting unit 500: central control center
220: viewport analysis unit 600: integrated traffic management system

Claims (10)

In the smart traffic information collection apparatus based on big data prediction viewport multi-resolution image transmission by vehicle number recognition,
Pan tilt housing camera unit 110 for collecting the video input information of the video;
A memory unit 120 that periodically receives and classifies and stores road traffic law violation vehicle image information 10 classified and stored in the database 520 of the central control center 500;
A first data transmission / reception unit 130 provided to periodically update the road traffic law violation vehicle image information 10 classified and stored in the database 520 of the central control center 500;
Big data analysis unit 200 for receiving a high-resolution CCTV image in the bitstream to analyze the big data viewport prediction technology;
Of the vehicle information analyzed by the big data analysis unit 200, the image data of the vehicle having a high possibility of violating the road traffic law is processed in high quality through the set prediction viewport 27, and the remaining background image except the prediction viewport 27 is processed. And a multi-resolution processor 300 for real-time transmission of the objects to the central control center 500 by processing the objects with low quality. And,
The CPU 160 is embedded in the traffic information collecting device A and performs calculation and control for data processing of the big data analysis unit 200 and the multi-resolution processing unit 300.
The big data analysis unit 200,
A tile setting unit 210 for dividing and setting the current vehicle image information 11 collected from the camera unit 110 into tiles 16 which are a predetermined frame unit;
Based on the tile 16 set by the tile setting unit 210, the road traffic law violation vehicle image information 10 and the current vehicle image information 11 stored in the memory unit 120 are compared through a big data based algorithm. A viewport analyzer 220 determining whether the similarity value 28 of the violating vehicle image information 10 corresponding to the specific vehicle S according to the compared vehicle image information value is equal to or greater than a threshold probability value T set by the comparison vehicle image information value;
It is composed of a viewport setting unit 230 to give and arrange the prediction viewport 27 through the viewport setting whether the threshold probability value (T) determined by the viewport analysis unit 220,
The multi-resolution processing unit 300,
A bandwidth exceeding determination unit 310 that receives the prediction viewports 27 arranged by the viewport setting unit 230 in real time and determines whether the allowable range of the bandwidth is exceeded based on the current bandwidth;
If the current bandwidth determined by the bandwidth over-determination unit 310 exceeds the allowable range, the prediction viewport 27 is set as the priority viewport 12 to process high quality and the remaining viewports 13 are processed to low quality. A resolution setting unit 320,
An image renderer 330 for rendering and generating image image data through fusion of image quality processed by the viewport by the resolution setting unit 320;
Encoding unit 340 for receiving and encoding the image data generated by the image rendering unit 330 and high-quality image information encoded by the encoding unit 340 to transmit the real-time wireless preloading to the central control center 500 Is composed of a communication unit 350,
And a vehicle number recognition unit 400 for recognizing the vehicle number and transmitting the image to the image rendering unit 330 through the current vehicle image information 11 of the image quality processed by the viewport by the resolution setting unit 320.
The vehicle number recognition unit 400,
A license plate candidate area detection unit 410 for converting the input RGB color vehicle image into a gray scale image and detecting a license plate area through the Kenny edge detector method 37 and the LSD algorithm 38;
A character region detector 420 for detecting a character region through local edge-based binary preprocessing, vertical filtering, CLNF, and blob analysis of the license plate candidate region 15 detected by the vehicle license plate candidate region detector 410;
Big data prediction through vehicle number recognition, comprising a license plate character recognition unit 430 that recognizes characters using image normalization, pixel matrix, and DBN through the character region detected by the character region detector 420. Smart traffic information collection device based on viewport multi-resolution image transmission.
delete The method of claim 1,
The central control center 500 is to grasp the traffic situation of the drive in real time and to manage and control the traffic flow,
Integrated control server 510 for receiving and processing the traffic situation as real-time video information,
A database 520 for collecting and storing data processed by the integrated control server 510;
A data transmission and reception unit 530 which receives image information on a real-time traffic situation and transmits a control command to the traffic information collecting device A;
A decoding unit 540 for decoding image information encoded and processed in high quality through a big data viewport prediction technology;
An output unit 550 for displaying the decoded image information on a screen;
CCTV control unit 560 for generating a control command to be transmitted through the data transmission and reception unit 530,
The warning unit 570 may be configured to warn the specific vehicle S in real time based on the high-definition image information of the specific vehicle S which has a high possibility of violating the road traffic law recognized by the output unit 550. Smart traffic information collection device based on big data prediction viewport multi-resolution image transmission by vehicle number recognition.
delete The method of claim 3,
The control unit built in the camera unit 110 receives a control command of the CCTV control unit 560 and can perform a series of commands including angle adjustment, image magnification, and image stop of the camera unit 110 according to the control command. Big traffic prediction viewport multi-resolution image transmission based smart traffic information collection device using a vehicle number recognition, characterized in that the control command receiving unit 111 is configured to be provided.
delete A smart traffic control method based on big data prediction viewport multi-resolution image transmission using vehicle number recognition,
Collecting image input information from a video (S110);
Tile the collected image input information and compare the road traffic law violation vehicle image information with the current vehicle image information through a big data based algorithm to predict the viewport 27 for a specific vehicle S having a high possibility of violation of the road traffic law. Big data analysis step (S120) to set;
Multi-resolution for receiving the prediction viewport 27 set in the big data analysis step (S120) and processing the set prediction viewport 27 as the priority viewport 12 as a high quality image and processing the remaining viewports 13 as low quality. Processing step 130; And,
Receiving the image information processed through the multi-resolution processing step (S130) at the central control center 500, decoding and outputting (S150),
The big data analysis step (S120),
A tileization process of classifying and setting current vehicle image information into tiles that are a unit of a predetermined frame (S121);
Based on the tile set in the tiled process (S121), the road traffic law violation vehicle image information 10 and the current vehicle image information 11 stored in the memory unit 120 are compared and compared using a big data based algorithm. A viewport analysis process (S122) for determining whether the similarity value 28 is equal to or greater than a threshold probability value T set;
It consists of a prediction viewport setting process (S123) to give and arrange the prediction viewport 27 through the viewport setting whether the threshold probability value (T) determined in the viewport analysis process (S122),
The multi-resolution processing step (S130),
An excess bandwidth determination step (S131) of receiving the prediction viewports 27 arranged in the viewport analysis process (S122) in real time and determining whether the allowable range of the bandwidth is exceeded based on the current bandwidth;
When the current bandwidth determined in the bandwidth exceeding determination process (S131) exceeds the allowable range, a resolution setting process of processing the prediction viewport 27 as the priority viewport 12 in high quality and performing the low quality processing of the remaining viewports 13 S132),
An image rendering process (S133) for rendering and generating image image data through fusion of image quality processed for each viewport in the resolution setting process (S132);
Receiving and encoding the image data generated in the image rendering process (S133) (S134);
It consists of a process of transmitting the real-time wireless preloading the high-definition video information encoded in the encoding process (S134) to the central control center (S135),
After the multi-resolution processing step (S130), further comprising a vehicle number recognition step (S140),
The vehicle number recognition step (S140)
Extracting the license plate area from the current vehicle image information 11 of the viewport received from the multi-resolution processor 300 through edge detection and line propagation (S141);
(B) separating the number and the text area by labeling the label plate area extracted in the license plate area extraction process (S141) by removing blobs except the text area by blob detection and labeling;
Big through the vehicle number recognition, characterized in that consisting of the process of performing the character recognition using the back propagation neural network algorithm based on the separated character area through the process of separating the number and the character area (S142). Smart traffic control method based on data prediction viewport multi-resolution image transmission.
delete The method of claim 7, wherein
After the process of performing the character recognition (S143),
A process of extracting feature vectors by performing prior learning with a limited Boltzmann machine (RBM) (S144),
Smart data based on the big data prediction viewport multi-resolution image transmission through vehicle number recognition, further comprising the step (S145) of performing the final character recognition through fine tuning through the deep learning model deep trust neural network (DBN) algorithm Traffic control method. delete

Images