KR102537818B1 - Traffic information management system for enhancing accuracy of sensing vehicle speed - Google Patents

Abstract

The present invention relates to a traffic information management system, which can accurately determine vehicle speed regardless of the vehicle driving speed by detecting the speed of vehicles using a radar sensor or LiDAR sensor, wherein when the vehicles are driving at low speeds, the system is configured to detect the speed of the vehicle through the image of a camera so the radar sensor or LiDAR sensor with excellent speed detection accuracy is used at normal times while when the vehicles are driving at low speeds to be close to each other, the system is configured to detect the speed of the vehicles through the image analysis of a camera with excellent object discrimination performance, and also can ensure the accuracy and reliability of vehicle speed detection through camera image analysis by being configured to calculate vehicle speed using the rate of change of a license plate image in the image captured by the camera.

Description

Traffic information management system for enhancing accuracy of sensing vehicle speed}

The present invention relates to a traffic information management system with improved vehicle speed detection accuracy, and more particularly, generates vehicle information using a sensing means such as a radar sensor and a lidar sensor, but detecting means having poor resolution in the case of a low-speed vehicle. Considering the characteristics of the object, the vehicle speed is calculated through image analysis of a camera with excellent object discrimination performance, and at the same time, the vehicle speed is calculated by utilizing the variation ratio of the license plate image in the video captured by the camera. By selecting an analysis method selectively according to the method, not only can the accuracy and reliability of vehicle speed detection and vehicle information generation be increased, but also vehicle speed can be calculated using images without setting a separate reference table in advance. Therefore, it relates to a traffic information management system that can be used immediately in various sites in conjunction with various installation heights and camera pan-tilt angles.

As automobiles become more popular and road penetration increases exponentially, vehicle information is collected using vehicle detection means, and traffic information such as vehicle density, congestion rate, vehicle speed, and violating vehicles is generated based on the collected vehicle information. An Intelligent Transportation System (ITS) is widely used to comprehensively manage and monitor generated traffic information.

As a vehicle detection method for collecting such vehicle information, a method using a radar signal and a method using a camera are commonly used.

The method using the radar signal is a method of collecting vehicle information by analyzing the collected reflected signal after collecting the reflected signal of the radar signal transmitted to the preset detection area, and has excellent detection accuracy for the vehicle position and speed. However, it is less affected by the external environment and has excellent object detection in the longitudinal direction. However, the method using such a radar has a disadvantage in that the accuracy of position and speed detection in the lateral direction and object classification and information detection is low.

The method using a camera is a device for collecting vehicle information by analyzing image information obtained by shooting a camera, and has excellent object classification and excellent accuracy in object classification and information detection, and position and speed in the lateral direction Detection has the advantage of being excellent. However, the method using the camera is easily affected by the external environment, and has a disadvantage in that detection accuracy for distance and speed is relatively low compared to radar signals.

Accordingly, various studies are being conducted to detect vehicle information by manufacturing a camera and a radar transceiver as an integral type and analyzing image information and radar signals.

1 is a block diagram showing a vehicle detector disclosed in Korean Patent Registration No. 10-1446546 (title of invention: location-based real-time vehicle information display system).

The vehicle detector (hereinafter referred to as the prior art) 100 of FIG. 1 includes a database unit 102 for storing data, a data transmission/reception unit 103 for transmitting and receiving data, and a radar signal received from the radar transmission/reception unit 110. Based on the vehicle trajectory information detected by the vehicle trajectory detection unit 104 that detects the trajectory of each vehicle by analyzing the vehicle trajectory detection unit 104, the speed, location, driving direction and distance between vehicles in front of each vehicle, etc. When a violating vehicle is detected by comparing the vehicle information detecting unit 105 for detecting vehicle information including the vehicle information and the vehicle speed and the distance between the vehicle in front of the vehicle information detected by the vehicle information detecting unit 105 to the preset speed limit and safety distance, The location, speed, driving direction, and distance between vehicles in front detected by the unexpected situation generating unit 106 and the vehicle information detection unit 105, and information about the violating vehicle detected by the unexpected situation generating unit 106 Controls the location-based vehicle information generation unit 107 and the radar transmission/reception unit 110 that generate location-based vehicle information by matching information, terminal identification ID value, location information of the detection area (S) and location information of the shooting area A radar driving unit 108 that controls the photographing means 120, a photographing means driving part 109 that controls the photographing means 120, and these control targets 102, 103, 104, 105, 106, and 107 , (108), (109) is composed of a control unit 101 for controlling.

In addition, the image information obtained by the photographing of the photographing means 120 is transmitted to the traffic management server, and the traffic management server analyzes the image information to analyze the vehicle location information, and then converts the vehicle location information into a location-based vehicle information generator 107 ) to match the location-based vehicle information generated by

In this way, the prior art 100 performs the radar detector 110 and the photographing means 120 at the same time to collect information, and then matches the collected vehicle information and displays it on a terminal through the traffic information user interface 200 to provide users with information. More accurate traffic information can be provided.

In general, since the resolution of the radar signal has an error range of 30 to 70 cm, in a field where vehicles are congested and consecutive vehicles are concentrated, due to errors and errors such as detecting two or more vehicles as one vehicle, The accuracy of vehicle speed detection is low.

That is, since the prior art 100 simply detects an object using only a radar signal, in an environment where a vehicle is congested, errors and errors occur in vehicle detection, resulting in a structural limitation in which the accuracy of vehicle speed detection is reduced.

In order to overcome this problem, various studies are being conducted on a vehicle detection system using a dual detection means to detect a vehicle using a separate detection means such as a camera in addition to a radar sensor.

In particular, as image analysis technology has recently developed and cameras have been advanced, research on vehicle detection systems using image analysis has been actively conducted.

2 is a conceptual diagram disclosed in Korean Patent Publication No. 10-2017-0088692 (Title of Invention: Apparatus and Method for Calculating Vehicle Speed Using Images).

In the apparatus for calculating vehicle speed using the image of FIG. 2 (hereinafter referred to as the second prior art) 200, the vehicle 120 is photographed by the photographing unit 210, and the first image photographed at the first time point t1. And, the traveling speed of the vehicle is calculated through a second image captured at a second time point t2 after a predetermined time has elapsed from the first time point t1.

At this time, when the width of the vehicle photographed in the first image is the first width and the width of the vehicle photographed in the second image is the second width, the amount of change in the width photographed in the image has a certain correlation with the actual mileage of the vehicle. Therefore, it is possible to calculate the mileage of the vehicle by utilizing the amount of change in the width captured in the image.

Due to this, the second prior art 200 can calculate the speed of the vehicle through the mileage and the time difference (Δt=t2-t1) of the image.

However, since the second prior art 200 does not have a means for measuring the distance between the camera and the vehicle photographed at the first point in time t1, the mileage according to the change in the width of the vehicle is prepared by actual measurement rather than a calculation formula. It is configured to be calculated by the table.

As a result, in the second prior art 200, when a camera is installed under conditions different from the existing table created conditions (camera height, angle, distance, etc.), a process of actual measurement is additionally performed to create a new table. Not only does the time taken to install and apply the photographing unit 210 increase, but also has a problem in that the measured vehicle speed and the actual vehicle speed are measured differently when an error occurs during the actual measurement process.

In addition, in the second prior art 200, when the shooting angle of the photographing unit 210 is changed due to external vibration, shock, wind, etc., a vehicle measured according to a change in a previously set table and photographing conditions due to the changed photographing angle It has a problem that an error occurs in the speed of .

That is, even if the photographing unit 210 of FIG. 2 is fused to the radar transceiver 110 of the prior art 100 of FIG. 1 described above, in order to implement the converged invention, the height, angle, distance, etc. of the camera Installation is limited because a reference table according to the installation environment must be designed separately in advance, and if an error occurs during the actual measurement process, the measured vehicle speed and the actual vehicle speed are measured differently.

The present invention is to solve this problem, and the problem of the present invention is to detect the speed of vehicles using a radar sensor or lidar sensor, but when the vehicles are traveling at low speed, the speed of the vehicle through the image of the camera As it is configured to detect, a radar sensor or lidar sensor with excellent speed detection accuracy is used in normal times, but when vehicles are driving at low speed and are driving close to each other, the speed of the vehicle is determined through image analysis of a camera with excellent object discrimination performance. It is an object of the present invention to provide a traffic information management system capable of accurately detecting vehicle speed regardless of the driving speed of vehicles by detecting it.

In addition, another problem of the present invention is configured to calculate the speed of the vehicle by utilizing the rate of change of the license plate image in the image captured by the camera, thereby securing the accuracy and reliability of vehicle speed detection by camera image analysis. Traffic information It is intended to provide a management system.

In addition, another problem of the present invention is configured to continuously acquire reference distance information, which is information for measuring the mileage of a vehicle, by a sensor, so that it can be installed at various angles and heights, as well as pan-tilt the camera. It is an object of the present invention to provide a traffic information management system capable of obtaining the mileage of a vehicle even when the angle and height are changed by external vibration or impact.

In addition, another problem of the present invention is to extract a plurality of frames from the time when the vehicle is detected, and to determine the license plate images captured in the extracted frames and the size of the license plate image in the first frame, which is the frame at the time when the vehicle is detected. Traffic information that can prevent errors from occurring in the process of calculating the driving speed by analyzing a plurality of frames even when the license plate is not accurately captured due to camera vibration or impact in some frames by measuring the driving speed of the vehicle by comparing them It is intended to provide a management system.

The solution of the present invention for solving the above problems is a traffic information management system including at least one photographing unit for detecting a vehicle in a preset area and then calculating the speed of the detected vehicle: sensing means for detecting; a camera that captures a preset area; The vehicle is detected by analyzing the data input from the sensing means and the camera, and then the speed of the sensing vehicle is detected, and the speed of the sensing vehicle is detected by analyzing the sensing data received from the sensing means. a controller for calculating the speed of the vehicle by analyzing the image information obtained from the camera when is less than a preset value; When detecting a vehicle, the detection signal is analyzed to calculate the detection distance (Ds), which is a straight-line distance on a plane from the camera to the detected vehicle, and the detection information including the calculated detection distance (Ds) information is generated and transferred to the controller. A memory for storing a license plate extraction algorithm, which is an algorithm for extracting a vehicle license plate from the detected vehicle object after detecting a vehicle object from an input image; a sensing means-based speed detection unit for detecting a speed of the vehicle by analyzing the sensing information obtained from the sensing means; an image-based speed detector for detecting the speed of the vehicle by analyzing the image information obtained from the camera; The first vehicle speed C1, which is the speed of the vehicle detected by the sensing unit-based speed detection unit, is determined as a final speed value, and when the first vehicle speed C1 is equal to or less than a preset threshold speed value, the image-based speed detection unit determines A speed determination unit for determining a second vehicle speed C2, which is the detected vehicle speed, as a final speed value, and the image-based speed detection unit determines the time point t1 when the vehicle is detected from the image obtained by the camera. a frame extraction unit for extracting a first frame F1, which is a frame, and a second frame F2, which is a frame at a time point t2 when a preset time T1 has elapsed; License plate images (B1) and (B2) are extracted from the first and second frames (F1) and (F2) extracted by the frame extraction unit, respectively, and the extracted first and second license plate images (B1) and (B2) are extracted. After calculating the variation ratio (Z) using ), considering the characteristic that the mileage (Dx) of the detected vehicle increases in proportion to the variation ratio (Z), from the calculated variation ratio (Z) and the auxiliary sensor a frame analyzer that calculates a travel distance Dx of the detected vehicle by utilizing the input detected distance Ds; and a traveling speed calculating unit for calculating the traveling speed v of the vehicle by dividing the traveling distance Dx of the detected vehicle calculated by the frame analyzing unit by the set time T1.

delete

In the present invention, the speed determination unit includes a speed value extraction module for extracting a first vehicle speed C1 and a second vehicle speed C2 from the sensing means-based speed detection unit and the image-based speed detection unit; a comparison module that compares the first vehicle speed C1 with a preset threshold speed value; When the first vehicle speed C1 is greater than or equal to the preset threshold speed value, the first vehicle speed C1 is determined as the final speed value, but when the first vehicle speed C1 is less than the threshold speed value, the second vehicle speed ( It is preferable to include a decision module for determining C2) as the final speed value.

delete

In addition, in the present invention, the frame analysis unit detects a vehicle object from the first frame (F1) using the license plate extraction algorithm, and then extracts the first license plate image (B1) from the detected vehicle object, and the license plate extraction algorithm License plate image extraction module for extracting the second license plate image (B2) from the second frame (F2) by using; After calculating the sizes of the first and second license plate images (B1) and (B2) extracted by the license plate image extraction module, the size of the first and second license plate images (B1) and (B2) calculated is used to change the size. a variable ratio calculation module that calculates a ratio (Z); Considering the characteristic that the driving distance (Dx) of the detected vehicle increases in proportion to the variation ratio (Z), the variation ratio (Z) calculated by the variation ratio calculation module and the sensing distance (Ds) input from the auxiliary sensor It is preferable to include a mileage calculation module that calculates the mileage Dx of the detected vehicle by utilizing .

In addition, in the present invention, the frame analysis unit further includes a pixel number calculation module executed after the license plate image extraction module, and the pixel number calculation module is the width of the first license plate image B1 extracted by the license plate image extraction module. A first number of horizontal pixels (Pb1), which is the number of pixels in a direction, and a second number of horizontal pixels (Pb2), which is the number of pixels in a horizontal direction of the second license plate image (B2) extracted by the license plate image extraction module, are calculated, and the variation The ratio calculation module divides the second number of horizontal pixels (Pb2) calculated by the number of pixels calculation module by the number of first horizontal pixels (Pb1) calculated by the number of pixels calculation module to calculate the variation ratio (Z). it is desirable

In the present invention, the mileage calculation module substitutes the variation ratio (Z) calculated by the variation ratio calculation module and the sensing distance (Ds) input from the auxiliary sensor into the following Equation 1 to determine the mileage of the vehicle ( Dx) is preferably calculated.

In addition, in the present invention, the frame extraction unit determines the time point (t3, t4, ..., tn + 1) when the predetermined set times (T2, T3, ..., Tn) have elapsed from the time point (t1) when the vehicle is detected by the auxiliary sensor The frames (F3, F4, ..., tn) are further extracted, and the frame analysis unit extracts license plate images from the additionally extracted frames (F3, F4, ..., tn) from the frame extraction unit, respectively, and extracts By calculating the number of horizontal pixels of the license plate images and substituting them into Equation 1, the traveling distances (Dx2, Dx3, ..., Dxn) of the detected vehicle are additionally calculated, and the traveling speed calculation unit detects the detection calculated by the frame analysis unit. Divide each mileage (Dx, Dx2, Dx3, ..., Dxn) of the vehicle by the corresponding set time (T1, T2, T3, ..., Tn) to calculate the vehicle's traveling speed (v1, v2, ..., vn) , After calculating the average driving speed (va), which is the average value of the calculated driving speeds (v1, v2, ..., vn), the calculated average driving speed (va) is determined as the driving speed (v) of the detected vehicle. desirable.

In addition, in the present invention, the frame analysis unit extracts the first license plate image (B1) from the first frame (F1) by utilizing the license plate extraction algorithm, and extracts the second license plate image (B2) from the second frame (F2). License plate image extraction module to extract; From the first license plate image B1, a first left side pixel number L1, which is the number of pixels on the left side, and a first right side pixel number R1, which is the number of pixels on the right side, are extracted, and the second license plate image B2 a pixel number calculation module for calculating a second left side pixel number (L2), which is the number of pixels on the left side, and a second right side pixel number (R2), which is the number of pixels on the right side, from ); The variation rate of the left side is calculated by dividing the second left side pixel number L2 by the first left side pixel number L1, and the calculated left side variation rate and the sensing distance Ds input from the auxiliary sensor are calculated by the following arithmetic The distance value D2 between the left sides is calculated by substituting into Equation 2, and the variation rate of the right side is calculated by dividing the second right side pixel number R2 by the first right side pixel number R1, and the calculated right side variation rate and the sensing distance Ds input from the auxiliary sensor are substituted into the following Equation 2 to calculate the right edge distance value D3, and the calculated left edge distance value D2 and the right edge distance value D3 After calculating the average distance value (D4), which is the average value, and then calculating the calculated average distance value (D4), including a mileage calculation module for determining the calculated average distance value (D4) as the mileage (Dx) it is desirable

In addition, in the present invention, the location information of the optimal region, which is the position on the frame of the second license plate image (B2), which can increase the accuracy of image analysis, is preset and stored in the memory, and the controller is configured for every preset period (T'). A set time optimizer further comprising: an optimal distance calculation module for calculating an optimal distance, which is a linear distance on a plane with the camera on an actual road matched to the optimal region; The driving speed (v) data of the vehicles calculated by the driving speed calculator during the period (T') is extracted from the memory, and the average driving speed (V), which is the average value of the extracted driving speeds (v), is calculated. driving speed calculation module; an average traveling distance calculating module that calculates an average traveling distance (Da) obtained by multiplying the average traveling speed (V) calculated by the average traveling speed calculating module and a predetermined set time (T1); an average distance calculation module that calculates an average distance by subtracting the average travel distance Da calculated by the average travel distance calculation module from the detection distance Ds; a difference calculation module for calculating a difference value by subtracting the optimum distance calculated by the optimum distance calculation module from the average distance calculated by the average distance calculation module; The absolute value of the difference value calculated by the difference value calculation module is compared with the threshold distance value, which is the minimum value of the difference value that can determine that the set time T1 needs to be reset, and 1) the absolute value of the difference value is critical If it is less than the distance value, it is determined that there is no need to reset the setting start (T1), 2) if the absolute value of the difference value is greater than or equal to the threshold distance value, a second determination module that determines that the setting time (T1) needs to be reset; It is executed when it is determined that the resetting of the set time T1 is necessary by the second determination module. 1) If the difference value is a positive number, the set time T1 is increased by a predetermined constant 'α', and 2) the difference If the value is a negative number, a setting time resetting module that reduces the setting time T1 by a constant 'α' is included.

In addition, in the present invention, the optimal distance calculation module extracts the second frames F2 collected during the period T', and then, among the extracted second frames F2, the second license plate image is placed in a preset optimal area. a selection frame extraction module for extracting selection frames, which are frames; After extracting the analysis data by the selection frames extracted by the selection frame extraction module, the second sensing distance information, which is a straight-line distance on a plane from the camera to the second license plate image, from the analysis data of each selection frame extracted. A second sensing distance extraction module for extracting; It is preferable to include an optimum distance determination module that calculates an average value of the second sensing distances extracted by the second sensing distance extraction module and then determines the average value of the calculated second sensing distances as an optimum distance.

According to the present invention having the above problems and solutions, the speed of vehicles is detected using a radar sensor or lidar sensor, but when the vehicles are traveling at low speed, the speed of the vehicle is detected through the image of the camera, so that it is normally A radar sensor or lidar sensor with excellent speed detection accuracy is used, but when vehicles are driving at low speeds and are driving close to each other, the vehicle speed is detected through image analysis of a camera with excellent object discrimination performance, thereby detecting the driving speed of vehicles. Regardless, it is possible to accurately detect the vehicle speed.

In addition, according to the present invention, it is possible to secure the accuracy and reliability of vehicle speed detection by camera image analysis by being configured to calculate the speed of the vehicle by utilizing the rate of change of the license plate image in the image captured by the camera.

In addition, according to the present invention, reference distance information, which is information for measuring the mileage of a vehicle, is configured to be continuously obtained by a sensor, so that it can be installed at various angles and heights, as well as the pan-tilt angle and height of the camera. The mileage of the vehicle can be obtained even when is changed by external vibration or impact.

In addition, according to the present invention, a plurality of frames are extracted from the time when the vehicle is detected, and the license plate images captured in the extracted frames are compared with the size of the license plate image in the first frame, which is the frame at the time when the vehicle is detected. By measuring the driving speed of the camera, it is possible to prevent errors from occurring in the process of calculating the driving speed by analyzing a plurality of frames even when the license plate is not accurately photographed due to vibration or impact of the camera in some frames.

1 is a block diagram showing a vehicle detector disclosed in Korean Patent Registration No. 10-1446546 (title of invention: location-based real-time vehicle information display system).
2 is a conceptual diagram disclosed in Korean Patent Publication No. 10-2017-0088692 (Title of Invention: Apparatus and Method for Calculating Vehicle Speed Using Images).
3 is a block diagram of a traffic information management system according to the present invention.
FIG. 4 is an exemplary view for explaining the photographing unit of FIG. 3 .
5 is another exemplary view of FIG. 3 .
6 is another exemplary view of FIG. 3 .
7 is a block diagram of the controller of FIG. 2;
FIG. 8 is a block diagram of a sensing means-based speed detection unit of FIG. 7 .
FIG. 9 is a block diagram of a speed determination unit of FIG. 7 .
FIG. 10 is a block diagram of the image-based speed detection unit of FIG. 7 .
FIG. 11 is a block diagram of a frame analysis unit of FIG. 10 .
12 is an exemplary diagram of a first frame.
13 is an exemplary diagram of a second frame.
14 is an exemplary diagram for explaining a process of calculating vehicle speed.
15 is an exemplary diagram for explaining a process of calculating Equation 2.
FIG. 16 is a block diagram of a second controller that is a second embodiment of the controller of FIG. 7 .
FIG. 17 is a block diagram of a second video-based speed detection unit that is a second embodiment of the video-based speed detection unit of FIG. 16 .
FIG. 18 is a block diagram of a third frame analyzer that is a third embodiment of the frame analyzer of FIG. 11 .
19 is a block diagram of a second mileage calculation module of FIG. 18 .
20 is an exemplary view of a license plate for explaining a process of calculating a mileage by a third frame analyzer.
FIG. 21 is a block diagram of a third image-based speed detector that is a third embodiment of the image-based speed detector of FIG. 10 .
22 is a block diagram of a setup time optimization unit of FIG. 21 .
FIG. 23 is a block diagram of the optimal distance calculation module of FIG. 22 .
24 is an exemplary diagram for explaining an optimal distance.
25 is an exemplary view of a vehicle photographed from the side.

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

3 is a block diagram of a traffic information management system according to the present invention.

The traffic information management system 1, which is an embodiment of the present invention, generates vehicle information using sensing means such as a radar sensor and lidar sensor, but in the case of a low-speed vehicle, considering the characteristics of the sensing means with low resolution, the object The analysis method selectively according to the vehicle speed by calculating the vehicle speed through image analysis of a camera with excellent discrimination performance and at the same time calculating the vehicle speed by utilizing the variation ratio of the license plate image within the video captured by the camera. By selecting , not only can the accuracy and reliability of vehicle speed detection and vehicle information generation be increased, but also vehicle speed can be calculated using images without setting a separate reference table in advance, so various installation heights and camera pans -It is to be used immediately in various fields in conjunction with the tilt angle.

As shown in FIG. 3, the traffic information management system 1 is installed in various places on the road where vehicles are driven, detects vehicles in a preset detection area, generates vehicle information, and simultaneously detects and generates vehicle information. photographing units (3-1), ..., (3-N) that transmit the obtained vehicle information and violation information to the central control server (6), and the photographing unit (3-1), ..., After generating traffic information by processing the vehicle information transmitted from (3-N), the generated traffic information is provided to connected clients, and at the same time, when violation information is received, fines are imposed on the vehicle in violation It consists of a central control server 6 that performs subsequent procedures, and a communication network 7 that supports data communication between the central control server 6 and the photographing units 3-1, ..., and (3-N).

At this time, the traffic information management system 1 of the present invention is preferably installed in sections where there are many vehicle conflicts, law violations, or traffic accidents, such as intersections, branch sections, and merged sections.

The communication network 7 provides a data movement path between the central control server 6 and the photographing units 3-1, ..., (3-N), in detail, a wide area network (WAN), Various known communication methods such as a communication network and a wired communication network may be applied.

FIG. 4 is an exemplary view for explaining the photographing unit of FIG. 3 , FIG. 5 is another example view of FIG. 3 , and FIG. 6 is another example view of FIG. 3 .

The photographing units (3-1), ..., (3-N) are installed all over the road to detect vehicles in a preset detection area, and then generate vehicle information and violation information, and generated vehicle information and violation information. is transmitted to the traffic control server (6).

In addition, the photographing units 3-1, ..., (3-N) include a sensing means 301 for detecting a preset area shown in FIG. 4, a camera 302 for capturing a preset area, and a preset It consists of an auxiliary sensor 303 for detecting a vehicle passing through the detection point P, a detection means 301, a camera 302, and a controller 30 for analyzing the information obtained by the auxiliary sensor 303.

At this time, in the drawing of the present invention, for convenience of explanation, it has been described as an example that the photographing unit 3 photographs the front of the vehicle and calculates the traveling speed of the vehicle, but the photographing unit 3 photographs the rear of the vehicle The driving speed of the vehicle can be calculated.

The detection means 301 is a means for detecting a vehicle object in a preset detection area, and is preferably a radar sensor or lidar sensor.

The sensing unit 301 outputs a sensing signal of the sensed vehicle object to the controller 30 .

For example, when the sensing means 301 is a radar sensor, the radar sensor transmits a radar signal to the sensing area, receives the reflected signal, and outputs the transmitted and received radar signal information to the controller 30. It can be.

At this time, the detection signal by the detection unit 301 is used by the controller 30 to generate vehicle information including the vehicle speed.

The camera 302 is installed on a structure such as a gantry on a road to photograph a preset detection area A.

At this time, the camera 302 acquires an image by photographing the detection area A including the road on which the vehicle is traveling.

In addition, the camera 302 outputs the obtained image to the controller 30, and in the case of the image taken by the camera 302, it is used to generate vehicle information including the speed of vehicles traveling at low speed.

The auxiliary sensor 303 includes a radar emitting unit (not shown) that transmits a laser signal to the detection point P, a radar receiving unit (not shown) that receives a signal transmitted and reflected by the radar emitting unit, and a laser transmitted and received. It consists of a controller (not shown) that analyzes the signal to determine whether a vehicle object is detected, and when the vehicle is detected, generates detection information and transmits it to the controller 30 .

At this time, the sensing information generated by the controller includes the sensing distance Ds, which is a linear distance on a plane from the photographing unit 3 to the sensing point P, and the first point of time, which is the time the vehicle entered the sensing point P ( It includes the information of t1).

In addition, the auxiliary sensor 303 is installed to have the same pan-tilt angle as the camera 302 .

The auxiliary sensor 303 is 1) integrally combined with the camera 302 and installed at the same pan-tilt angle, and 2) continuously detecting distance Ds, which is the distance information to the sensing point P, by a laser signal. Even if the installation angle of the camera 302 is changed by external shock or vibration and the detection point P, which is the point detected by the auxiliary sensor 303, is changed, up to the changed detection point P'. The sensing distance (Ds') information of can be obtained.

At this time, in the present invention, for convenience of explanation, the auxiliary sensor 303 has been described as being a laser sensor as an example, but the auxiliary sensor 303 may be composed of a radar or the like.

The controller 30 temporarily stores the detection signal obtained by the sensing means 301 and the image obtained by the camera 302 and transmits the obtained image to the central control server 6 at the same time. At this time, the period of temporarily storing the detection signal and the image in the controller 30 may be preset or input by a manager.

In addition, the controller 30 calculates the vehicle speed by utilizing the detection signal obtained from the sensing means 301 and at the same time analyzes the image taken from the camera 302 to calculate the vehicle speed.

At this time, the controller 30 determines that the reliability of the vehicle speed obtained by analyzing the detection signal is low when the vehicle speed obtained by using the detection signal of the detection unit 301 is less than a predetermined threshold speed value, and the camera 302 The vehicle speed calculated by analyzing the video is determined as the final speed.

That is, the controller 30 1) calculates vehicle information using the speed of the vehicle and the detection signal of the detection means 301 having excellent detection accuracy of the speed when the vehicles are traveling at high speed and the distance between vehicles is large, 2) When vehicles are driving at a low speed, vehicle information is calculated using an image of the camera 302, which has excellent object discrimination performance and can distinguish adjacent vehicles.

Through this, the controller 30 analyzes the image of the camera 302 and calculates the vehicle speed when it is difficult to detect by the radar or lidar, which is the sensing means 301, because the distance between vehicles is reduced due to vehicles traveling at a low speed. Reliability of the calculated vehicle speed can be secured by determining the final speed.

In addition, the controller 30 uses the sensing means 301 and the camera 302 at the same time, so that the range in which vehicle information can be obtained is increased compared to the case of using only the camera 302 having a relatively narrow field of view (FOV). As a result, it is possible to obtain vehicle information by detecting an intersection requiring a wide range of detection or a shoulder that is difficult to check with the camera 302 through the detection means 301.

7 is a block diagram of the controller of FIG. 2;

As shown in FIG. 7, the controller 30 includes a control unit 31, a memory 32, a communication interface unit 33, a data input/output unit 34, a monitoring means-based speed detection unit 35, and an image-based speed It consists of a detection unit 36, a speed determination unit 37, and a violation vehicle enforcement unit 38.

The control unit 31 is the O.S (Operating System) of the controller 30, and manages and controls the control targets 32, 33, 34, 35, 36, 37, and 38. do.

In addition, the control unit 31 outputs the sensing signal input from the sensing unit 301 through the data input/output unit 34 to the sensing unit-based speed detection unit 35 .

In addition, the control unit 31 outputs an image input from the camera 302 through the data input/output unit 34 to the image-based speed detection unit 36 .

In addition, the control unit 31 uses the communication interface unit 33 so that the detection signal received from the detection means 301 through the data input/output unit 34 and the image received from the camera 302 are transmitted to the central control server 6. Control.

In addition, the controller 31 stores the sensing distance Ds calculated by the controller of the auxiliary sensor 303 in the memory 32 .

In addition, the memory 32 stores location information of the camera 302 and location information of the sensing area A of the camera 302 .

At this time, the detection area (A) is location information of the area being photographed by the camera 302 .

In addition, the memory 32 temporarily stores the detection signal detected by the sensing means 301 and the image captured by the camera 302 .

In addition, a preset image analysis algorithm and a license plate extraction algorithm are stored in the memory 32 .

At this time, the image analysis algorithm means an algorithm for detecting a vehicle as an object from an input image, and the license plate extraction algorithm means an algorithm for extracting a license plate image from a detected vehicle object.

In addition, the memory 32 stores the number of horizontal pixels of the image (Pa), which is the number of pixels in the horizontal direction among pixels constituting a frame of an image captured by the camera 302 .

Also, the sensing distance Ds, which is the distance from the camera 302 to the sensing point P, is stored in the memory 32.

Also, the threshold speed value is stored in the memory 32 .

At this time, the threshold speed value is the image-based speed among the traveling speed detected from the sensing means-based speed detection unit 35 (hereinafter referred to as 'first vehicle speed C1') and the traveling speed detected from the image-based speed detection unit 36. The maximum of the traveling speed detected by the detecting means-based speed detecting unit 35 that can determine that the traveling speed detected by the detecting unit 36 (hereinafter referred to as 'second vehicle speed C2') should be determined as the final speed value. means value.

The communication interface unit 33 transmits and receives data with the central control server 6 .

The data input/output unit 34 inputs/outputs data with the sensing means 301, the camera 302, and the auxiliary sensor 303.

FIG. 8 is a block diagram of a sensing means-based speed detection unit of FIG. 7 .

As shown in FIG. 8, the sensing unit-based speed detection unit 35 includes a signal analysis module 351, a vehicle detection module 352, a tracking module 353, a speed calculation module 354, and a vehicle information generation module ( 355).

When the signal analysis module 351 receives the detection signal transmitted from the detection means 301 through the data input/output unit 34, it analyzes the input detection signal and transmits the analysis data of the detection signal to the vehicle detection module 352. print out

The vehicle detection module 352 detects a vehicle object by utilizing the analysis data of the detection signal received from the signal analysis module 351 .

The tracking module 353 tracks the trajectory of the vehicle object detected by the vehicle detection module 352 .

The speed calculation module 354 utilizes the trajectory information detected by the tracking module 353 to calculate the first vehicle speed C1, which is the speed of the vehicle.

At this time, since the technology and method for calculating the object speed using the detection signal are well-known technologies, a detailed description thereof will be omitted.

The vehicle information generation module 355 utilizes and processes the trajectory information detected by the tracking module 353 to generate vehicle information including the location and speed of a vehicle object.

At this time, the vehicle information generated by the vehicle information generating module 355 is output to the speed determining unit 37 .

Since the sensing means-based speed detection unit 35 configured as described above is configured to generate vehicle information by receiving a detection signal from the sensing means 301 using radar or lidar, the accuracy of the vehicle position and speed is the speed through the image. It is not only more accurate than the detection method, but also has the advantage of being able to detect a wide place such as an intersection or a position placed in a blind spot of the image of the camera 302, such as a shoulder, because the detectable range is wide.

However, since the detection means 301 has a disadvantage in that classification accuracy for objects is low, vehicle information of some vehicles cannot be acquired when vehicles are driving at low speeds and are driving adjacent to each other.

In order to prevent this problem from occurring, the present invention provides a speed determination unit 37 when the first vehicle speed C1, which is the speed of the vehicle detected by the sensing unit-based speed detection unit 35, is less than or equal to a preset threshold speed value. Determines that it is difficult to obtain vehicle information by the sensing unit-based speed detection unit 35, and determines the second vehicle speed C2, which is the speed of the vehicle detected by the image-based speed detection unit 36, as the final speed value.

The image-based speed detection unit 36 extracts the license plate images from the frames of the image acquired by the camera 302, and then utilizes the change ratio Z, which is the size change ratio of the license plate image, to the second vehicle speed C2 ) is calculated.

A detailed description of the image-based speed detection unit 36 will be described in detail with reference to FIGS. 10 to 15 to be described later.

FIG. 9 is a block diagram of a speed determination unit of FIG. 7 .

As shown in FIG. 9 , the speed determination unit 37 includes a speed value extraction module 371 , a comparison module 372 , a determination module 373 , and a vehicle information correction module 374 .

The speed value extraction module 371 extracts the first and second vehicle speeds C1 and C2 detected from the sensing unit-based speed detection unit 35 and the image-based speed detection unit 36 from the memory 32 .

The comparison module 372 compares the first vehicle speed C1 with a threshold speed value previously stored in the memory 32 .

The determination module 373 1) determines the first vehicle speed C1 as the final speed value when the first vehicle speed C1 is greater than or equal to the threshold speed value, and 2) determines that the first vehicle speed C1 is the threshold speed value If the value is less than the first vehicle speed C1, it is determined that the reliability of the first vehicle speed C1 is low, and the second vehicle speed C2 is determined as the final speed value.

When the second vehicle speed C2 is determined as the final speed value by the determination module 373, the vehicle information correcting module 374 converts the vehicle information generated from the vehicle information generating module 355 into the second vehicle speed C2. corrected with

At this time, the vehicle information corrected by the vehicle information correction module 374 is finally stored in the memory 32 and output to the violating vehicle enforcement unit 38 and the central control server 6 at the same time.

The speed determining unit 37 configured as described above determines the first vehicle speed C1, which is the vehicle speed measured by the sensing means 301, as the final speed value in normal times, but the vehicles are traveling at a low speed and the sensing means 301 When it is determined that it is difficult to accurately measure the speed of vehicles due to difficulty in classifying vehicles by ), the second vehicle speed C2, which is the measured speed based on the image of the camera 302, is determined as the final speed value of the vehicle. Highly reliable vehicle information can be obtained regardless of speed.

The violating vehicle enforcement unit 38 utilizes the vehicle information determined by the speed determination unit 37 and analyzes the video taken by the camera 302 to detect the violating vehicle, and the violation information about the detected violating vehicle. is output to the central control server (6).

At this time, the violation information includes the vehicle location, speed, and details of the violation.

For example, the violating vehicle enforcement unit 38 utilizes the detection information of the detection unit 301 and the image information of the camera 302 to detect a shoulder stop, speeding, lane change, illegal U-turn, etc. through By extracting the license plate image of the violating vehicle from the image of the camera 302 and outputting the extracted license plate image and corresponding vehicle information to the central control server 6, it is possible to crack down on vehicles violating traffic laws.

FIG. 10 is a block diagram of the image-based speed detection unit of FIG. 7 .

As shown in FIG. 10 , the image-based speed detector 36 includes a frame extractor 361, a frame analyzer 362, and a traveling speed calculator 363.

When receiving detection information from the auxiliary sensor 303, the frame extraction unit 361 is executed under the control of the control unit 31 and receives an image obtained by taking a picture of the camera 302.

In addition, the frame extraction unit 361 is a frame of the first time point t1, which is the time point at which sensing information is input, that is, the time point at which the vehicle enters the sensing point P by the auxiliary sensor 303, from the input image. Frame F1 is extracted.

In addition, the frame extraction unit 361 includes a timer (not shown), and when the second time point t2, which is the time point at which the preset time T1 has elapsed based on the first time point t1, is obtained from the input image. The second frame F2, which is the frame of the second time point t2, is extracted.

11 is a block diagram of the frame analysis unit of FIG. 10, FIG. 12 is an example of a first frame, FIG. 13 is an example of a second frame, and FIG. 14 is an example for explaining a process of calculating vehicle speed. It is also

As shown in FIG. 11, the frame analysis unit 362 includes an object detection module 3621, a license plate image extraction module 3622, a pixel number calculation module 3623, a variable ratio calculation module 3624, and a mileage calculation module 3625.

When the object detection module 3621 receives the first frame F1 from the frame extractor 361, the detection point P is obtained from the input first frame F1 by utilizing the image analysis algorithm stored in the memory 32. Detects a vehicle object image, which is an entry vehicle that has entered.

Also, when receiving the second frame F2 from the frame extractor 361, the object detection module 3621 detects a vehicle object image from the input second frame F2 by using an image analysis algorithm.

The license plate image extraction module 3622 analyzes the vehicle object image of the first frame F1 detected by the object detection module 3621 using the license plate extraction algorithm stored in the memory 32 to obtain the license plate image from the vehicle object. The first license plate image B1 is extracted.

In addition, the license plate image extraction module 3622 analyzes the vehicle object image of the second frame F2 detected by the object detection module 3621 using the license plate extraction algorithm to extract the second license plate image B2.

The pixel count calculation module 3623 calculates the first number of horizontal pixels Pb1, which is the number of pixels in the horizontal direction of the first license plate image B1 extracted from the license plate image extraction module 3622, and the horizontal number of the second license plate image B2. The second number of horizontal pixels Pb2, which is the number of directional pixels, is calculated.

The pixel count calculation module 3623 is configured to calculate the mileage of the vehicle using only horizontal pixels among horizontal pixels and vertical pixels constituting the pixels of the frame, so that the calculation process is simpler than when using all the pixels. It has the advantage of improving the processing speed.

The variable ratio calculation module 3624 utilizes the first number of horizontal pixels (Pb1) of the first license plate image (B1) calculated by the pixel number calculation module 3623 and the number of horizontal pixels (Pa) of the preset image to determine the first number of horizontal pixels (Pa). The license plate ratio (Z1) is calculated.

At this time, the first license plate ratio (Z1) is a value obtained by dividing the number of horizontal pixels (Pa) of the image by the number of first horizontal pixels (Pb1) (Z1 = Pb1 / Pa), and the first license plate image within the first frame (F1) (B1) means the ratio occupied in the horizontal direction.

In addition, the variable ratio calculation module 3624 utilizes the second number of horizontal pixels (Pb2) of the second license plate image (B2) calculated by the number of pixels calculation module (3623) and the number of horizontal pixels (Pa) of the preset image to determine the second number of horizontal pixels (Pa). Calculate license plate ratio (Z2).

At this time, the second license plate ratio (Z2) is a value obtained by dividing the number of horizontal pixels (Pa) of the image by the number of second horizontal pixels (Pb2) (Z2 = Pb2 / Pa), and the second license plate image within the second frame (F2) (B2) means the ratio occupied in the horizontal direction.

In addition, when the first and second license plate ratios (Z1) and (Z2) are calculated, the variable ratio calculation module 3624 divides the second license plate ratio (Z2) by the first license plate ratio (Z1) to divide the second license plate image (B2). ) calculates the variation ratio (Z), which is a ratio that is changed from the first license plate image (B1).

At this time, the variation ratio (Z) is a value (Z = Z2 / Z1) divided by the second license plate ratio (Z2) by the first license plate ratio (Z1), and the size of the license plate photographed in the second frame (F2) is the first frame It means the ratio changed from the size of the license plate photographed in (F1).

The following Equation 1 is a calculation formula for calculating the variation ratio.

Here, Z is the variation ratio, Z2 is the second license plate ratio, Z1 is the first license plate ratio, Pb1 is the first number of horizontal pixels, and Pb2 is the number of second horizontal pixels.

That is, the variation ratio calculation module 3624 converts the second number of horizontal pixels (Pb2) of the license plate photographed in the second frame (F2) to the first horizontal pixel of the license plate photographed in the first frame (F1) through Equation 1. By dividing by the number (Pb1), it is possible to quickly and simply calculate the variation ratio (Z) of the license plate.

In addition, the variable ratio Z calculated by the variable ratio calculation module 3624 is used to calculate the mileage Dx of the vehicle.

The mileage calculation module 3625 calculates the mileage Dx of the vehicle by substituting the variation ratio Z calculated using Equation 1 into Equation 5.

At this time, the traveling distance Dx is calculated by substituting the sensing distance Ds, which is the distance from the camera 302 to the sensing point P, and the variation rate Z in Equation 5 described later, and from the sensing point P It means the distance that the vehicle has moved during the set time (T1).

15 is an exemplary diagram for explaining a process of calculating Equation 2.

As shown in FIG. 15 , the camera 302 is installed facing the road to photograph a vehicle traveling on the road, and captures an image at an angle equal to a preset field of view (FOV).

At this time, the angle of view (F) means an angle at which the camera 302 can capture an image when viewed from above.

As the camera 302 moves away from the camera, the picture width Pw captured in the image increases.

Also, for convenience of calculation, as shown in FIG. 15 , the angle of view F is 2θ.

The picture width Pw means an actual width photographed at a distance d away from the camera 302, and increases in proportion to the distance d.

At this time, the canvas width Pw is calculated using Equation 2 below.

At this time, Pw is the field width, θ is the value obtained by dividing the angle of view by half (θ=F/2), and d is the distance away from the camera.

When the width of the first frame F1 is the first screen width Pw1 and the width of the second frame F2 is the second screen Pw, the first screen width Pw1 is 2*tan(θ)* Ds, and the second picture width Pw2 is 2*tan(θ)*(Ds-Dx).

Also, in the image of the camera 302, since the actual width Pw is reduced to correspond to the number of horizontal pixels Pa of the image, the rate at which objects photographed in the image are reduced increases as the distance d increases. .

That is, in the image captured by the camera 302, as the distance d increases, a wider image is captured in the image, but the object is displayed smaller in the image.

In addition, the number of pixels Pb in the horizontal direction of the license plate inside the image captured by the camera 302 is calculated using Equation 3 below.

At this time, Pb means the number of pixels in the horizontal direction of the license plate, Pa means the number of pixels in the horizontal direction of the image, H means the actual horizontal length of the license plate, and Pw means the width of the screen.

If the first screen width Pw is substituted into Equation 3, the first number of horizontal pixels (Pb1 = Pa*H/Pw1) can be calculated, and if the second screen width Pw is substituted, the second number of horizontal pixels ( Pb2=Pa*H/Pw2) can be calculated.

In addition, when the first number of horizontal pixels (Pb1 = Pa * H / Pw1) and the second number of horizontal pixels (Pb2 = Pa * H / Pw2) calculated by Equation 3 are substituted into Equation 1, the variation ratio (Z) this is calculated

The following Equation 4 is a calculation formula for calculating the variation ratio by substituting the first number of horizontal pixels (Pb1) and the second number of horizontal pixels (Pb2) calculated by Equation 3 into Equation 1, and Equation 5 is This is a formula for calculating the mileage by transforming Equation 4.

At this time, θ is a value obtained by dividing the angle of view in half (θ=F/2), Ds is the sensing distance, which is the distance from the camera 302 to the sensing point P, and Dx is the traveling distance of the vehicle.

That is, the mileage calculation module 3625 substitutes the sensing distance Ds previously stored in the memory 32 and the variation ratio Z calculated by Equation 1 into Equation 5 to calculate the mileage Dx of the vehicle. yield

The frame analyzer 362 configured as described above extracts the first license plate image B1 in the first frame F1 extracted by the frame extractor 361 and simultaneously displays the width of the first license plate image B1. The first number of horizontal pixels Pb1, which is the number of directional pixels, is calculated.

In addition, the frame analyzer 362 extracts the second license plate image B2 in the second frame F2 extracted by the frame extractor 361 and simultaneously extracts the number of pixels in the horizontal direction of the second license plate image B2. Calculate the second number of horizontal pixels Pb2.

At this time, the frame analysis unit 362 utilizes the number of horizontal pixels (Pa) of the image stored in the memory 32 to calculate the ratio occupied by the license plate images (B1) and (B2) in each of the frames (F1) and (F2). License plate ratios Z1 and Z2, which are ratios occupied by license plates in the first frame F1 and the second frame F2, are calculated.

In addition, the frame analyzer 362 calculates the variation ratio (Z) by utilizing the license plate ratios (Z1) and (Z2), and calculates the mileage (Dx) of the vehicle by calculating the calculated variation ratio (Z).

Returning to FIG. 10 and looking at the travel speed calculator 363, the travel speed calculator 363 divides the travel distance Dx calculated by the frame analyzer 362 by a preset set time period T1 to determine the vehicle's Calculate the travel speed (v).

The video-based speed detection unit 36 configured as described above extracts the license plate image of the entering vehicle from the first frame (F1) and the second frame (F2), respectively, and utilizes the change rate (Z), which is the size change rate of the license plate image, The installation environment (position and angle, etc. ) is configured so that it can be used immediately without conducting actual measurement whenever it is changed, so that the time required for installation is reduced, and even if the installation angle of the camera 302 is changed due to long-term use, the process of calculating the driving speed of the vehicle errors can be prevented from occurring.

16 is a block diagram of a second controller that is a second embodiment of the controller of FIG. 7 , and FIG. 17 is a block diagram of a second video-based speed detector that is a second embodiment of the video-based speed detector of FIG. 16 .

As shown in FIG. 16, the second controller 40 includes a control unit 41, a second memory 42, a communication interface unit 43, a data input/output unit 44, and a sensing unit-based speed detection unit 45. , a second image-based speed detection unit 46, a speed determination unit 47, and a violation vehicle enforcement unit 48.

At this time, the control unit 41, the communication interface unit 43, the data input/output unit 44, the sensing means-based speed detection unit 45, the speed determination unit 47, and the violation vehicle control unit 48 are the first embodiment of the control unit ( 31), the communication interface unit 33, the data input/output unit 34, the sensing means-based speed detection unit 35, the speed determination unit 37, and the violating vehicle control unit 38 perform the same operation. should be omitted.

In the second memory 42, set times T2, T3, ..., Tn are additionally stored.

As shown in FIG. 17 , the second image-based speed detector 46 includes a second frame extractor 461, a second frame analyzer 462, and a second travel speed calculator 463.

When receiving detection information from the auxiliary sensor 303, the second frame extraction unit 461 is executed under the control of the control unit 41 and receives an image acquired by the camera 302.

In addition, the second frame extraction unit 461 is a frame of the first time point t1, which is the time point at which sensing information is input, that is, the time point at which the vehicle enters the sensing point P by the auxiliary sensor 303, from the input image. The first frame F1 is extracted.

In addition, the second frame extraction unit 461 includes a timer, and the second, third, ... time points at which preset times T1, T2, ..., Tn have elapsed based on the first time point t1. At the nth time points (t2, t3, ..., tn+1), the second, third, ... The frames F2, F3, ..., Fn+1 of the nth time points t2, t3, ..., tn+1 are extracted.

Unlike the frame extractor 361 of FIG. 10, the second frame extractor 461 operates to extract a plurality of frames F2, F3, ..., Fn+1.

The second frame analyzer 462 performs the same operation as the frame analyzer 362 of FIG. 10 , but unlike the frame extractor 361 of FIG. 10 , a plurality of frames from the second frame extractor 461 ( Since F2, F3, ..., Fn+1) are input, a plurality of travel distances (Dx1, Dx2, ..., Dxn) are calculated.

At this time, since the process of calculating the mileage is the same as that of the frame analyzer 362 of FIG. 10 described above, a detailed description thereof will be omitted.

The second travel speed calculator 463 divides the travel distances (Dx1, Dx2, ..., Dxn) calculated from the second frame analyzer 462 into corresponding set times (T1, T2, ..., Tn) for the vehicle The traveling speeds (v1, v2, ..., vn) of are calculated.

In addition, the second travel speed calculator 463 calculates the average travel speed va, which is an average value of the calculated travel speeds v1, v2, ..., vn.

The second controller 40 configured as described above uses the license plate images captured in the first frame F1 and the second frame F2 in the controller 30 of FIG. 7 according to the first embodiment to determine the driving speed v. Unlike calculation, license plate images are extracted from a plurality of frames (F1, F2, F3, ..., Fn + 1) to calculate driving speeds (v1, v2, ..., vn), and calculated driving speeds (v1, Since the average traveling speed va is calculated from v2, ..., vn, some of the frames F2, F3, ..., Fn+1 extracted from the second frame extraction unit 461 are selected from the camera 302. ) Minimize the occurrence of errors in the vehicle's traveling speed by calculating the average traveling speed (va) of the traveling speeds (v1, v2, ..., vn) calculated from a plurality of frames even if the image is blurry due to vibration or shock You can do it.

18 is a block diagram of a third frame analyzer that is a third embodiment of the frame analyzer of FIG. 11, FIG. 19 is a block diagram of a second travel distance calculation module of FIG. 18, and FIG. 20 is a travel by the third frame analyzer. It is an exemplary view of a license plate to explain the process of calculating the distance.

In general, an image has a characteristic of being large when photographed from the front and small when photographed from the rear, even though the object is of the same size due to perspective.

At this time, the controller 30 of FIGS. 3 to 15 described above is configured to calculate the mileage under the assumption that the vehicle moves in a straight line along the lane (straight line movement), but in actual roads, inclination movements such as changing lanes of the vehicle are infrequent. In the case of an oblique movement, the right side and the left side of the license plate image have different sizes due to perspective, and the number of pixels in the horizontal direction of the license plate image is reduced.

For example, when it is assumed that the vehicle tilts between the first time point t1 and the second time point t2 due to a lane change or the like, the left and right sides of the license plate image at the second time point t2 are different from each other. size, and the second number of horizontal pixels (Pb2) of the license plate image at the second time point (t2) is taken in a size smaller than the number of second horizontal pixels when the vehicle moves in a straight line.

That is, when the variation ratio (Z) is calculated by using the second number of horizontal pixels (Pb2) reduced by perspective, the variation ratio (Z) is reduced compared to when the vehicle is traveling in a straight line, which is different from the actual driving distance of the vehicle. A problem arises in that an error occurs in the calculation of the travel speed v due to the calculation of the travel distance.

The third frame analysis unit 492 is a first license plate image B1 extracted from the first frame F1 and a second license plate image extracted from the second frame F2 in order to prevent these problems from occurring ( The lengths of the left and right sides of B2) are extracted, respectively, and the distance between the left and right sides of the license plate image is calculated using the ratio of change of the left side and the right side of the extracted license plate images, respectively, and the calculated distance between the left and right sides of the license plate image is calculated. The driving distance (Dx) of the vehicle is calculated using the distance between the edges.

The third frame analysis unit 492 is a third embodiment of the frame analysis unit 362 of FIG. 11, and as shown in FIG. 18, the object detection module 4921, the license plate image extraction module 4922, the second It consists of a pixel count calculation module 4923, a first determination module 4924, a separation distance calculation module 4925, and a second travel distance calculation module 4926.

At this time, the object detection module 4921 detects the entering vehicle entering the detection point P from the first frame F1, and detects the same vehicle as the entering vehicle detected from the first frame F1 in the second frame F2. detect from

The license plate image extraction module 4922 is executed when an entering vehicle is detected from the first frame F1 and the second frame F2 by the object detection module 4921, and the first license plate image in the first frame F1. (B1) is extracted, and the second license plate image (B2) is extracted from the second frame (F2).

The second pixel count calculation module 4923 calculates the number of pixels from the first license plate image B1 and the second license plate image B2.

At this time, the second pixel count calculation module 4923 1) analyzes the first license plate image B1, and determines the first number of pixels L1 on the left side of the first license plate image B1, which is the number of pixels on the left side, and the first number plate The first number of pixels on the right side R1, which is the number of pixels on the right side of the image B1, is calculated, and 2) the number of pixels on the left side of the second license plate image B2 is analyzed by analyzing the second license plate image B2. 2 The number of pixels on the left side (L2) and the number of pixels on the right side of the second license plate image (B2), that is, the second number of pixels on the right side (R2) are calculated.

The first determination module 4924 compares the first left side pixel number L1 and the first right side pixel number R1 calculated by the second pixel number calculation module 4923, and selects the side with the larger number of pixels. It is determined that the side disposed in the front side (hereinafter referred to as 'front side').

At this time, the front side is a side disposed in front of the rear side (hereinafter referred to as 'rear side') among the left and right sides of the license plate image, and is closer to the camera 302 than the rear side.

In addition, for convenience of explanation, the left side of the license plate image is described as an example as the front side.

The separation distance calculation module 4925 calculates the separation distance D1, which is the distance at which the first number of pixels on the left side L1 and the number of pixels on the right side R1 are separated in the front-back direction.

At this time, the separation distance calculation module 4925 substitutes the first number of pixels on the left side (L1) and the number of pixels on the right side (R1) into Equation 1, so that the first change rate (Y1), which is the change rate of the front side, ) is calculated.

Also, the separation distance calculation module 4925 calculates the separation distance D1 by substituting the first variable ratio Y1 into Equation 5.

At this time, the method of calculating the first variable ratio Y1 and the method of calculating the separation distance D1 are the same as the method of calculating the aforementioned variable ratio Z and the method of calculating the mileage Dx of the vehicle. A detailed description is omitted.

As shown in FIG. 19 , the second travel distance calculation module 4926 includes a left edge distance calculation module 49261, a right edge distance calculation module 49262, and an average distance calculation module 49263.

The left side-to-side distance calculation module 49261 calculates the left side change rate by substituting the first left side pixel number L1 and the second left side pixel number L2 into Equation 1, and calculates the left side change rate. Substitute into Equation 5 to calculate the left edge distance value (D2).

At this time, since the method of calculating the distance value D2 between the left edges is the same as the method of calculating the distance Dx previously described, detailed descriptions thereof will be omitted.

The right-side distance calculation module 49262 calculates the right-side variation ratio by substituting the first right-side pixel number R1 and the second right-side pixel number R2 into Equation 1, and the calculated right-side variation ratio is substituted into Equation 5 to calculate the distance value D3 between the right edges.

At this time, in the case of the right side of the first license plate image B1, the left side, which is the front side, is disposed rearward by the distance D1 from the sensing distance Ds where the left side is disposed, and the sensing distance Ds and the distance D1 from the camera 302 ), it is preferable to substitute Ds' for Ds in Equation 5 because they are spaced apart by the sum of (Ds', Ds' = Ds + D1).

The average distance calculation module 49263 calculates an average distance value D4 that is an average value of the left edge distance value D2 and the right edge distance value D3.

The average distance calculation module 49263 calculates the distance between the left sides even when the first vehicle capturing angle θ1 and the second vehicle capturing angle θ2, which is the vehicle capturing angle in the second frame F2, are different from each other due to an inclined movement of the vehicle. The distance traveled by the vehicle during the set time period T1 may be calculated by calculating the average distance value D4, which is an average value of the value D2 and the distance value D3 between the right edges.

The third frame analysis unit 492 configured as described above calculates the average distance value D4, which is the average value of the distance value D2 between the left edges and the distance value D3 between the right edges of the license plate even when the vehicle moves in an inclined direction, so that the vehicle It is configured to calculate the distance traveled during the set time period (T1).

21 is a block diagram of a third video-based speed detector, which is a third embodiment of the image-based speed detector of FIG. 10, FIG. 22 is a block diagram of a set time optimization unit of FIG. 21, and FIG. 23 is an optimal distance calculation module of FIG. , FIG. 24 is an exemplary diagram for explaining an optimal distance, and FIG. 25 is an exemplary diagram of a vehicle photographed from the side.

The third image-based speed detector 56 is a third embodiment of the image-based speed detector 36 of FIG. 10, and as shown in FIG. 21, a frame extractor 561, a frame analyzer 562, It consists of a travel speed calculation unit 563 and a set time optimization unit 564.

At this time, the frame extractor 561, the frame analyzer 562, and the travel speed calculator 563 include the frame extractor 361, the frame analyzer 362, and the travel speed calculator 363 of FIG. 10 described above. ), so detailed descriptions will be omitted.

At this time, the memory 32 of the controller 30 additionally stores the period T' and location information of the optimal area.

At this time, the period T' means a period in which the set time optimization unit 564 is executed.

In addition, the location information of the optimal area means location information on the frame of the second license plate image (B2) that can increase the accuracy of image analysis, and means a location adjacent to the center of the frame.

Also, the average distance calculated by the average distance calculation module 5644 and the threshold distance value are stored in the memory 32 .

At this time, the threshold distance value means the minimum value of the difference value calculated by the difference value calculation module 5645 that can determine that the set time T1 needs to be reset.

In general, the traffic volume on a road is changed by various variables such as location, time of day, and weather, and the average traveling speed of vehicles is changed by the traffic volume on the changed road.

In particular, the present invention is the driving speed ( Since v) is calculated, 1) when driving at low speed due to increased traffic volume, the distance between the license plate images captured in the first frame (F1) and the second frame (F2) is reduced, and 2) when driving at high speed due to reduced traffic volume In the case of, the distance between the license plate images photographed in the first frame (F1) and the second frame (F2) is increased to calculate the driving speed by comparing the first license plate image (B1) and the second license plate image (B2) Errors may occur in the process.

Hereinafter, a problem in which an error occurs in the calculated driving speed when the driving speed of the vehicle is low or high will be described with reference to FIGS. 24 and 25 .

As shown in FIG. 24, when the vehicle passes the detection point P, the license plate image B1 of the first frame F1 is disposed at the pixel position of the detection point P, and the preset period T1 ) elapses, the license plate image moves forward from the detection point P because the vehicle moves toward the camera.

At this time, the pixel position of the license plate image in the second frame F2 moves forward as the driving speed v increases.

That is, as shown in FIG. 24, 1) when the traffic volume is reduced and the traveling speed (v) of the vehicles is increased, the second license plate image (B4) at the time when the set time (T1) has elapsed is forward within the frame. (P4), and 2) when the traveling speed (v) of the vehicles is reduced due to increased traffic volume, the second license plate image (B2) at the time when the set time (T1) has elapsed is the detection point (P) within the frame. ) and is disposed at a position adjacent to (P3).

At this time, 1) When the second license plate image B2 is excessively adjacent to the first license plate image B1, the fluctuation rate of the license plate image is minute, resulting in a high error rate in the process of calculating the mileage Dx. 2) When the second license plate image (B2) is excessively spaced apart from the first license plate image (B1) and is located in front of the frame, as shown in FIG. will have

That is, by optimizing the setting time T1 according to the driving speed of the vehicle, it is necessary to arrange the second license plate image B2 in the optimum region, which is the midpoint of the frame.

As shown in FIG. 22, the set time optimization unit 564 includes an optimal distance calculation module 5641, an average traveling speed calculation module 5642, an average travel distance calculation module 5643, and an average distance calculation module 5644. , a difference value calculation module 5645, a second decision module 5646, and a set time resetting module 5647.

As shown in FIG. 23, the optimum distance calculation module 5641 includes a selection frame extraction module 56411, a second detection distance extraction module 56412, and an optimum distance determination module 56413.

After the selection frame extraction module 56411 collects the second frames F2 collected in the memory 32 during the period T', the second license plate image B2 among the collected second frames F2 is Frames placed in the optimal region are extracted. At this time, the extracted frame will be referred to as a selection frame.

The second detection distance extraction module 56412 extracts the analysis data by the selection frames extracted by the selection frame extraction module 56411 from the memory 32, and from the extracted analysis data of each selection frame, the camera 302 ) to extract the second sensing distance information, which is a linear distance on a plane from the second license plate image B2.

At this time, the second sensing distance means a straight line distance on a plane between the sensing vehicle and the camera 302 at the second time point t2 from the camera 302, a value obtained by subtracting the sensing distance Ds and the traveling distance Dx ( Ds-Dx).

The optimum distance determination module 56413 calculates an average value of the second sensing distances extracted from the second sensing distance extraction module 56412, and determines the average value of the calculated second sensing distances as the optimum distance.

In this case, the optimal distance means a straight line distance on a plane from the camera 302 at a position corresponding to the location information of the optimal area set on the frame.

In general, in order to match the location on the frame with the location of the actual road, a separate preliminary work such as setting a reference position for matching the image and the actual road is required, but the present invention can be applied without such preliminary work. , Since it is configured to detect the optimal distance matching the optimal area (on the frame) through the calculation of the average value of the previous selected frames for each predetermined period (T '), even if installed at various heights and shooting angles, the optimum matching the optimal area distance can be calculated.

In addition, the optimum distance calculated by the optimum distance calculation module 5641 is input to a difference value calculation module 5645 to be described later.

The average traveling speed calculation module 5642 is executed every period T', and extracts the traveling speeds v of vehicles calculated by the traveling speed calculating unit 363 during the period T' from the memory 32 and , Calculate the average traveling speed (V), which is the average value of the extracted traveling speeds (v).

The average travel distance calculation module 5643 calculates the average travel distance Da, which is a value obtained by multiplying the average travel speed V calculated by the average travel speed calculation module 5642 by the set time T1 stored in the memory 32. do.

The average distance calculation module 5644 calculates an average distance obtained by subtracting the average travel distance Da calculated by the average travel distance calculation module 5643 from the detected distance Ds.

The difference calculation module 5645 calculates a difference value obtained by subtracting the optimum distance calculated by the optimum distance calculation module 5641 from the average distance calculated by the average distance calculation module 5644.

At this time, the difference value means a distance value at which the vehicles photographed by the camera 302 are spaced apart on average from the optimal distance, and the larger the difference value, the more the vehicles are separated from the optimal distance.

In addition, if the difference value is 1) positive, it means that the vehicles are photographed in a state where they do not reach the optimal distance and the second license plate image (B2) is photographed adjacent to the first license plate image (B1), and 2) if it is a negative number , This means that the second license plate image B2 is captured in a state excessively spaced apart from the first license plate image B1.

The second determination module 5646 compares the absolute value of the difference value calculated by the difference value calculation module 5645 with the threshold distance value stored in the memory 32, and 1) by the difference value calculation module 5645 If the absolute value of the calculated difference value is less than the threshold distance value, it is determined that the photograph was taken normally. 2) If the absolute value of the difference value calculated by the difference value calculation module 5645 is greater than or equal to the threshold distance value, the set time (T1) It is determined that resetting is necessary and the set time resetting module 5647 is executed.

The set time reset module 5647 is executed when the absolute value of the difference value is greater than or equal to the threshold distance value. 1) If the difference value is a positive number, the set time T1 is increased by a preset constant 'α', and 2) the difference value If this is a negative number, the set time (T1) is reduced by a predetermined constant 'α' so that the second license plate image (B2) is taken at a position adjacent to the optimal distance.

Through this, the set time reset module 5647 induces the second license plate image B2 to be taken at the optimum distance through resetting the set time T1, thereby preventing an error from occurring in measuring the driving speed v of the vehicle. .

The third image-based speed detection unit 56 configured as described above is to be photographed adjacent to or spaced apart from the first license plate image B1 in the position of the second license plate image B2 photographed in the second frame F2. In this case, the second license plate image (B2) is taken too close to the first license plate image (B1) by resetting the set time (T1) so that the second license plate image (B2) is taken at a position adjacent to the optimal distance, or too close to the first license plate image (B1). It is photographed at a distance to prevent an error from occurring in the process of calculating the mileage Dx of the vehicle.

1: Traffic information management system 3: Filming department
6: central control server 7: communication network
30: controller 31: control unit
32: memory 33: communication interface unit
34: data input/output unit 35: detection means-based speed detection unit
36: image-based speed detection unit 37: speed determination unit
38: violation vehicle control unit 301: detection means
302: camera 303: auxiliary sensor

Claims (11)

In a traffic information management system including at least one photographing unit that detects a vehicle in a preset area and then calculates the speed of the detected vehicle:
The filming department
Detecting means for detecting a preset area;
a camera that captures a preset area;
The vehicle is detected by analyzing the data input from the sensing means and the camera, and then the speed of the sensing vehicle is detected, and the speed of the sensing vehicle is detected by analyzing the sensing data received from the sensing means. a controller for calculating the speed of the vehicle by analyzing the image information obtained from the camera when is less than a preset value;
When detecting a vehicle, the detection signal is analyzed to calculate the detection distance (Ds), which is a straight-line distance on a plane from the camera to the detected vehicle, and the detection information including the calculated detection distance (Ds) information is generated and transferred to the controller. Including an auxiliary sensor that outputs,
The controller
a memory in which a license plate extraction algorithm, which is an algorithm for extracting a vehicle license plate from the detected vehicle object after detecting a vehicle object from an input image, is stored;
a sensing means-based speed detection unit for detecting a speed of the vehicle by analyzing the sensing information obtained from the sensing means;
an image-based speed detector for detecting the speed of the vehicle by analyzing the image information obtained from the camera;
The first vehicle speed C1, which is the speed of the vehicle detected by the sensing unit-based speed detection unit, is determined as a final speed value, and when the first vehicle speed C1 is equal to or less than a preset threshold speed value, the image-based speed detection unit determines And a speed determination unit for determining the second vehicle speed C2, which is the detected vehicle speed, as a final speed value,
The video-based speed detection unit
A first frame (F1), which is a frame at the time point (t1) when the vehicle is detected from the image acquired by the camera, and a second frame (F1), which is a frame at the time point (t2) when the preset time (T1) has elapsed ( F2) a frame extraction unit for extracting;
License plate images (B1) and (B2) are extracted from the first and second frames (F1) and (F2) extracted by the frame extraction unit, respectively, and the extracted first and second license plate images (B1) and (B2) are extracted. After calculating the variation ratio (Z) using ), considering the characteristic that the mileage (Dx) of the detected vehicle increases in proportion to the variation ratio (Z), from the calculated variation ratio (Z) and the auxiliary sensor a frame analyzer that calculates a travel distance Dx of the detected vehicle by utilizing the input detected distance Ds;
and a driving speed calculation unit for dividing the traveling distance (Dx) of the detected vehicle calculated by the frame analyzing unit by a set time (T1) to calculate the traveling speed (v) of the vehicle. delete The method of claim 1, wherein the speed determining unit
a speed value extraction module for extracting a first vehicle speed (C1) and a second vehicle speed (C2) from the sensing unit-based speed detection unit and the image-based speed detection unit;
a comparison module that compares the first vehicle speed C1 with a preset threshold speed value;
When the first vehicle speed C1 is greater than or equal to the preset threshold speed value, the first vehicle speed C1 is determined as the final speed value, but when the first vehicle speed C1 is less than the threshold speed value, the second vehicle speed ( A traffic information management system comprising a decision module for determining C2) as the final speed value. delete The method of claim 3, wherein the frame analyzer
After detecting a vehicle object from the first frame (F1) using the license plate extraction algorithm, and extracting a first license plate image (B1) from the detected vehicle object, and using the license plate extraction algorithm, the second frame ( F2) from the license plate image extraction module for extracting the second license plate image (B2);
After calculating the sizes of the first and second license plate images (B1) and (B2) extracted by the license plate image extraction module, the size of the first and second license plate images (B1) and (B2) calculated is used to change the size. a variable ratio calculation module that calculates a ratio (Z);
Considering the characteristic that the driving distance (Dx) of the detected vehicle increases in proportion to the variation ratio (Z), the variation ratio (Z) calculated by the variation ratio calculation module and the sensing distance (Ds) input from the auxiliary sensor Traffic information management system comprising a mileage calculation module for calculating the mileage (Dx) of the detected vehicle by using. The method of claim 5, wherein the frame analyzer
Further comprising a pixel count calculation module executed after the license plate image extraction module,
The pixel count calculation module
The first number of horizontal pixels (Pb1), which is the number of pixels in the horizontal direction of the first license plate image (B1) extracted by the license plate image extraction module, and the width of the second license plate image (B2) extracted by the license plate image extraction module Calculate the second horizontal pixel number (Pb2), which is the number of directional pixels;
The variable ratio calculation module
Characterized in that the variation ratio (Z) is calculated by dividing the second number of horizontal pixels (Pb2) calculated by the pixel number calculation module by the first number of horizontal pixels (Pb1) calculated by the pixel number calculation module. Traffic information management system. The method of claim 6, wherein the mileage calculation module
Transportation characterized in that the vehicle mileage (Dx) is calculated by substituting the variation ratio (Z) calculated by the variation ratio calculation module and the sensing distance (Ds) input from the auxiliary sensor into the following Equation 1 information management system.
[Equation 1]

At this time, 'Dx' is the travel distance, which is the distance the vehicle travels for the set time from the detection point, 'Ds' is the detection distance, which is the distance from the camera to the detection point, and 'Z' is the change rate. The method of claim 7, wherein the frame extraction unit
Frames (F3, F4, ...) of time points (t3, t4, ..., tn+1) at which predetermined set times (T2, T3, ..., Tn) have elapsed from the time point (t1) when the vehicle is detected by the auxiliary sensor. , tn) are further extracted,
The frame analyzer
License plate images are extracted from the frames (F3, F4, ..., tn) additionally extracted from the frame extractor, and the number of horizontal pixels of the extracted license plate images is calculated and substituted into Equation 1, thereby detecting vehicle mileage (Dx2, Dx3, ..., Dxn) are additionally calculated,
The driving speed calculator
The traveling speed of the vehicle (v1, v2, ..., vn) are calculated, and after calculating the average traveling speed (va), which is the average value of the calculated traveling speeds (v1, v2, ..., vn), the calculated average traveling speed (va) is calculated for the detected vehicle. A traffic information management system characterized in that it is determined by the driving speed (v). The method of claim 3, wherein the frame analyzer
A license plate image extraction module for extracting a first license plate image (B1) from the first frame (F1) by utilizing the license plate extraction algorithm and extracting a second license plate image (B2) from the second frame (F2);
From the first license plate image B1, a first left side pixel number L1, which is the number of pixels on the left side, and a first right side pixel number R1, which is the number of pixels on the right side, are extracted, and the second license plate image B2 a pixel number calculation module for calculating a second left side pixel number (L2), which is the number of pixels on the left side, and a second right side pixel number (R2), which is the number of pixels on the right side, from );
The variation rate of the left side is calculated by dividing the second left side pixel number L2 by the first left side pixel number L1, and the calculated left side variation rate and the sensing distance Ds input from the auxiliary sensor are calculated by the following arithmetic The distance value D2 between the left sides is calculated by substituting into Equation 2, and the variation rate of the right side is calculated by dividing the second right side pixel number R2 by the first right side pixel number R1, and the calculated right side variation rate and the sensing distance Ds input from the auxiliary sensor are substituted into the following Equation 2 to calculate the right edge distance value D3, and the calculated left edge distance value D2 and the right edge distance value D3 After calculating the average distance value (D4), which is the average value, and then calculating the calculated average distance value (D4), including a mileage calculation module for determining the calculated average distance value (D4) as the mileage (Dx) Traffic information management system characterized in that.
[Equation 2]

At this time, 'Dx' is the travel distance, which is the distance the vehicle travels for the set time from the detection point, 'Ds' is the detection distance, which is the distance from the camera to the detection point, and 'Z' is the change rate. The method of claim 3, wherein the memory
The positional information of the optimal region, which is the position on the frame of the second license plate image (B2), which can increase the accuracy of image analysis, is preset and stored,
The controller
Further comprising a set time optimization unit that is executed every preset period (T '),
The setting time optimization unit
an optimal distance calculation module that calculates an optimal distance, which is a linear distance on a plane with the camera on an actual road matched to the optimal area;
The driving speed (v) data of the vehicles calculated by the driving speed calculator during the period (T') is extracted from the memory, and the average driving speed (V), which is the average value of the extracted driving speeds (v), is calculated. driving speed calculation module;
an average traveling distance calculating module that calculates an average traveling distance (Da) obtained by multiplying the average traveling speed (V) calculated by the average traveling speed calculating module and a predetermined set time (T1);
an average distance calculation module that calculates an average distance by subtracting the average travel distance Da calculated by the average travel distance calculation module from the detection distance Ds;
a difference calculation module for calculating a difference value by subtracting the optimum distance calculated by the optimum distance calculation module from the average distance calculated by the average distance calculation module;
The absolute value of the difference value calculated by the difference value calculation module is compared with the threshold distance value, which is the minimum value of the difference value that can determine that the set time T1 needs to be reset, and 1) the absolute value of the difference value is critical If it is less than the distance value, it is determined that there is no need to reset the setting start (T1), 2) if the absolute value of the difference value is greater than or equal to the threshold distance value, a second determination module that determines that the setting time (T1) needs to be reset;
It is executed when it is determined that the resetting of the set time T1 is necessary by the second determination module. 1) If the difference value is a positive number, the set time T1 is increased by a predetermined constant 'α', and 2) the difference If the value is a negative number, the traffic information management system characterized in that it comprises a set time resetting module that reduces the set time (T1) by a constant 'α'. 11. The method of claim 10, wherein the optimal distance calculation module
After extracting the second frames (F2) collected during the period (T'), a selection frame for extracting selection frames, which are frames in which the second license plate image is disposed in a predetermined optimal area, among the extracted second frames (F2). extraction module;
After extracting the analysis data by the selection frames extracted by the selection frame extraction module, the second sensing distance information, which is a straight-line distance on a plane from the camera to the second license plate image, from the analysis data of each selection frame extracted. A second sensing distance extraction module for extracting;
and an optimal distance determination module that calculates an average value of the second sensing distances extracted by the second sensing distance extraction module and then determines the average value of the calculated second sensing distances as an optimal distance. system.

Images