KR102061104B1 - System for cognizing vehicle number plate and method for operating the same - Google Patents

Abstract

Embodiments of the present invention include a front camera which is installed at a first point in a vehicle traveling direction and outputs N front images photographed in time-series order with respect to a front license plate of a vehicle in which the vehicle is traveling; A rear camera installed at a second point rearward from the first point and outputting N rear images photographed in time series with respect to a rear license plate of the same vehicle photographed by the front camera; And extracting any one image among the N front images, selecting the front representative image, extracting any one image among the N rear images, selecting the rear representative image, and then extracting an individual character extracted from the front representative image. A controller for calculating the front individual character reliability, which is the reliability, and the rear individual character reliability, which is the reliability of the individual characters extracted from the rear representative image, and determining the vehicle number by comparing the front individual character reliability and the rear individual character reliability; Can be.

Description

Recognizing vehicle number plate and method for operating the same using individual character reliability

The present invention relates to a license plate recognition system and a recognition method, and to a license plate recognition system and a recognition method through the reliability of individual characters.

Recently, as interest in the smart city construction has increased nationwide, researches on building an intelligent transportation system have been actively conducted. In particular, the technology that recognizes and automatically manages license plates of vehicles running at constant speed on the road is a key element of building an intelligent transportation system.

In general, when acquiring a vehicle image from CCTV installed on the road, the image is distorted due to the inclination of the license plate or the influence of sunlight or lighting. In particular, when acquiring images of a vehicle driving by installing a front CCTV camera and a rear CCTV camera on the same axis as shown in FIG. It may be difficult to secure an optimal visual field according to the conditions, and furthermore, the geometric distortion of the license plate image or the degradation of the image quality may become more difficult, thereby making it difficult to automatically recognize the license plate image.

The license plate image acquired through the front CCTV camera and the rear CCTV camera fixedly installed on the same axis may have various geometrical shapes and appear to be in contact with the border of letters and license plates according to the vertical shooting angle. Can be. In addition, a smear phenomenon may occur under the influence of sunlight or a distorted image may be input due to a blurring phenomenon under the influence of night light. When distortion of the license plate image occurs, detection of license plate candidate regions becomes relatively difficult, which may act as a decisive factor in reducing the overall license plate recognition rate.

On the other hand, since 2016, after taking front and rear images of new and old replacement lanes, the vehicle number is read out with high reliability images.

For example, as shown in FIG. 2, 15 front images are obtained through the front CCTV camera, 15 rear images are obtained through the rear CCTV camera, and the optimal two cut images are obtained. Finally, one cut is selected as the license plate image from the selected two cut images.

However, in the past, when one of the front image and the rear image is selected as the license plate image in a state where the front and rear images are different, the license plate image is not determined without determining the reliability of each individual character in the front image or the rear image. Because of this, it can act as a decisive factor to lower the license plate recognition rate.

For example, when the front and rear images are different from each other, a license plate recognition error may occur because the front image is collectively determined as a vehicle number.

Korean Patent Publication No. 10-005-0075989

The technical problem of the present invention is to provide a license plate recognition means that can improve the recognition rate of the license plate.

Embodiments of the present invention include a front camera which is installed at a first point in a vehicle traveling direction and outputs N front images photographed in time-series order with respect to a front license plate of a vehicle in which the vehicle is traveling; A rear camera installed at a second point rearward from the first point and outputting N rear images photographed in time series with respect to a rear license plate of the same vehicle photographed by the front camera; And extracting any one image among the N front images, selecting the front representative image, extracting any one image among the N rear images, selecting the rear representative image, and then extracting an individual character extracted from the front representative image. A controller for calculating the front individual character reliability, which is the reliability, and the rear individual character reliability, which is the reliability of the individual characters extracted from the rear representative image, and determining the vehicle number by comparing the front individual character reliability and the rear individual character reliability; Can be.

The front camera may be installed at one side of the road in the vehicle traveling direction, and the rear camera may be installed at the other side of the road in the vehicle traveling direction.

If the corresponding individual characters between the individual characters of the front representative image and the corresponding rear representative images match each other, the controller extracts the individual character of the front representative image as a vehicle number member without comparing the reliability of the individual characters, and the individual characters of the front representative image and the rear representative image. If the corresponding individual characters do not match each other, it compares the front individual character reliability with the rear individual character reliability, extracts individual characters with high reliability as the vehicle number member, and combines the extracted vehicle number members to determine the vehicle number. It can be characterized.

In addition, the embodiment of the present invention, the front camera provided at the first point in the direction of the vehicle, the front image providing process for outputting the N front images photographed in time-series order to the front license plate of the vehicle traveling to provide to the controller ; The rear camera installed at the second point rearward from the first point outputs the N rear images photographed in time series with respect to the rear license plate of the same vehicle photographed by the front camera and provides the rear image to the controller. Video presentation process; A representative image selection process of the controller extracting any one image among the N front images and selecting the front representative image, and extracting any one image among the N rear images and selecting the rear representative image; An individual character reliability calculation process of the controller calculating the front individual character reliability which is the reliability of the individual character extracted from the front representative image and the back individual character reliability which is the reliability of the individual character extracted from the rear representative image; And a vehicle number combining process of combining and determining the vehicle number by comparing the reliability of the front individual character and the rear individual character reliability.

In the representative image selection process, the front image located at the center of the time series among the N front images is selected as the front representative image, and the rear image located at the center of the time series among the N rear images is selected as the rear representative image. It may be characterized by.

The representative image selection process calculates the standard deviation between the N front images, selects the front image closest to the standard deviation '0' among the N front images as the front representative image, and selects the standard among the N rear images. The deviation may be calculated to select a rear image closest to the standard deviation '0' among the N rear images as the rear representative image.

The individual character reliability calculation process may be determined according to an image condition including a resolution and a character size of the front representative image or the rear representative image.

The vehicle number combination process may include: extracting a vehicle number member without comparing individual character reliability when the corresponding individual characters between the individual characters of the front representative image and the corresponding rear representative images match each other; When the individual characters of the front representative image and the corresponding individual characters between the rear representative image do not coincide with each other, extracting a high reliability individual character as a vehicle number member by comparing the front individual character reliability and the rear individual character reliability; And a vehicle number confirmation process of combining the extracted vehicle number members to determine the vehicle number.

According to the embodiment of the present invention, it is possible to minimize visual interference due to site conditions (obstacles, light reflections, etc.). In addition, according to an embodiment of the present invention it is possible to improve the recognition rate through the reliability analysis for the individual character.

1 is a view showing the arrangement position of the conventional front camera and the rear camera.
2 is a diagram illustrating a conventional image extraction method.
3 is a block diagram of a license plate recognition system according to an embodiment of the present invention.
4 is a view showing the arrangement position of the front camera and the rear camera according to an embodiment of the present invention.
5 is a block diagram illustrating a controller configuration according to an embodiment of the present invention.
Figure 6 is an illustration of the individual character reliability comparison according to an embodiment of the present invention.
7 is a flowchart illustrating a license plate recognition process according to an embodiment of the present invention.
8 is a flowchart embodying license plate recognition processes according to an embodiment of the present invention.
9 is an exemplary diagram for selecting a representative image of the first scheme according to an embodiment of the present invention.
10 is an exemplary diagram for selecting a representative image of the second scheme according to an embodiment of the present invention.

Advantages and features of the present invention, and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms, and is provided to fully inform the scope of the invention to those skilled in the art. The invention is defined only by the scope of the claims. In addition, in the following description of the present invention, if it is determined that related related art and the like may obscure the gist of the present invention, detailed description thereof will be omitted.

3 is a configuration diagram of a license plate recognition system according to an embodiment of the present invention, Figure 4 is a view showing the arrangement position of the front camera and the rear camera according to an embodiment of the present invention, Figure 5 is an embodiment of the present invention Fig. 6 is a block diagram showing a server configuration according to an example.

According to the present invention, an optimal vehicle license plate reading is performed according to an island type obstacle, light reflection, and the like through the cross arrangement of the front and rear image photographing devices. In addition, the present invention compares the reliability of the individual characters of the license plate with respect to the image taken front / rear to derive the optimal vehicle number.

To this end, the license plate recognition system using the individual character reliability of the present invention may include a front camera 110, a rear camera 120, and a controller 200, as shown in FIG.

As shown in FIG. 4, the front camera 110 is a camera installed at a first point in a vehicle traveling direction. Will print Here, photographing in sequence in time series means that photographing is performed at a constant cycle, for example, that photographing is performed sequentially at 0.01 second intervals. Therefore, when 15 front images are taken in time series, 15 repetitive shots are taken for 0.15 seconds at 0.01 second intervals.

For reference, the front camera 110 may include a lens assembly, a filter, a photoelectric conversion module, and an analog / digital conversion module. The lens assembly includes a zoom lens, a focus lens and a compensation lens. The filter is composed of an optical low pass filter. It may include an Infra-Red cutFilter. Optical low pass filter (Optical Low Pass Filter) to remove the high-frequency components of the optical noise, Infra-red cut filter (Infra-Red cutFilter) blocks the infrared component of the incident light. The photoelectric conversion module converts light from the optical system OPS into an electrical analog signal. The analog / digital conversion module may process the analog signal from the photoelectric conversion module, remove the high frequency noise, adjust the amplitude, and convert the digital signal into a digital signal to generate a front image.

In addition, the rear camera 120 is installed at a second point, which is a rear point than the first point, as shown in FIG. 4, and photographs the rear license plate of the same vehicle photographed by the front camera 110 in time-series order. One or more (N) images are output as rear images.

The output N front images and N rear images are transmitted to an external controller 200 through a wired or wireless communication network.

In addition, the present invention, as shown in Figure 4, the front camera 110 is to be installed on one side of the road in the vehicle travel direction, the rear camera 120 is to be installed on the other side of the road in the vehicle traveling direction front camera 110 The rear camera 120 is arranged to cross each other in an oblique direction. Therefore, by interposing the front camera 110 and the rear camera 120 in an oblique direction, it is possible to minimize the influence of island-like obstacles, light reflection, etc. and to read the license plate optimally.

The controller 200 has the same configuration as a conventional web server in hardware, and the software is implemented through various types of languages such as C, C ++, Java, Visual Basic, Visual C, and the like to perform various functions. Contains modules

The controller 200 extracts one image from the N front images and selects it as the front representative image, and extracts any one image among the N rear images and selects the back representative image. Then, the front individual character reliability, which is the reliability of the individual characters extracted from the front representative image, and the back individual character reliability, which is the reliability of the individual characters extracted from the rear representative image, are calculated, and the comparison is made between the front individual character reliability and the rear individual character reliability. Confirm the vehicle number.

To this end, the controller 200 may include a communication unit 210, a representative image selection unit 220, an individual character reliability calculation unit 230, and a vehicle number combination unit 240 as shown in FIG. 5. .

The communication unit 210 supports hardware and software protocols for communicating with the front camera 110 and the rear camera 120, respectively. The communication unit 210 may perform data communication according to an Internet protocol such as, for example, Transmission Control Protocol / Internet Protocol (TCP / IP).

The representative image selecting unit 220 is a module that extracts any one image from the N front images and selects the front representative image, and extracts any one image from the N rear images and selects the rear representative image.

The individual character reliability calculator 230 is a module for calculating the front individual character reliability that is the reliability of the individual character extracted from the front representative image and the rear individual character reliability that is the reliability of the individual character extracted from the rear representative image. Here, the individual character reliability may be determined according to an image condition including a resolution and a character size of the front representative image or the back representative image. For example, the higher the resolution of the representative image, the higher the individual character confidence. On the contrary, the lower the resolution of the representative image, the lower the individual character confidence is calculated.

For reference, the individual character refers to a unit of a license plate number, letters (Korean, alphabet, etc.), and calculates reliability for each individual character.

The vehicle number combination unit 240 is a module that combines and determines the vehicle number through a comparison between the front individual character reliability and the rear individual character reliability.

That is, the vehicle number combination unit 240 extracts the individual character in the vehicle number member without comparing the individual character reliability when the corresponding individual characters between the individual characters of the front representative image and the corresponding rear representative images match each other.

For example, as shown in FIG. 6, when the front individual character reliability of the front representative image and the rear individual character reliability of the rear representative image are calculated, '1' and the first digit which are the individual characters displayed on the front representative image are calculated. , The second digit '2', the fourth digit '3', the fifth digit '4', the sixth digit '5' and the seventh digit '6' correspond The first digit '1', the second digit '2', the fourth digit '3', the fifth digit '4' and the sixth digit Since '5' coincides with '7' which is the seventh digit, respectively, '1' is extracted as the vehicle number member of the first digit without comparing individual character reliability. The remaining vehicle number members are also determined by combining them without comparing the individual character reliability.

In addition, when the individual characters of the front representative image and the corresponding individual characters between the rear representative image do not coincide with each other, the individual characters having high reliability are extracted as vehicle number members by comparing the reliability of the front individual characters with the rear individual characters.

7 is a flowchart illustrating a license plate recognition process according to an embodiment of the present invention, FIG. 8 is a flowchart embodying a license plate recognition process according to an embodiment of the present invention, and FIG. 9 is a flowchart according to an embodiment of the present invention. FIG. 10 is an exemplary diagram for selecting a representative image of the first scheme, and FIG. 10 is an exemplary diagram for selecting a representative image of the second scheme according to an embodiment of the present invention.

7 and 8, the license plate recognition method of the present invention, the front image providing process (S710), the rear image providing process (S720), representative image selection process (S730), individual character reliability calculation process (S740) , And a vehicle number combination process (S750).

In the front image providing process (S710), the front camera 110 installed at the first point in the vehicle traveling direction outputs N images captured by the front vehicle license plate of the driving vehicle (S712) in time series order as a front image. It provides a process to the controller 200 (S714). Here, photographing in sequence in time series means that photographing is performed at a constant cycle, for example, that photographing is performed sequentially at 0.01 second intervals.

In the rear image providing process (S720), the rear camera 120 installed at the second point behind the first point is photographed in chronological order with respect to the rear license plate of the same vehicle photographed by the front camera 110 ( S722) is a process of outputting the N images as a rear image provided to the controller 200 (S724).

In the representative image selection process (S730), the controller 200 extracts any one image among the N front images and selects the front representative image, extracts any one image among the N rear images, and displays the rear representative image. Is the process of selecting.

There are two ways to select a representative image.

In the first method, as shown in FIG. 9, the front image located at the center of the time series among the N front images is selected as the front representative image, and the rear image located at the center of the time series among the N rear images is rear-viewed. It is to select a representative video. When photographing is performed sequentially in time series, the image located at the center of the time series has a high probability of being the average of the corresponding images, and thus is selected as the representative image.

In the second method of selecting a representative image, a front image having the highest similarity among N front images is selected as a front representative image, and a rear image having the highest similarity among N rear images is selected as a rear representative image. to be. To this end, the representative image selection process (S730) calculates the standard deviation between the N front images, as shown in FIG. 10, and displays the front image closest to the standard deviation '0' among the N front images. The standard deviation between the N rear images is calculated, and the rear image closest to the standard deviation '0' among the N rear images is selected as the rear representative image. A standard deviation, known as a standard deviation, is a number obtained by using the square of the deviation, which is the frequency characteristic value representing the scatter plot of the quantitative characteristic value of the unit of the statistical group. Thus, the standard deviation of '0' in a group is the average value in that group.

In the individual character reliability calculation process (S740), the controller 200 calculates the front individual character reliability, which is the reliability of the individual character extracted from the front representative image, and the back individual character reliability, which is the reliability of the individual character extracted from the rear representative image. It's a process.

The individual character reliability calculation may be determined according to an image condition including a resolution and a character size of the front representative image or the back representative image. For example, even if the front representative image has a resolution of the first digit of the vehicle number having the first resolution, and the resolution of the second digit of the vehicle number has the second resolution, the first digit of the vehicle number to be read The reliability of the individual character can be determined to be higher than the reliability of the individual character in the second digit of the vehicle number to be read.

The vehicle number combining process (S750) is a process in which the controller 200 combines and determines the vehicle number through a comparison between the front individual character reliability and the rear individual character reliability.

To this end, the vehicle number combination process (S750) compares the corresponding individual characters between the individual characters of the front representative image and the corresponding rear representative images (S752), and compares the individual characters of the front representative image with the rear representative images. If the corresponding individual characters are identical to each other, the process of extracting the individual character reliability without comparing the individual character reliability (S754), and the corresponding individual characters between the individual characters of the front representative image and the rear representative image do not match each other. In this case, the process of extracting the individual characters having high reliability as the vehicle number member by comparing the reliability of the front individual character and the reliability of the rear individual character (S756), and the vehicle number confirmation process of determining the vehicle number by combining the extracted vehicle number members (S758). ) May be included.

In detail, the process of extracting the individual character reliability into the vehicle number member without comparing the individual character reliability (S756) may include, for example, the front individual character reliability of the front representative image and the rear individual character reliability of the rear representative image as illustrated in FIG. 6. Is calculated, the first digit '1', the second digit '2', the fourth digit '3' and the fourth digit '4' And the sixth digit '5' and the seventh digit '6' are the first digit '1', the second digit '1', and the second digit '2', Since the fourth digit '3', the fifth digit '4', the sixth digit '5' and the seventh digit '6' are matched, the individual character reliability is not compared. Replace '1' with the first number number member Extract with The remaining vehicle number members are also determined by combining them without comparing the individual character reliability.

In addition, the process of extracting a high reliability individual character as a vehicle number member by comparing the front individual character reliability and the rear individual character reliability (S754), for example, if there is a front representative image and a rear representative image as shown in FIG. In this case, the characters of the third digit, which are the individual characters displayed on the front representative image, become 'almost', and the characters of the third digit, which are the individual characters displayed on the rear representative image become 'A' and do not match with each other. In this case, a character having a high reliability is extracted as a vehicle number member by comparing the reliability of the individual character of 'almost' of the front representative image with the reliability of the individual character of 'a' of the rear representative image. Referring to the drawing, since the reliability of the individual characters of 'almost' of the front representative image is '26% ', and the reliability of the individual characters of' a 'of the rear representative image has '79%', so the character 'ga' is the third Extracted to the seat number member of the seat. Accordingly, the vehicle number finally recognized is determined to be '12'3456'.

For reference, the number of front / back individual character combinations may be as follows.

The embodiments in the above description of the present invention are presented by selecting the most preferred examples to help those skilled in the art from the various possible examples, and the technical spirit of the present invention is not necessarily limited or limited only by the embodiments. However, various changes, modifications, and other equivalent embodiments may be made without departing from the technical spirit of the present invention.

110: front camera
120: rear camera
200: controller

Claims (8)

A front camera installed at a first point in a direction in which the vehicle travels and outputting N front images photographed in a time-series sequence with respect to the front license plate of a traveling vehicle;
A rear camera installed at a second point rearward from the first point and outputting N rear images photographed in time series with respect to a rear license plate of the same vehicle photographed by the front camera; And
Extract any one of the N front images to select the front representative image, extract one of the N rear images to select the rear representative image, and then extract the reliability of the individual character extracted from the front representative image. Calculating the front individual character reliability and the rear individual character reliability, which is the reliability of the individual character extracted from the rear representative image, and including a controller for determining the vehicle number through comparison between the front individual character reliability and the rear individual character reliability,
The front face image is a front face image of which standard deviation is closest to 0 by calculating standard deviations of the N front face images.
The rear representative image is a rear image having a standard deviation closest to 0 by calculating a standard deviation of the N rear images.
The front individual character reliability is calculated from a resolution and a text size of a corresponding text area for each character of the front representative image.
The rear individual character reliability is calculated from the resolution and the character size of the corresponding character area for each character of the rear representation image.
The front camera is installed on one side of the road in the vehicle traveling direction,
The rear camera is installed on the other side of the road in the vehicle travel direction, the front camera and the rear camera is a vehicle license plate recognition system with individual character reliability, characterized in that arranged in an oblique direction to each other.
delete The method according to claim 1, wherein the controller,
If the corresponding individual characters between the individual characters of the front representative image and the corresponding rear representative images match each other, the individual character of the front representative image is extracted as a vehicle number member without comparing the reliability of the individual characters. If the individual characters do not match with each other, the individual characters with high reliability are extracted as the vehicle number member by comparing the reliability of the front and rear characters.
A license plate recognition system using individual character reliability, characterized in that the combination of the extracted vehicle number members to determine the vehicle number.
delete delete delete delete delete

Images