KR20140093407A - Recognition device, vehicle model recognition apparatus and method - Google Patents

Abstract

According to the present invention, a vehicle model recognition apparatus includes: a learner which builds a database by acquiring a reference image of a vehicle; and a recognizer which acquires an image of the vehicle in any direction of the vehicle, processes the acquired image, and compares the processed image with information stored in the database to recognize the vehicle. The recognizer includes: an image acquisition unit which acquires an image in any direction; a license plate detection unit which detects a license plate of a vehicle from the acquired image; an interest region extraction unit which extract a region of interest in a direction of the reference image acquired by the learner; a descriptor generation unit which represents the region of interest as descriptors; and a reading unit which compares the descriptors with the database. Therefore, the present invention has the advantages of improving a recognition rate of a vehicle model, accurately recognizing a vehicle model from even an image of a vehicle with an arbitrary rotating angle, enabling an industrial application by improving recognition speed, and conveniently imposing fines on a vehicle with a problem that the license plate is damaged.

Description

{Recognition Device, VEHICLE MODEL RECOGNITION APPARATUS AND METHOD}

The present invention relates to a recognizer and an apparatus and method for recognizing a model of a vehicle. More particularly, the present invention relates to an apparatus and method for acquiring an image of a vehicle and automatically obtaining a model of the vehicle name and model.

The recognition technology of the license plate information of the vehicle is widely used in toll roads and the like. The license plate information is information capable of recognizing a specific vehicle. However, when the camouflaged license plate is attached, there is a problem in that information related to the vehicle can not be tracked at all. To improve such a situation, a technique of recognizing a vehicle model has been proposed. If the vehicle model is recognized along with the license plate, it can be determined whether the license plate is a license plate or not.

In the case of detecting the vehicle model, it is possible to conveniently charge the vehicle on the toll road, confirm whether or not the vehicle is a stolen vehicle by confirming whether the license plate and the vehicle model are matched with each other, And can perform a role as unique recognition information.

The apparatus and method for recognizing a model of an existing vehicle have a problem of low recognition rate. This is because the rotation angle with respect to the image of the vehicle to be acquired is not taken into consideration as a factor that lowers the recognition rate. For example, the database includes a front image of the vehicle, and the current vehicle image to be compared may be an image that is rotated a little bit. However, the image of a slightly rotated vehicle is significantly different from the front image recognition process, so high accuracy can not be expected. Therefore, it is necessary to make the photographing direction of the image stored in the database coincide with the photographing direction of the currently acquired image. However, it is practically difficult to acquire an accurate front image while acquiring an image of a running vehicle.

It is an object of the present invention to provide a vehicle model recognition apparatus capable of easily acquiring a vehicle image, raising a recognition rate at the time of recognizing a vehicle model, And method.

A vehicle model recognition apparatus according to the present invention includes a learning device for acquiring a reference image of a vehicle and constructing a database, and a learning device for acquiring an image in an arbitrary direction of the vehicle, processing the image and comparing the information with the information of the database Wherein the recognizer includes: an image acquiring unit that acquires an image in an arbitrary direction; A license plate detector for detecting a license plate of the vehicle from the acquired image; An interest region extractor for extracting a region of interest having a rotation angle and a size similar to those of the learning device, based on the license plate; A descriptor generator for expressing the region of interest as a descriptor; And a reader for comparing the descriptor with the database.

Here, the learning device acquires an image in the frontal direction, and the recognizer is an apparatus that acquires an image in an arbitrary direction on the front surface. Also, the ROI extractor may compare the left and right images with respect to the license plate to obtain a ROI.

The recognizer of the present invention according to another aspect includes: an image obtaining unit that obtains an image in an arbitrary direction; A license plate detector for detecting the license plate in the acquired image; An interest region extractor for extracting a region of interest on the basis of the license plate; A descriptor generator for expressing the region of interest as a descriptor; And a reader for comparing the descriptor with the database, wherein the ROI extractor includes: a contrast image generator for generating a pair of right and left contrasted images based on the plate detected; A rotation information extractor for comparing rotation images of the vehicle with the reference images; And an image calibrator for calibrating the image using the rotation information.

Here, the rotation information may be provided in the form of a conversion function, and the pair of images to be collated may be left and right images at the center of the license plate, and one of them may be Y-axis reversed image.

According to another aspect of the present invention, there is provided a vehicle model recognition method comprising: inputting a vehicle image in an arbitrary direction; The license plate is detected in the incoming vehicle image; Extracting a region of interest using the license plate; Expressing the region of interest with a descriptor; And comparing the vehicle model in the DB to identify the vehicle model, wherein extracting the ROI includes: generating a pair of contrastable images; Comparing the pair of images to find rotation information of the vehicle; And correcting the image using the rotation information.

Here, the rotation information may be expressed by a conversion function. Also, generating a pair of images may include adjusting the height of the vehicle image using an average of upper and lower edges of the license plate; Selecting a predetermined region as a region of interest according to the magnification of the license plate; And Y-axis inversion of either image about the longitudinal center axis of the license plate. In addition, finding the rotation information of the vehicle is based on the marking, which removes an area unnecessary for marking in the image; Extracting feature points with horizontal lines drawn at regular intervals; Mapping feature points in the pair of images that can be contrasted; And expressing the rotation information as a transformation function with reference to the minutiae to be matched.

There is an advantage that the recognition rate of the vehicle model according to the present invention is increased, the image of the vehicle having an arbitrary rotation angle can be recognized accurately, and the speed of recognition is fast. Vehicles with problems such as plate breakage may be fined for convenience.

1 is a flowchart of a vehicle model recognition method;
Figure 2 is a flow chart detailing the steps of extracting the region of interest;
Figure 3 is a flow diagram illustrating in more detail the creation of a pair of images that can be contrasted.
Figure 4 is an illustration of the area of the license plate as the images are aligned.
5 illustrates an example in which a front surface of a vehicle is included in an area of interest;
FIG. 6 is a table showing a setting area of a region of interest according to the type of the plate; FIG. 7 is a diagram showing a practical example of a region of interest;
8 is a diagram illustrating that the left side of the ROI is Y-axis inverted.
9 is a flowchart showing details of a step of extracting rotation information of a vehicle.
FIG. 10 is a view showing a pair of images in which an area close to a license plate and a blank space due to a lack of image size are removed in the verification area of FIG. 8;
11 is a view showing a pair of right-side drawings showing a light area, and a pair of left-side drawings showing minutiae with a light area removed.
12 is a diagram showing corner points and vertical edge components extracted as feature points in a pair of right and left images;
In Fig. 13, the left figure shows the result of mapping between arbitrary feature points, and the right figure shows the result of mapping the feature points of similar height.
FIG. 14 is a diagram showing the left figure before applying the proposed error elimination algorithm and the right figure after being applied. FIG.
15 shows an example of a conversion function obtained by using the conversion points and the conversion points.
16 is a diagram illustrating that an image is corrected;
17 is a block diagram of a vehicle model recognition apparatus according to the present invention.
18 is a diagram showing a detailed configuration of the ROI extractor.
19 shows a detailed configuration of the contrast image generator.
20 is a view showing a detailed configuration of the rotation information extractor;

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the spirit of the present invention is not limited to the following embodiments, but a person skilled in the art who understands the present invention can easily suggest other embodiments included in the scope of the same idea by adding, changing, deleting, However, this will also be included in the spirit of the present invention.

<Vehicle model recognition method>

1 is a flowchart of a vehicle model recognition method.

Referring to FIG. 1, a front image of a vehicle that can be rotated at an arbitrary angle is input (S1). A license plate is detected in the input image (S2). The input S1 of the image of the vehicle and the detection S2 of the license plate in the input image can be performed by various methods known in the art.

When the vehicle license plate is detected, the input image is corrected and the region of interest is extracted so as to match with the rotation state of the image stored in the database (S3).

The extracted region of interest is represented by a descriptor (S4), and the vehicle model is compared with the image stored in the database (S5).

Extraction (S3) of the region of interest may include the steps shown in Fig.

First, a pair of images capable of cross-checking are generated using the left and right partial regions around the license plate (S31). The pair of images are compared to extract information related to the degree of rotation of the vehicle (S32). Herein, the information related to the degree of rotation of the vehicle may refer to a conversion function. The transform function serves as a key for correcting the image. An image of a region of interest is corrected using the transformation function to generate an image of a region of interest (S33).

FIG. 3 is a flow chart illustrating in more detail the generation of a pair of contrastable images (S31).

Referring to FIG. 3, the slope of the upper edge and the lower edge of the license plate detected in step S2 in which the license plate is detected is calculated (S41). The average of the slopes of the upper and lower corners of the license plate is obtained, and the image is newly rearranged according to the average slope (S42). Figure 4 illustrates the area of the license plate as the image is aligned. This allows the skew image to be aligned to the same height level due to the rotated image.

Thereafter, the region of interest is set with the license plate as a predetermined reference (S43). In setting the area of interest, the front view of the vehicle, which can be used to identify the vehicle model, must be visible. Therefore, even in the case of a vehicle showing a predetermined rotation state, the front surface shape of the vehicle must be included in the ROI, and the front surface shape of the vehicle must be included in the ROI even when the vehicle type is different, such as passenger car / SUV / truck / heavy equipment. An example of a region of interest that meets these requirements is shown in FIG.

A method of setting the region of interest so as to satisfy the above requirements will be described in more detail. FIG. 6 is a table showing a setting area of a region of interest according to the type of plate, and FIG. 7 shows a practical example. 6 and 7, the area of interest is set according to how many times the horizontal and vertical sizes of the license plate are centered on the center of the upper edge of the license plate detected in the license plate detecting step S2. For example, '39 or 2764', shown in the first row in FIG. 6, is a plate of 335 * 170 mm in size. In this case, the width of the area of interest is a range of multiplying the width of the plate by 6.21, and the height of the plate is an area multiplied by 1.29 downward from the center of the upper edge of the plate, multiplied by 2.59 upward . As a result, the width of the entire region of interest is 2080 mm in height and 660 mm in height. 7 shows that the region of interest is set in the case of a license plate of the size of 335 * 170, and the region of interest is set in the case of the license plate of 520 * 110 in the left image of FIG.

The size of the region of interest is unified for data comparison. For example, an image of 360 * 96 pixels in size would be advantageous to increase processing speed in subsequent image processing, which would allow the concept to be applied to the vehicle recognition process of a moving vehicle . On the other hand, in Fig. 5, the blank left of the image may be a part where information disappears in the image alignment, or information of a part where there is no image at the time of setting a region of interest.

After the step of setting the ROI (S43), the ROI is divided into two based on the central longitudinal axis of the license plate, and the arbitrarily specified one is reversed by Y axis. It is better to compare the divided areas than before. 8 illustrates that the left side of the ROI is Y-axis reversed. Only the region with much information is used for classification through the later process.

Through the above process, it can be seen that a pair of images that can be verified are extracted. That is, when the left image of the ROI is reversed in FIG. 8, the images are different images as the left and right sides of the front surface of the vehicle, but they can be contrasted with each other as information having the same directionality.

Thereafter, the rotation information extraction step S32 of the vehicle is performed. 9 is a flowchart showing in detail the step of extracting rotation information of the vehicle.

Referring to FIG. 9, in order to suppress an error at the time of collating a pair of images that can be verified, margins such as a middle region of the vehicle close to the license plate and a blank region due to insufficient image information are removed (S51 ). FIG. 10 is a view showing a pair of images from which margins are removed in the verification area of FIG. 8. FIG.

On the other hand, in the light area of the vehicle, there are many wavy shapes in the lens in order to improve the refraction and straightness of the light, and when light is reflected, many outliers may occur due to refraction and the like. As a result, there are many possibilities of causing a lot of errors in collating images. Therefore, the light area is removed from the image processing (S52). In Fig. 11, a pair of right drawing shows the light area, and a pair of left drawing shows the minutiae with the light area removed. It can be seen that the minutiae disappear inside the light area.

In order to collate a pair of images, feature points are extracted from each image (S53). Specifically, the horizontal lines are drawn at regular intervals (8 to 10) on each image. At this time, the pixel value of the horizontal line can be changed to "255-original pixel value". Then FAST feature points are extracted using the FAST (Features from Accelerated Segment Test) along the horizontal line. At this time, corner and vertical edge components will be extracted as FAST feature points. 12 is a diagram showing FAST feature points extracted from a pair of right and left images. When extracting minutiae points, the feature points are obtained by drawing a horizontal line and using FAST. However, without being limited to this method, SURF and SIFT methods may be used as another method of extracting feature points. It is desirable to extract the FAST feature points after the horizontal line is drawn so that the correct feature points are matched with each other. However, FAST can efficiently detect corner points in the embodiment in which the size of the ROI used is fixed small. In addition, points having strong vertical edges can be detected constantly by a simple method of arbitrarily drawing horizontal lines, and as the search direction is horizontal as in the embodiment, it can be a useful feature point. By increasing the number of available feature points through this, the feature points become more marked. Therefore, more accurate conversion functions can be created based on the increased information of the metric and used for image correction.

After the minutiae points are extracted from the left and right pair of images, minutiae of similar height are respectively referred to in the left and right images (S54). The matching of feature points of similar height is intended to reduce omega. In Fig. 13, the left figure shows the result of matching the feature points having the most similar information, and the right figure shows the result of weighting the feature points of similar height. It can be confirmed that the result of the matching of the minutiae points is improved. In order to further increase the accuracy of the matching of the minutiae points, mis-matched minutiae are removed using the concept of RANSAC (Radom SAmple Consensus) algorithm, which is one of the algorithms for reducing errors (S55). In Fig. 14, the left figure before the omega elimination algorithm is applied and the right figure after the application are compared well. In the figure on the right, we can see that the minutiae are more correctly matched to each other.

After the feature points to be matched with each other are detected, a conversion function is obtained so that the rotation information of the vehicle can be obtained (S56). The generation process of the conversion function is exemplarily shown in Fig. The horizontal axis of FIG. 15 is a position on the horizontal axis of the feature point when the left or right is taken as a reference, and the vertical axis of FIG. 15 represents the difference of horizontal values of the feature points to be matched. The information is indicated by dots in the lower figure of Fig. Finds the curve that contains the most points and classifies the information that is not included in the curve as an error. In FIG. 15, the points included in the final curve to be used as the conversion function are shown in red, and the points classified as errors are shown in green.

That is to say, the conversion function of FIG. 15 shows that the feature points that are mapped to the right become much different in the relative positions on the image. In this case, it may mean that the image has to be corrected more in the correction step S33 of the image. Figure 16 shows an example in which the image is corrected. In detail, referring to the conversion function, the left image is corrected to the right image. The corrected image is represented by a descriptor (S4), and the model of the vehicle can be recognized by comparing the information in the database (S5). It can be easily guessed that when the image is calibrated, the rotation state is corrected to the front state similar to the rotation state of the image stored in the database.

The vehicle model can be recognized according to the above-described method.

The results of experiments on vehicle model recognition under the above background are as follows. The frontal model of the vehicle should be used as database information. In the case where the image of the currently recognized vehicle is processed by using the front face (Case 1), the vehicle model recognition method of suggesting the rotated image (Case 2), the case of directly recognizing the rotated image 3).

As a result, it was confirmed that the recognition rate of Case 1 was 94 ~ 100%, that of Case 2 was 88 ~ 95%, and that of Case 3 was 67 ~ 85%. The recognition rate of the model was further improved.

For the recognition time, 0.08s for case 1/3, and 0.06s for case 2, which took 0.14s because of image correction. It was confirmed that there was no problem in any case or industrial application.

&Lt; Vehicle model recognition device >

Hereinafter, a vehicle model recognizing apparatus according to an embodiment will be described. The vehicle model recognition apparatus performs a process similar to the vehicle model recognition method, and can refer to the vehicle model recognition method.

17 is a block diagram of a vehicle model recognition apparatus.

17, a learning device 1 capable of producing a database 15 and a recognizer 2 recognizing a vehicle model by referring to the information of the database 15 are included.

The learning device 1 is provided with a reference direction image acquiring device 11 for acquiring an image of a vehicle in a reference direction, a detector 12 for detecting a license plate, and an interest area extracting device 13 are provided. If the vehicle is completely in front, the region of interest can be extracted as it is, expressed in the descriptor in the descriptor generator 14, and stored in the DB 15.

The recognizer 2 includes an arbitrary direction image acquiring unit 21 for acquiring an image of the vehicle in an arbitrary direction, a detector 22 for detecting a license plate, and a region of interest extractor 23 for extracting a region of interest do. The ROI extractor 23 should correct the rotated image so that it can be compared with the image of the reference direction. The region of interest extracted by the ROI extractor 23 is represented by a descriptor in the descriptor generator 24 and then the vehicle model is recognized through the operation in which the reader 25 verifies the information in the DB 15 .

18 shows a detailed configuration of the ROI extractor. The interest area extractor 23 includes a contrast image generator 31 for generating a pair of images so as to process an image, a rotation information extractor 32 for extracting rotation information using a pair of images to be collated, And an image calibrator 33 for calibrating the image using the conversion information extracted by the rotation information extractor 32. [

19 shows a detailed configuration of a contrast image generator. First, the contrast image generator 31 includes a tilt regulator 41 for aligning the vertical height of the image using the slope information of the license plate, and a right and left image within a predetermined range as the center of the license plate in the slope- An interest region setter 42 for setting a region of interest as much as possible, and a Y-axis inverter 43 for inverting the Y-axis on the basis of the center of the license plate.

20 is a diagram showing the detailed configuration of the rotation information extractor. The rotation information extractor includes an error region remover 51 for removing an error region such as a blank and a light region on a pair of images to be collated, a feature point extractor 52 for extracting a feature point in a state where an error region is removed, And a conversion function generator 54 for generating a conversion function by comparing feature points matched in the pair of images with each other. The rotation information of the vehicle can be expressed in the form of a conversion function.

The determined conversion function is passed to the image calibrator 33 and used to calibrate the image. The image corrector uses the transformation function to transform the image so that it is as close as possible to the orientation of the image acquired in the reference direction image acquirer 11. [ Image correction can be understood as drawing the image at each point corresponding to the intensity of the transform function.

In the vehicle model recognizing method, the recognizer, and the vehicle model recognizing device, the front image proposed as the data of the database is inputted and the front image of the rotated vehicle is inputted in the field. However, not so, you can also enter a rotated image in the database. However, for better recognition results and convenience of data management (if rotated data should be categorized for each angle), the image with the base image stored in the database in front is better.

Although the front of the vehicle is used as a reference in the above embodiment, the model may be grasped by using the rear surface.

1: Learning machine
2: recognizer

Claims (10)

A learning device for acquiring a reference image of a vehicle and constructing a database; and a recognizer for acquiring an image in an arbitrary direction of the vehicle, processing the image, and recognizing the vehicle by comparing the information with the information of the database ,
In the recognizer,
An image obtaining device for obtaining an image in an arbitrary direction;
A license plate detector for detecting a license plate of the vehicle from the acquired image;
An interest region extractor for extracting a region of interest in the direction of a reference image of the learning device based on the license plate;
A descriptor generator for expressing the region of interest as a descriptor; And
And a reader for comparing the descriptor with the database. The method according to claim 1,
The learning device acquires an image in the front direction, and the recognizer acquires an image in an arbitrary direction on the front face. The method according to claim 1,
Wherein the interest region extractor compares left and right images centering on the license plate to obtain a region of interest. An image acquiring unit that acquires an image in an arbitrary direction;
A license plate detector for detecting the license plate in the acquired image;
An interest region extractor for extracting a region of interest on the basis of the license plate;
A descriptor generator for expressing the region of interest as a descriptor; And
And a reader for comparing the descriptor with the database,
In the ROI extractor,
A contrast image generator for generating a pair of right and left contrasted images based on the detected plate;
A rotation information extractor for comparing rotation images of the vehicle with the reference images; And
And an image calibrator for calibrating the image using the rotation information. 5. The method of claim 4,
Wherein the rotation information is provided in the form of a transform function. 5. The method of claim 4,
Wherein the pair of images to be collated is an image of left and right sides at the center of the license plate, and one of the images is Y-axis reversed image. Inputting a vehicle image in an arbitrary direction;
The license plate is detected in the incoming vehicle image;
Extracting a region of interest using the license plate;
Expressing the region of interest with a descriptor; And
And comparing the vehicle model with the DB,
Extracting the region of interest,
Generating a pair of images that can be contrasted;
Comparing the pair of images to find rotation information of the vehicle; And
And calibrating the image using the rotation information
A vehicle model recognition method. 8. The method of claim 7,
Wherein the rotation information is represented by a transform function. 8. The method of claim 7,
To generate a pair of images,
Adjusting the height of the vehicle image using an average of upper and lower edges of the license plate;
Selecting a predetermined region as a region of interest according to the magnification of the license plate; And
And reversing the Y axis of either one of the images about the longitudinal center axis of the license plate. 8. The method of claim 7,
To obtain the rotation information of the vehicle,
Removing unnecessary areas of the image;
Extracting feature points with a horizontal line drawn;
Mapping feature points in the pair of images that can be contrasted; And
And expressing the rotation information as a conversion function by referring to the minutiae to be matched.

Images