TWI739548B - Method and system for identifying vehicle type - Google Patents

Abstract

The invention provides a method and a system for identifying a vehicle type. The method includes: obtaining, via a camera, a lane image of a lane; in response to determining that a vehicle appears in the lane image, continuously shooting multiple images of the lane through the camera, and converting the images into multiple to-be-stitched images based on a perspective transformation matrix; comparing the to-be-stitched images in pairs to obtain a motion vector between the to-be-stitched images; using a non-linear image fusion method to stitch the to-be-stitched images as a stitched image of the entire vehicle based on the movement vector between the to-be-stitched images; identifying a vehicle type based on the stitched image of the entire vehicle.

Description

Vehicle type identification method and system

The present invention relates to a vehicle identification technology, and particularly relates to a vehicle type identification method and system.

In container yards, port customs and other places, vehicle access control is a very important task. In the past, gates were used to register vehicle data and control access at designated locations. This was time-consuming and labor-intensive. Therefore, automated gates became modern containers. Important construction needs of the customs of the yard and port. Among them, regarding the automatic doorpost model identifier, SICK (SICK LMS 5XX) currently provides radar scanning model identifier technology, but each doorpost requires hardware such as SICK radar, steel frame and civil construction, which is costly. .

In addition, some gates are equipped with cameras for license plate recognition, but due to their limited imaging range and angle of view, they can only capture local features of the vehicle. Therefore, if they are directly used for vehicle type recognition, the accuracy will be lower.

In view of this, the present invention provides a vehicle type identification method and system, which can be used to solve the above technical problems.

The present invention provides a vehicle type identification method suitable for a vehicle type identification system. The method includes: obtaining a lane image of a lane through a camera; responding to the presence of a vehicle in the determined lane image, continuously shooting multiple images of the lane through the camera , And convert the aforementioned image into a plurality of images to be spliced based on a perspective transformation matrix, wherein each image includes an image area corresponding to a part of the vehicle; A motion vector between the two; a non-linear image fusion method is used to splice the previously-mentioned image to be spliced into a whole-vehicle spliced image corresponding to the vehicle based on the motion vector between the two images to be spliced; the method of identifying vehicles based on the whole-vehicle image One vehicle type.

The invention provides a vehicle type identification system, which includes a camera, a storage circuit and a processor. The storage circuit stores multiple modules. The processor is coupled to the camera, the storage circuit, and the processor, and accesses the aforementioned modules to perform the following steps: control the camera to obtain a lane image of a lane; in response to determining that a vehicle appears in the lane image, control the camera to continuously shoot multiple lanes The aforementioned images are converted into multiple images to be spliced based on a perspective transformation matrix, where each image includes an image area corresponding to a part of the vehicle; the aforementioned images to be spliced are compared in pairs to obtain the aforementioned images to be spliced A motion vector between two; a non-linear image fusion method is used to splice the aforementioned images to be spliced into a whole-vehicle spliced image corresponding to the vehicle based on the motion vectors of the aforementioned images to be spliced; A type of vehicle.

Roughly speaking, considering that a single image cannot obtain a complete image of a vehicle entering the gate for correct identification, it is different from the existing static multi-view image splicing technology. The present invention uses computer vision technology to enter multiple continuous dynamic parts of the vehicle in front of the gate. The image uses dynamic single-view image splicing technology to generate a whole-vehicle spliced image of a vehicle for the AI image recognition model to correctly classify and recognize. This method will greatly increase the recognition rate of the vehicle type and avoid additional hardware construction costs.

Please refer to FIG. 1, which is a schematic diagram of a vehicle type identification system according to an embodiment of the present invention. In FIG. 1, the vehicle type identification system 100 includes a camera 101, a storage circuit 102 and a processor 104. In the embodiment of the present invention, the camera 101 is, for example, a license plate recognition camera, which can be fixedly arranged near a lane (for example, a lane located at a gate post), and can be used to capture license plates of vehicles entering this lane. But it is not limited to this.

The storage circuit 102 is, for example, any type of fixed or removable random access memory (Random Access Memory, RAM), read-only memory (Read-Only Memory, ROM), flash memory (Flash memory), hard disk Disk or other similar devices or a combination of these devices can be used to record multiple codes or modules.

The processor 104 is coupled to the storage circuit 102, and can be a general purpose processor, a special purpose processor, a traditional processor, a digital signal processor, multiple microprocessors, one or more combined digital signal processing Microprocessor, controller, microcontroller, Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), any other type of integrated circuit , State machines, processors based on Advanced RISC Machine (ARM) and similar products.

In the embodiment of the present invention, the processor 104 can access the modules and program codes recorded in the storage circuit 102 to implement the vehicle type identification method proposed by the present invention. The details are as follows.

Please refer to FIG. 2, which is a flowchart of a vehicle type method according to an embodiment of the present invention. The method of this embodiment can be executed by the vehicle type identification system 100 of FIG. In addition, in order to make the concept of the present case easier to understand, the following description will be supplemented with FIG. 3, where FIG. 3 is a schematic diagram of multiple images taken of a lane according to an embodiment of the present invention.

First, in step S210, the processor 104 may control the camera 101 to obtain a lane image of the lane. In one embodiment, the processor 104 can detect whether a vehicle appears in the acquired lane image.

In step S220, in response to determining that a vehicle appears in the lane image, the processor 104 may control the camera 101 to continuously shoot multiple images 301 to 318 of the lane, and convert the images 301 to 318 into multiple images based on a perspective transformation matrix. Images to be stitched. In one embodiment, when determining that a vehicle appears in the lane image, the processor 104 may control the camera 101 to shoot an image every 1/4 second, until the total number of captured images reaches a preset number (for example, 18) So far, but the present invention may not be limited to this. That is, the camera 101 will continuously shoot multiple images of the moving vehicle with the image capturing range fixed.

Generally speaking, when the camera 101 is implemented as a license plate recognition camera, its image capturing range is generally limited to the area near the license plate of a conventional vehicle. In this case, each of the images 301 to 318 captured by the camera 101 in step S220 will include an image area corresponding to a part of the vehicle. In other words, when the camera 101 is implemented as a license plate recognition camera, none of the above-mentioned images captures the full picture of the above-mentioned vehicle. For example, images 301 to 311 only capture part of the front of vehicle 399, images 312 and 313 only partially capture the roof and rear of vehicle 399, and images 314 to 318 only capture the rear of vehicle 399. However, the present invention It is not limited to this.

In one embodiment, the purpose of converting the images 301 to 318 into the images to be spliced by the perspective transformation matrix is to make the far and near car-shaped images in the images 301 to 318 can be spliced in the form of histograms with the same width. In one embodiment, the four points in the lane image that are actually the same width at the far end and the near end can be manually selected to transform into the corresponding rectangular four-vertex coordinates with the same width from the near end to the far end (for example, take one of the near end points in the image). The two ends of the front edge of the roof and its middle image have a total of four coordinates at the two ends of the front edge of the roof, which are transformed into the corresponding rectangular four-vertex coordinates of the same width from the near end to the far end) to solve the three-dimensional perspective transformation matrix. Parameters to obtain the perspective transformation matrix used in step S220, but the present invention may not be limited to this.

In one embodiment, after obtaining the perspective transformation matrix, the processor 104 may projective mapping the images 301 to 318 to the above-mentioned images to be spliced based on the perspective transformation matrix, but it is not limited to this.

，N為上述待拼接影像的數量（例如18）。之後，對於上述待拼接影像中的第i+1個待拼接影像而言，處理器104可在所述第i+1個待拼接影像中找出匹配於上述第一特徵樣板的第二特徵樣板，並基於此第二特徵樣板與上述第一特徵樣板之間的位移定義所述第i+1個待拼接影像與所述第i個待拼接影像之間的移動向量，但本發明可不限於此。有關於樣板匹配技術的細節可參考相關的現有技術文獻，於此不另贅述。 After that, in step S230, the processor 104 may compare the images to be spliced in pairs to obtain a motion vector between the images to be spliced. In an embodiment, the processor 104 may obtain the movement vector between the two images to be stitched based on a template matching method, for example. For example, for the i-th image to be spliced in the above-mentioned image to be spliced, the processor 104 may extract a specific block (for example, a part of the body corresponding to the vehicle 399) in the i-th image to be spliced. An image area of) as the first feature model, where

, N is the number of images to be spliced (for example, 18). Afterwards, for the i+1th image to be spliced in the image to be spliced, the processor 104 may find a second feature template matching the first feature template in the i+1th image to be spliced , And define the movement vector between the i+1th image to be spliced and the i-th image to be spliced based on the displacement between the second feature template and the first feature template, but the present invention may not be limited to this . For details about the template matching technology, please refer to related prior art documents, which will not be repeated here.

Then, in step S240, the processor 104 may adopt a non-linear image fusion method to stitch the images to be stitched into a whole-vehicle stitched image corresponding to the vehicle based on the motion vector between the images to be stitched. In different embodiments, the above-mentioned nonlinear image fusion method is, for example, an exponential function or the like, but it may not be limited thereto. In the embodiment of the present invention, two consecutive images may have different exposure conditions, which may cause an obvious junction boundary to appear after the images are spliced and superimposed. In order to make the above-mentioned images to be spliced can be spliced into a smooth full-car spliced image, a non-linear image fusion method (such as an exponential function) is used for improvement.

Please refer to FIG. 4, which is a schematic diagram of the entire vehicle stitched image according to FIG. 3. In this embodiment, the whole-vehicle stitched image 400 is, for example, after the processor 104 converts the images 301 to 318 in FIG. 3 into corresponding images to be stitched, and then stitches the images to be stitched according to the above teaching, but it is not limited to this.

Please refer to FIG. 2 again. In step S250, the processor 104 may identify the vehicle type of the vehicle 399 based on the entire vehicle stitched image 400. In one embodiment, the processor 104 may input the whole-vehicle mosaic image 400 into a pre-trained vehicle type recognition model, where the vehicle type recognition model may reflect the whole-vehicle mosaic image 400 and output the vehicle type of the vehicle 399.

In one embodiment, the above-mentioned vehicle type identification model is, for example, a convolutional neural network (CNN), which may be capable of identifying the type of vehicle 399 based on the entire vehicle mosaic image 400 through a certain training process.

For example, when training the vehicle type recognition model, the entire vehicle mosaic images corresponding to various vehicle types can be collected as training data, so that the vehicle type recognition model can learn the characteristics of vehicles corresponding to various vehicle types. In this way, after completing the training of the vehicle type identification model, the vehicle type identification model can identify the vehicle type of the unknown vehicle when it receives the entire vehicle mosaic image corresponding to an unknown vehicle, but the invention is not limited to this.

In different embodiments, the aforementioned vehicle type identification model may be implemented as LeNet, AlexNet, VGGNet, GoogLeNet, ResNet, DenseNet, etc., but the present invention may not be limited thereto.

In summary, the present invention has at least the following features: (1) It can share a camera with the license plate recognition system, eliminating the need for additional related equipment and hardware costs; (2) Because it can obtain a complete mosaic image of the whole vehicle and the camera The acquisition range of the vehicle is fixed, which can make the generated mosaic images of the whole vehicle have a simple and similar field of view, which can greatly improve the accuracy and recognition rate of the vehicle type recognition model learning; (3) If the camera is shared with the license plate recognition system, this There is a unified standard for the erection height and angle of similar cameras. Therefore, in the future, license plate recognition cameras installed everywhere can obtain a large number of mosaic images of the whole vehicle with the same field of view for training of the vehicle type recognition model, which can further improve the vehicle type recognition rate in the future.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to those defined by the attached patent application scope.

100: Vehicle Type Identification System 101: Camera 102: storage circuit 104: processor 301~318: Video 399: Vehicle 400: Stitching images of the whole car S210~S250: steps

FIG. 1 is a schematic diagram of a vehicle type identification system according to an embodiment of the present invention. Fig. 2 is a flowchart of a vehicle type method according to an embodiment of the present invention. FIG. 3 is a schematic diagram of multiple images taken of a lane according to an embodiment of the present invention. FIG. 4 is a schematic diagram of the stitched image of the whole car according to FIG. 3.

S210~S250: steps

Claims (10)

A vehicle type identification method is suitable for a vehicle type identification system, and the method includes: Obtain a lane image of a lane through a camera; In response to determining that a vehicle appears in the lane image, multiple images of the lane are continuously captured by the camera, and the images are converted into multiple images to be spliced based on a perspective transformation matrix, wherein each of the images includes corresponding to the vehicle Part of the image area; Perform a pairwise comparison of the images to be spliced to obtain a movement vector between the images to be spliced; Adopting a non-linear image fusion method to stitch the images to be stitched into a whole-vehicle stitched image corresponding to the vehicle based on the motion vector between the images to be stitched; Identify a vehicle type of the vehicle based on the stitched image of the entire vehicle. The method according to claim 1, wherein the camera is a license plate recognition camera. The method according to claim 1, wherein at least one of the images to be stitched includes an image area corresponding to the rear of the vehicle. The method according to claim 1, wherein the step of comparing the images to be spliced in pairs to obtain the movement vector between the images to be spliced includes: For i images to be spliced, a specific block is taken from the i-th image to be spliced as a first feature template, where

, N is the number of the images to be spliced; for the i+1th image to be spliced in the images to be spliced, find the i+1th image to be spliced that matches the first feature A second feature template of the template; based on a displacement between the second feature template and the first feature template, the movement between the i+1th image to be spliced and the i-th image to be spliced is defined vector. The method according to claim 1, wherein the step of identifying the vehicle type of the vehicle based on the stitched image of the entire vehicle includes: The whole-vehicle stitched image is input into a pre-trained vehicle type identification model, wherein the vehicle type identification model reacts to the whole-vehicle stitched image and outputs the vehicle type of the vehicle. A vehicle type identification system, including: A camera A storage circuit for storing multiple modules; and A processor, which is coupled to the camera, the storage circuit and the processor, and accesses the modules to perform the following steps: Control the camera to obtain a lane image of a lane; In response to determining that a vehicle appears in the lane image, the camera is controlled to continuously shoot multiple images of the lane, and the images are converted into multiple images to be spliced based on a perspective transformation matrix, wherein each of the images includes corresponding to the vehicle Part of the image area; Comparing the images to be spliced in pairs to obtain a motion vector between the images to be spliced; Adopting a nonlinear image fusion method to stitch the images to be stitched into a whole-vehicle stitched image corresponding to the vehicle based on the motion vector between the images to be stitched; Identify a vehicle type of the vehicle based on the stitched image of the entire vehicle. The system according to claim 6, wherein the camera is a license plate recognition camera. The system according to claim 6, wherein at least one of the images to be stitched includes an image area corresponding to the rear of the vehicle. The system according to claim 6, wherein the step of comparing the images to be spliced in pairs to obtain the movement vector between the images to be spliced includes: For i images to be spliced, a specific block is taken from the i-th image to be spliced as a first feature template, where

, N is the number of the images to be spliced; for the i+1th image to be spliced in the images to be spliced, find the i+1th image to be spliced that matches the first feature A second feature template of the template; based on a displacement between the second feature template and the first feature template, the movement between the i+1th image to be spliced and the i-th image to be spliced is defined vector. The system according to claim 6, wherein the step of identifying the vehicle type of the vehicle based on the stitched image of the entire vehicle includes: The whole-vehicle stitched image is input into a pre-trained vehicle type identification model, wherein the vehicle type identification model reacts to the whole-vehicle stitched image and outputs the vehicle type of the vehicle.

Images