KR102654570B1 - Method, system and computer readable storage medium to recognize vehcle model based on computer vision - Google Patents

Abstract

The present disclosure relates to a method, system, and computer-readable storage medium for computer vision-based vehicle model recognition. A system for computer vision-based vehicle type recognition includes an image database unit that stores bottom images of a plurality of vehicles; a lower camera that captures the bottom of the entering vehicle; and an integrated management server. The integrated management server may include a vehicle type determination unit that determines the vehicle type of the entering vehicle based on the bottom image of the entering vehicle captured from the lower camera. The vehicle type determination unit segments the object of interest from the bottom image of the entering vehicle, and calculates the object of interest from the predicted bounding box of the object of interest in the bottom image of the entering vehicle and the reference bounding box of the object of interest in the bottom image of the plurality of vehicles stored in the image database unit. Each IoU can be measured, and the first candidate vehicle model can be determined based on the measured IoU.

Description

Method, system and computer-readable storage medium for computer vision-based vehicle model recognition {METHOD, SYSTEM AND COMPUTER READABLE STORAGE MEDIUM TO RECOGNIZE VEHCLE MODEL BASED ON COMPUTER VISION}

The present disclosure relates to a method, system, and computer-readable storage medium for computer vision-based vehicle model recognition. More specifically, a method, system, and computer-readable storage medium for determining a vehicle type by utilizing IoU (Intersection over Union), which is generally used to evaluate performance in the field of object detection technology, for vehicle type recognition are described.

Unless otherwise stated herein, the material described in this section is not prior art to the claims in this application and should not be acknowledged as prior art by virtue of being described in this section.

There are many types of cars on the road with different purposes, appearance, size, color, or manufacturer. Depending on the vehicle type, the vehicle's tax benefits, fees collected on toll roads or paid parking lots, and accessible locations may vary. Additionally, since each vehicle type has unique characteristics that are distinct from each other, it can also be used to identify the vehicle. In general, LPR (license plate recognition) technology, which recognizes and reads a vehicle's license plate, is mainly used to identify a vehicle. However, in cases where a vehicle's license plate is forged or altered for the purpose of violating the law or avoiding charging fees for the vehicle, LPR In case the vehicle's license plate is misrecognized or unrecognized by the device, it is necessary to recognize the vehicle using a means other than the vehicle's license plate. Accordingly, a method of identifying a vehicle using the vehicle model instead of the vehicle license plate may be considered.

For example, in Registered Patent Publication No. 10-1301868, a vehicle type classification system that extracts the vehicle type from the captured image by clearing the image to extract characteristics according to the type of vehicle and then comparing it with various vehicle data preset in a database; and The method is disclosed. Here, the vehicle data used to classify the vehicle type is the shape of the rearview mirror, the shape of the outer peripheral surface of the side window, and the outer peripheral shape of the roof, and the prior patent focuses on the visible exterior characteristics of the vehicle to extract the vehicle type.

However, there is a problem that the exterior of the vehicle used to extract vehicle characteristics in the prior patent can be easily modified, and the method of simply comparing captured images and vehicle data is limited, so the vehicle type classification performance is limited. Therefore, there is a need to implement a reliable vehicle type recognition system by increasing the accuracy of vehicle type recognition.

KR 10-1301868 B1

The present disclosure is intended to solve the above problems. It increases the efficiency of vehicle type recognition by using the bottom part of the vehicle, which has better distinguishing features than the exterior of the vehicle and is difficult to modify after shipment, and uses IoU when analyzing images to improve vehicle type recognition. The performance of deep learning models can be improved.

In one embodiment, a system for computer vision-based vehicle type recognition is described. The system includes a lower camera that captures the bottom of the entering vehicle; and an integrated management server. The integrated management server may include a vehicle type determination unit that determines the vehicle type of the entering vehicle based on the bottom image of the entering vehicle captured from a lower camera and an image database unit that stores bottom images of a plurality of vehicles. The vehicle type determination unit segments an object of interest from the bottom image of the entering vehicle, and a plurality of vehicles stored in the image database unit corresponding to the predicted bounding box of the object of interest in the bottom image of the entering vehicle generated by segmentation. Each IoU (Intersection over Union) can be measured from the reference bounding box of the object of interest in the bottom image. The vehicle type determination unit may determine the first candidate vehicle type based on the measured IoU.

In one example, the vehicle type determination unit may determine the largest among the measured IoUs as the first candidate vehicle type. Additionally, the object of interest that the vehicle type determination unit focuses on may be a pipe of the vehicle.

In one example, the vehicle type determination unit may recognize a plurality of tires from the bottom image of the entering vehicle and detect the midpoint of each recognized tire before segmenting the object of interest. The vehicle type determination unit improves the accuracy of IoU measurement by standardizing the bottom images of the entering vehicle captured based on the plurality of detected midpoints, thereby matching the midpoints between the bottom images of the entering vehicle and the bottom images of the multiple vehicles stored in the image database unit. You can do it.

In one example, the system may further include an upper camera that photographs the exterior of the entering vehicle. Here, the vehicle type determination unit may determine the second candidate vehicle type based on the exterior image of the entering vehicle captured from the upper camera. The vehicle type determination unit can improve the reliability of vehicle type recognition by checking whether the first candidate vehicle type and the second candidate vehicle type match.

In one example, the system may further include an LPR device that recognizes a license plate number of an entering vehicle. The integrated management server may further include a forgery detection unit that determines whether the entering vehicle has been forged or altered, and a specifications database unit that stores the registered vehicle model corresponding to each vehicle number. Here, the forgery detection unit can check the registered vehicle model corresponding to the recognized vehicle number of the entering vehicle from the specifications database unit and check whether the registered vehicle model and the first candidate vehicle model match. In addition, if the forgery detection unit determines that the first candidate vehicle model and the registered vehicle model do not match, a warning notification is sent to the management server that manages forgery and alteration of the vehicle, for example, for the purpose of avoiding illegal or vehicle fees. Vehicles with forged or altered license plates can be cracked down on.

In another example, the system may further include an RFID reader that recognizes the vehicle's unique ID from an RFID tag included in the entering vehicle. The integrated management server may further include a forgery detection unit that determines whether the entering vehicle has been forged or altered, and a specifications database unit that stores the registered vehicle model corresponding to the unique ID of each vehicle. Here, the forgery detection unit can check the registered car model corresponding to the unique ID of the entering vehicle from the specifications database unit and check whether the registered car model and the first candidate car model match. In addition, if the forgery detection unit determines that the first candidate vehicle model and the registered vehicle model do not match, a warning notification is sent to the management server that manages forgery and alteration of the vehicle, so that, for example, vehicles with illegal RFID tags attached can be cracked down. .

In another embodiment, a method for computer vision-based vehicle model recognition is described. The method includes receiving a bottom image of an entering vehicle captured by a lower camera; Segmenting an object of interest from a bottom image of an entering vehicle; And it may include measuring each IoU from the reference bounding box of the object of interest in the bottom image of the plurality of vehicles stored in the image database unit corresponding to the predicted bounding box of the object of interest in the bottom image of the entering vehicle. Additionally, the method may include determining a first candidate vehicle model based on the measured IoU.

In one example, determining the first candidate vehicle model may include determining the largest of the measured IoUs as the first candidate vehicle model. Additionally, the object of interest may be a pipe of a vehicle.

In one example, the method includes recognizing a plurality of tires from an underside image of an entering vehicle prior to segmenting an object of interest; detecting the midpoint of each recognized tire; And it may further include standardizing the bottom image of the entering vehicle captured based on the plurality of detected midpoints.

In another embodiment, a computer program stored on a recording medium of the integrated management server and executed by the integrated management server for computer vision-based vehicle model recognition is described. The computer program stored in the recording medium of the integrated management server includes at least one command related to receiving a bottom image of an entering vehicle captured by a lower camera; at least one instruction related to segmenting an object of interest from an underside image of an entering vehicle; At least one command related to measuring each IoU from a reference bounding box of the object of interest in the bottom image of a plurality of vehicles stored in the image database unit corresponding to the predicted bounding box of the object of interest in the bottom image of the entering vehicle; and at least one command related to determining the first candidate vehicle model based on the measured IoU.

The above brief summary and description of effects are merely illustrative and are not intended to limit the technical details intended in the present disclosure. By referring to the following detailed description and the accompanying drawings, additional embodiments and technical features, in addition to the above-described exemplary embodiments and technical features, may be understood.

The features of the present disclosure described above and other additional features will be described in detail below with reference to the attached drawings. These drawings illustrate only a few embodiments according to the present disclosure and should not be considered to limit the scope of the technical idea of the present disclosure. The technical idea of the present disclosure will be described more specifically and in detail using the attached drawings.
1 is a schematic diagram illustrating an exemplary system for vehicle type recognition, according to at least some embodiments of the present disclosure.
2 is a block diagram illustrating an example integrated management server for vehicle type recognition, according to at least some embodiments of the present disclosure.
3 illustrates an example bottom view image of an approaching vehicle, according to at least some embodiments of the present disclosure.
4 illustrates an integrated management server measuring IoU, according to at least some embodiments of the present disclosure.
5 is a flowchart illustrating an example process performed on a computing device for computer vision-based vehicle model recognition, according to at least some embodiments of the present disclosure.
6 illustrates an example computer program product that can be used to recognize a vehicle type based on computer vision, in accordance with at least some embodiments of the present disclosure.

Hereinafter, with reference to the attached drawings, implementation examples and embodiments of the present disclosure will be described in detail so that those skilled in the art can easily implement the present disclosure. However, the present application may be implemented in many different forms and is not limited to the implementation examples and examples described herein.

The present disclosure generally relates to a method, system, and program stored in a computer-readable storage medium for computer vision-based vehicle model recognition.

1 is a schematic diagram illustrating an example system 100 for vehicle type recognition, in accordance with at least some embodiments of the present disclosure. The exemplary system 100 includes an integrated management server 110, a vehicle 120, a bottom image recognition device 130, an LPR device 140, a blocking device 145, an RFID reader 150, and an upper camera 160. may include.

The integrated management server 110 may be an application server, a stand-alone server, a web server, and any other computing device equipped with data transmission/reception functions, data identification functions, and data processing functions. The integrated management server 110 can, for example, process a process for computer vision-based vehicle model recognition, including a deep learning model, and the specific configuration will be described in more detail in FIG. 2.

The vehicle 120 may be any type of vehicle with a different purpose, appearance, size, color, or manufacturer. The vehicle 120 may wait for vehicle recognition at any location where vehicle recognition is required, such as a location where security is required or where payment of fees charged to the vehicle 120 is required.

The bottom image recognition equipment 130 may include a first control unit, a first vehicle detection unit, and a first communication unit. The first control unit can control the first vehicle detection unit and the first communication unit of the bottom image recognition equipment 130 and can be linked with the integrated management server 110. When the vehicle 120 approaches the bottom image recognition equipment 130, the first vehicle detector may recognize the vehicle 120 using, for example, laser or pressure detection. The first vehicle detection unit may include a lower camera and may capture a bottom image of the recognized vehicle 120. The lower camera may be a computer vision camera, for example, a line scan camera. The first communication unit may transmit the bottom image of the vehicle 120 captured by the lower camera to the integrated management server 110.

The LPR device 140 may include an LPR control unit, an LPR vehicle detection unit, and an LPR communication unit. The LPR control unit can control the LPR vehicle detection unit and the LPR communication unit of the LPR device 140 and can be linked with the integrated management server 110. When a vehicle 120 approaches the LPR device 140, the LPR vehicle detection unit may recognize the vehicle number by capturing and analyzing an exterior image of the vehicle 120. The LPR vehicle detection unit may include a camera and may use appropriate image analysis and recognition algorithms, such as OCR technology, to analyze vehicle images. The LPR communication unit may transmit the vehicle number recognized by the LPR vehicle detection unit to the integrated management server 110. Alternatively, the LPR communication unit may transmit the exterior image captured by the LPR vehicle detection unit to the integrated management server 110, and the integrated management server 110 may recognize the vehicle number.

For example, the blocking device 145 may function to block the vehicle 120 waiting for security screening or fare payment until entry or exit is permitted through a vehicle type recognition process. When the integrated management server 110 allows entry or exit of the vehicle 120, the blocking device 145 opens the blocking bar to allow the vehicle 120 to pass.

The RFID reader 150 can transmit and receive information to and from the RFID tag included in the vehicle 120, and can transmit information about the unique ID of the vehicle 120 collected from the RFID tag to the integrated management server 110. The upper camera 160 can capture the exterior of the vehicle 120 and transmit the exterior image of the vehicle 120 to the integrated management server 110.

Although the bottom image recognition equipment 130, LPR device 140, RFID reader 150, and top camera 160 are illustrated as performing separate functions, components that perform the same function in each device are integrated. can be used For example, the camera of the LPR device 140 and the upper camera 160 are not configured separately and can be used as one. In addition, the first vehicle detection unit of the bottom image recognition equipment 130 and the LPR vehicle detection unit of the LPR device 140 are not configured separately, but can be used as one to detect the vehicle 120 entering in a random manner.

The system 100 in FIG. 1 is merely an example, and those skilled in the art may appropriately modify or modify the system 100 in FIG. 1 to suit the application. For example, in FIG. 1, the blocking device 145 of the system 100 may have any shape, and in some cases, the blocking device 145 may not be included in the system 100. Additionally, LPR device 140, RFID reader 150, and/or top camera 160 may be used selectively or in combination in system 100.

2 is a block diagram illustrating an example integrated management server 110 for vehicle type recognition, according to at least some embodiments of the present disclosure. In FIG. 2 , with reference to the system 100 of FIG. 1 , specific operations of the integrated management server 110 are described.

The integrated management server 110 may include a vehicle type determination unit 210, a forgery detection unit 220, a communication unit 230, an image database unit 240, and a specifications database unit 250. The vehicle type determination unit 210 may determine the vehicle type of the entering vehicle 120 based on the bottom image of the entering vehicle 120 captured from the lower camera installed in the bottom image recognition equipment 130. The vehicle type determination unit 210 may include an artificial intelligence model, for example, a deep learning model, for recognizing the vehicle type. The forgery detection unit 220 can determine whether the entry vehicle 120 has been forged or altered. The communication unit 230 transmits and receives data to each device of the system 100, that is, the bottom image recognition equipment 130, the LPR device 140, the blocking device 145, the RFID reader 150, and the upper camera 160. can do. The image database unit 240 may store bottom images of a plurality of vehicles, each corresponding to the vehicle type. The specifications database unit 250 may store the registered vehicle model corresponding to each vehicle number of a plurality of vehicles or the vehicle's unique ID. In Figure 2, one integrated management server is illustrated as including a vehicle type determination unit 210, a forgery detection unit 220, a communication unit 230, an image database unit 240, and a specifications database unit 250, but each The configuration can be implemented on a separate, independent server.

Specifically, the vehicle type determination unit 210 may segment an object of interest from the bottom image of the entering vehicle 120, which is received from the bottom image recognition equipment 130 through the communication unit 230. The object of interest may be a pipe in a vehicle. In this regard, FIG. 3 illustrates an example bottom view of an entering vehicle, according to at least some embodiments of the present disclosure. As can be seen in examples (a)-(d) of FIG. 3, the bottom image of the vehicle, especially the pipe portion of the vehicle, may have distinct characteristics depending on the vehicle type. Additionally, since it is difficult to arbitrarily modify the underside of a vehicle after it is shipped, it can be effective to use the underside image of the vehicle to identify the vehicle.

The vehicle type determination unit 210 is configured to determine an object of interest in the bottom image of the entering vehicle 120 generated by segmentation, for example, in the bottom images of a plurality of vehicles stored in the image database unit 240 corresponding to the predicted bounding box of the pipe of the vehicle. Each IoU (Intersection over Union) can be measured from the reference bounding box of the object of interest. IoU is typically an indicator that evaluates model performance in the field of object detection technology using deep learning models. It is an indicator of the area of the bounding box that represents the location of the object predicted by the model and the ground truth that represents the location of the actual object. ) can be measured by comparing the areas of the two areas, dividing the overlapping area by the combined area of the two areas. This can be expressed as an equation:

In this disclosure, the IoU is not used to evaluate the object detection performance of the deep learning model, but the reference bounding box of the object of interest is set in advance within the bottom images of a plurality of vehicles stored in the image database unit 240, and the entering vehicle (120) ) is characterized in that it is used to recognize the vehicle type by comparing it with the predicted bounding box of the object of interest in the bottom image. When such a deep learning model is implemented, vehicle type recognition performance can be improved compared to simply comparing two images.

The vehicle type determination unit 210 may determine the first candidate vehicle type based on the measured IoU. In one example, the vehicle type determination unit 210 may determine the largest among the measured IoUs as the first candidate vehicle type. In another example, the vehicle type determination unit 210 may determine the measured IoU that is greater than or equal to a predetermined threshold as the first candidate vehicle type.

Before segmenting the object of interest, the vehicle type determination unit 210 may recognize a plurality of tires from the bottom image of the entering vehicle 120 and detect the midpoint of each recognized tire. Thereafter, the vehicle type determination unit 210 may standardize the bottom image of the entering vehicle 120 captured based on the plurality of detected midpoints. If the bottom image is standardized, the midpoint between the bottom image of the entering vehicle 120 and the bottom image of the plurality of vehicles stored in the image database unit 240 can be matched, thereby improving the accuracy of IoU measurement.

The vehicle type determination unit 210 may determine the second candidate vehicle type based on the exterior image of the entering vehicle 120 captured by the upper camera 160 and received through the communication unit 230. The vehicle type determination unit may pay attention to visible parts of the vehicle 120, such as color, size, shape of each component, emblem, etc., and determine the second candidate vehicle type through image analysis. The vehicle type determination unit 210 can improve the reliability of vehicle type recognition by checking whether the first candidate vehicle type and the second candidate vehicle type match.

In one embodiment, the forgery detection unit 220 may receive the vehicle number of the entering vehicle 120 from the LPR device 140 through the communication unit 230. The forgery detection unit 220 may check the registered vehicle model corresponding to the recognized vehicle number of the entering vehicle 120 in the specifications database unit 250 and check whether the registered vehicle model matches the first candidate vehicle model. In addition, if it is determined that the first candidate vehicle model and the registered vehicle model do not match, the forgery detection unit 220 transmits a warning notification to the management server or enforcement device that manages forgery and forgery of the vehicle, for example, to detect illegal or vehicle-related issues. Vehicles that have forged or altered license plates for the purpose of avoiding tolls can be cracked down on.

In another embodiment, the forgery detection unit 220 may receive the unique ID of the entering vehicle 120 from the RFID reader 150 through the communication unit 230. The forgery detection unit 220 may check the registered vehicle model corresponding to the unique ID of the entering vehicle 120 in the specifications database unit 250 and check whether the registered vehicle model matches the first candidate vehicle model. In addition, if it is determined that the first candidate vehicle model and the registered vehicle model do not match, the forgery detection unit 220 transmits a warning notification to the management server or enforcement device that manages forgery and forgery of the vehicle, for example, by attaching an illegal RFID tag. One vehicle can be intercepted.

In FIG. 2, the components of the integrated management server 110 and the operations of each component (e.g., the operation of the vehicle type determination unit 210 or the forgery detection unit 220) are merely exemplary, and those skilled in the art can use them based on the present disclosure. It may be used by modifying or modifying it appropriately.

4 illustrates an integrated management server measuring IoU, according to at least some embodiments of the present disclosure.

Figure 4(a) illustrates that the integrated management server 110 recognizes a plurality of tires from the bottom image of an entering vehicle and detects the midpoint of each recognized tire. For example, in the case of the four-wheeled vehicle illustrated in FIG. 4(a), the midpoints of the four tires can be detected. The integrated management server 110 may resize and rotate the captured bottom image of the entering vehicle based on a plurality of detected midpoints, for example, four midpoints, and adjust the bottom image to meet predetermined standards. The integrated management server 110 may generate a predicted bounding box of an object of interest (eg, a pipe of a vehicle) by segmenting a standardized bottom image.

FIG. 4(b) illustrates a bottom image of one of a plurality of vehicles stored in the image database unit of the integrated management server 110. As illustrated in FIG. 4(b), the bottom images of all vehicles stored in the image database unit are already standardized based on tires. For example, in the case of a four-wheeled vehicle shown in FIG. 4(b), the midpoints of the four tires may be displayed. Additionally, for all vehicles stored in the image database unit, the reference bounding box of the object of interest (eg, pipe of the vehicle) is already designated.

Figure 4(c) shows that the integrated management server 110 uses the bottom image of the entering vehicle in Figure 4(a) and the bottom image of one of the plurality of vehicles stored in the image database unit in Figure 4(b) to measure IoU. This is an example of overlapping. The bottom image of the entering vehicle in FIG. 4(a) and the bottom image of the vehicle stored in the image database unit in FIG. 4(b) are both in a standardized form, and when overlapping, the midpoints between the bottom images can match. The integrated management server 110 can measure the IoU by dividing the area where the bottom image of the entering vehicle and the bottom image of the image database unit match into an area that combines the bottom image of the entering vehicle and the bottom image of the image database unit. The integrated management server 110 may measure each IoU by repeating the same process for all bottom images stored in the image database unit.

The integrated management server 110 may determine the first candidate vehicle model based on a plurality of measured IoUs. In one example, the integrated management server 110 may determine the largest among the measured IoUs as the first candidate vehicle model. In another example, the integrated management server 110 may determine the first candidate vehicle model among the measured IoUs that is greater than or equal to a predetermined threshold. In the example of FIG. 4, the bottom image of the entering vehicle in FIG. 4(a) and the bottom image of the vehicle stored in the image database unit of FIG. 4(b) are different, so the IoU between the two images measured by the integrated management server 110 is It is low, and accordingly, the model of the vehicle in FIG. 4(b) may not be selected as the first candidate model for the entering vehicle in FIG. 4(a).

4(a) and 4(b) illustrate the bottom image of a four-wheeled vehicle, but the entering vehicle and the vehicle stored in the image database unit may have any shape, such as a two-wheeled vehicle, a three-wheeled vehicle, a five-wheeled vehicle, or a six-wheeled vehicle. Please note that this may be possible. As an example, the integrated management server 110 measures the IoU only when the number of tires of the entering vehicle and the vehicle stored in the image database unit match, that is, if the number of tires of both bottom images do not match, additional image analysis is performed. By excluding, the efficiency of vehicle type recognition can be increased.

FIG. 5 is a flow diagram illustrating an example process 500 performed on a computing device for computer vision-based vehicle type recognition, in accordance with at least some embodiments of the present disclosure. For example, process 500 may be performed under the control of a computing device, such as integrated management server 110 of FIGS. 1 and 2. Process 500 may be performed to recognize the vehicle type of an entering vehicle at any location where vehicle recognition is required, for example, for security screening or fare payment.

Process 500 shown in FIG. 5 may include one or more operations, functions or actions as illustrated by blocks 501 , 502 , 503 and/or 504 . Meanwhile, the schematic operations illustrated in FIG. 5 are provided as examples only, and without departing from the essence of the disclosed embodiment, some of the operations may be optional, may be combined into fewer operations, or may be expanded into additional operations. You can.

Process 500 may begin at block 501, which receives a bottom image of an entering vehicle captured by a bottom camera. At block 501, the computing device may receive a bottom image of the entering vehicle captured by a bottom camera. Here, the lower camera may include a computer vision camera, such as a line scan camera.

Process 500 may continue from block 501 to block 502, which segments objects of interest from the bottom image. At block 502, the computing device may segment an object of interest from an underside image of an entering vehicle. The object of interest may be a pipe in a vehicle. Before segmentation, the computing device can recognize a plurality of tires from the bottom image of the entering vehicle, detect the midpoint of each recognized tire, and standardize the captured bottom image of the entering vehicle based on the plurality of detected midpoints. there is.

Process 500, at block 502, measures each IoU from the reference bounding box of the object of interest in the bottom image of a plurality of vehicles stored in the image database unit corresponding to the predicted bounding box of the object of interest in the bottom image ( 503). In block 503, the computing device performs a reference bounding of the object of interest in the bottom image of a plurality of vehicles stored in the image database unit corresponding to a prediction bounding box of the object of interest in the bottom image of the entering vehicle, the midpoints of which coincide with each other through normalization. Each IoU can be measured from the box.

Process 500 may continue from block 503 to block 504 where, based on the measured IoU, a first candidate vehicle type is determined. At block 504, the computing device may determine a first candidate vehicle model based on the measured IoU. The computing device may determine the largest among the measured IoUs as the first candidate vehicle model. Alternatively, the computing device may determine the measured IoU that is greater than or equal to a predetermined threshold as the first candidate vehicle type.

Optionally, after the first candidate vehicle type is determined, the computing device may recognize the vehicle in different ways to determine whether the first candidate vehicle type is well recognized or that the vehicle does not contain any incorrect identification means (e.g., a counterfeit license plate or illegal vehicle type). You can check whether RFID is attached or not. In a first embodiment, the computing device determines a second candidate vehicle model based on an exterior image of the entry vehicle captured from an upper camera that photographs the exterior of the entry vehicle, and determines whether the first candidate vehicle type and the second candidate vehicle model match. You can. In a second embodiment, the computing device may search the specifications database unit for the registered vehicle model corresponding to the vehicle license plate number of the entering vehicle recognized by the LPR device and check whether the registered vehicle model matches the first candidate vehicle model. In a third embodiment, the computing device may search the specifications database for the registered vehicle model corresponding to the unique ID of the entering vehicle recognized by the RFID reader and check whether the registered vehicle model matches the first candidate vehicle model. Additionally, if it is determined that the first candidate vehicle model and the registered vehicle model do not match, the computing device may transmit a warning notification to a management server or control device that manages forgery and alteration of the vehicle.

Among other possible advantages, according to the present disclosure, the efficiency of vehicle type recognition can be increased by using the bottom part of the vehicle, which has better features than the exterior of the vehicle and is difficult to modify after shipment, and standardizing the bottom image. Additionally, the performance of deep learning models for vehicle type recognition can be improved by using IoU when analyzing images.

6 illustrates an example computer program product 600 that can be used to recognize vehicle types based on computer vision, in accordance with at least some embodiments of the present disclosure. An exemplary computer program product embodiment is provided using signal bearing media 602. In some embodiments, signal-bearing medium 602 of one or more computer program products 600 includes instructions 604, computer-readable medium 606, recordable medium 608, and/or communication medium 610. can do.

Instructions 604 included in signal-bearing medium 602 may be executed by a computing device, such as integrated management server 110 of FIGS. 1 and 2. Instructions 604 may include at least one instruction that, when executed, causes the computing device to receive an underside image of an entering vehicle captured by an underside camera; at least one instruction related to segmenting an object of interest from an underside image of an entering vehicle; At least one command related to measuring each IoU from a reference bounding box of the object of interest in the bottom image of a plurality of vehicles stored in the image database unit corresponding to the predicted bounding box of the object of interest in the bottom image of the entering vehicle; and at least one command related to determining the first candidate vehicle model based on the measured IoU.

The description of the present application described above is for illustrative purposes, and those skilled in the art will understand that the present application can be easily modified into other specific forms without changing its technical idea or essential features. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. For example, each component described as single may be implemented in a distributed manner, and similarly, components described as distributed may also be implemented in a combined form.

Above, we have looked in detail at the object sought to be claimed in this disclosure. The subject matter claimed in this disclosure is not limited in scope to the specific implementation examples described above. For example, in some implementations it may be in the form of hardware operably used on a device or combination of devices, in other implementations it may be implemented in the form of software and/or firmware, and in still other implementations it may be implemented in the form of a signal bearing medium, It may contain one or more items such as storage media. Here, the storage medium, such as a CD-ROM, computer disk, flash memory, etc., may contain instructions that, when executed by a computing device, such as a computing system, computing platform, or other system, may cause execution of the corresponding processor according to the implementation described above. can be saved. Such a computing device may include one or more processing units or processors, one or more input/output devices such as a display, keyboard, and/or mouse, and one or more memory such as static random access memory, dynamic random access memory, flash memory, and/or a hard drive. may include.

Meanwhile, whether to implement a system in hardware or software is generally a design choice issue that represents a trade-off between cost and efficiency. There are a variety of means (e.g., hardware, software and/or firmware) that may be utilized in the present disclosure, with the preferred means being the process and/or system and/or other technology used. It will change depending on the context. For example, if the implementer determines that speed and accuracy are most important, the implementer may choose primarily hardware and/or firmware means; if flexibility is most important, the implementer may choose primarily software implementation; Or, alternatively, the implementer may choose any combination of hardware, software and/or firmware.

The foregoing detailed description illustrates various embodiments of devices and/or processes through block diagrams, flow diagrams, and/or other examples. Such block diagrams, flowcharts, and/or other examples will include one or more functions and/or operations, and those of ordinary skill in the art will understand that each function and/or operation within the block diagrams, flowcharts, and/or other examples includes hardware, software, firmware, Alternatively, it will be understood that any combination thereof may be implemented individually or collectively. In one embodiment, some portions of the subject matter described in this disclosure may be implemented through an Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), Digital Signal Processor (DSP), or other form of integration. Alternatively, some aspects of embodiments of the present disclosure may include one or more computer programs running on one or more computers (e.g., one or more programs running on one or more computer systems), one or more programs running on one or more processors (e.g., one or more programs running on one or more computer systems), (e.g., one or more programs running on one or more microprocessors), firmware, or substantially any combination thereof, may also be implemented in whole or in part on an integrated circuit, including writing code for software and/or firmware. and/or design of circuits are within the scope of skill of those skilled in the art in light of this disclosure. Additionally, those skilled in the art will understand that the mechanisms of the subject matter of this disclosure may be distributed in a variety of program products, and the examples of the subject matter of this disclosure apply regardless of the specific type of signal bearing medium used to actually perform the distribution. You will understand.

Although certain example techniques have been described and shown herein using various methods and systems, those skilled in the art will recognize that various other modifications or equivalents may be substituted without departing from the claimed subject matter. Additionally, many modifications may be made to adapt the teachings of the claimed subject matter to particular situations without departing from the central concept described herein. Accordingly, it is intended that the claimed subject matter not be limited to the specific examples disclosed, but that such claimed subject matter may also include all embodiments that fall within the scope of the appended claims and their equivalents.

Throughout the present disclosure, when a part is said to be “connected” to another part, this includes not only the case where it is “directly connected,” but also the case where it is “electrically connected” with another element in between. do. Additionally, throughout the present disclosure, when a member is said to be located “on” another member, this includes not only the case where the member is in contact with the other member, but also the case where another member exists between the two members. Furthermore, throughout the present disclosure, when a part “includes” a certain element, this means that it may further include other elements rather than excluding other elements, unless specifically stated to the contrary. As used in this disclosure, the terms “about,” “substantially,” and the like are used to mean at or close to a numerical value when manufacturing and material tolerances inherent in the stated meaning are presented, and to aid in the understanding of the present application. It is used to prevent unscrupulous infringers from unfairly exploiting disclosures in which precise or absolute figures are mentioned.

The scope of the present disclosure is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present application. do.

Claims (12)

A system for computer vision-based vehicle type recognition,
a lower camera that captures the bottom of the entering vehicle; and
An integrated management server including a vehicle type determination unit that determines the vehicle type of the entering vehicle based on the bottom image of the entering vehicle captured from the lower camera and an image database unit that stores bottom images of a plurality of vehicles;
Including,
The vehicle type determination unit,
Recognize a plurality of tires from the bottom image of the entering vehicle,
detecting the midpoint of each of the recognized tires;
Standardizing the captured bottom image of the entering vehicle based on the plurality of detected midpoints,
Segmenting an object of interest from the bottom image of the entering vehicle,
Each IoU from the reference bounding box of the object of interest in the bottom image of the plurality of vehicles stored in the image database unit corresponding to the predicted bounding box of the object of interest in the bottom image of the entering vehicle Measure (Intersection over Union),
Based on the measured IoU, a first candidate vehicle model is determined,
Determining the first candidate vehicle model includes determining the largest value among the measured IoUs as the first candidate vehicle model,
The system of claim 1, wherein the object of interest is a pipe of a vehicle. delete delete delete According to paragraph 1,
The system further includes an upper camera that photographs the exterior of the entering vehicle,
The system wherein the vehicle type determination unit determines a second candidate vehicle type based on an exterior image of the entering vehicle captured from the upper camera, and determines whether the first candidate vehicle type and the second candidate vehicle type match. According to paragraph 1,
The system further includes an LPR device that recognizes the vehicle number from the license plate of the entering vehicle,
The integrated management server further includes a forgery detection unit that determines whether the entering vehicle has been forged or altered, and a specifications database unit that stores the registered vehicle model corresponding to each vehicle number,
The forgery detection unit searches the specifications database unit for a registered vehicle model corresponding to the recognized license plate number of the entering vehicle, and checks whether the registered vehicle model matches the first candidate vehicle model. According to paragraph 1,
The system further includes an RFID reader that recognizes the unique ID of the vehicle from the RFID tag included in the entering vehicle,
The integrated management server further includes a forgery detection unit that determines whether the entering vehicle has been forged or altered, and a specifications database unit that stores the registered vehicle model corresponding to the unique ID of each vehicle,
The system wherein the forgery detection unit searches the specifications database unit for a registered vehicle model corresponding to the unique ID of the entering vehicle, and checks whether the registered vehicle model matches the first candidate vehicle model. According to clause 6 or 7,
When the forgery detection unit determines that the first candidate vehicle model and the registered vehicle model do not match, the system transmits a warning notification to a management server that manages forgery and alteration of the vehicle. As a method for computer vision-based vehicle model recognition,
Receiving a bottom image of an entering vehicle captured by a lower camera;
Recognizing a plurality of tires from a bottom image of the entering vehicle;
detecting the midpoint of each of the recognized tires;
standardizing the captured bottom image of the entering vehicle based on the plurality of detected midpoints;
Segmenting an object of interest from a bottom image of the entering vehicle;
Measuring each IoU from a reference bounding box of an object of interest in a bottom image of a plurality of vehicles stored in an image database unit corresponding to a predicted bounding box of the object of interest in the bottom image of the entering vehicle; and
Based on the measured IoU, determining a first candidate vehicle model
Including,
The step of determining the first candidate vehicle model includes determining the largest value among the measured IoUs as the first candidate vehicle model,
The method of claim 1, wherein the object of interest is a pipe of a vehicle. delete delete A computer program stored on a recording medium of an integrated management server for computer vision-based vehicle model recognition and executed by the integrated management server,
At least one command related to receiving a bottom image of an entering vehicle captured by a bottom camera;
Recognize a plurality of tires from the bottom image of the entering vehicle and detect the midpoint of each of the recognized tires; and at least one command related to standardizing the captured bottom image of the entering vehicle based on the plurality of detected midpoints;
at least one command related to segmenting an object of interest from a bottom image of the entering vehicle;
At least one command related to measuring each IoU from a reference bounding box of an object of interest in a bottom image of a plurality of vehicles stored in an image database unit corresponding to a predicted bounding box of the object of interest in the bottom image of the entering vehicle; and
At least one instruction related to determining a first candidate vehicle model based on the measured IoU
includes
Determining the first candidate vehicle model includes determining the largest value among the measured IoUs as the first candidate vehicle model;
A computer program stored in a recording medium of an integrated management server, wherein the object of interest is a pipe of a vehicle.