CN116935370A - Cloud license plate recognition method and device, electronic equipment and storage medium - Google Patents
Cloud license plate recognition method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116935370A CN116935370A CN202310819763.8A CN202310819763A CN116935370A CN 116935370 A CN116935370 A CN 116935370A CN 202310819763 A CN202310819763 A CN 202310819763A CN 116935370 A CN116935370 A CN 116935370A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- detection result
- license plate
- information
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000003860 storage Methods 0.000 title claims abstract description 21
- 238000001514 detection method Methods 0.000 claims abstract description 272
- 238000012545 processing Methods 0.000 claims description 11
- 238000011897 real-time detection Methods 0.000 claims description 8
- 238000012544 monitoring process Methods 0.000 claims description 7
- 238000012800 visualization Methods 0.000 abstract description 2
- 238000004590 computer program Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 230000000875 corresponding effect Effects 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Traffic Control Systems (AREA)
Abstract
The application discloses a cloud license plate recognition method and device, electronic equipment and storage media, wherein the recognition method comprises the following steps: acquiring a first detection result uploaded by a road side; obtaining a second detection result sent by the vehicle side; and identifying license plate information of the target vehicle according to the first detection result and the second detection result, wherein the second detection result and the first detection result both comprise position information of the target under the same coordinate system. The license plate recognition method and the license plate recognition device can realize license plate recognition at the cloud and information visualization in digital twin data. The method and the device can be used for the vehicle-road cooperative scene.
Description
Technical Field
The application relates to the technical field of smart cities and digital twinning, in particular to a cloud license plate recognition method and device, electronic equipment and storage media.
Background
The digital display of the smart city is generally to detect the position, speed, heading and type of the vehicle in the road according to the road side sensing equipment, and process the information at the cloud end to display the information on the local terminal, namely, the information is used as digital twin data in a digital twin system. None of the vehicle information in the digital twinning data is uniquely identifiable by being independently distinguishable over a long period of time (e.g., days and months) because the vehicles are each given a unique and random ID during the detection and tracking process.
In the related art, a license plate of a vehicle is detected through a road side camera, so that the license plate is endowed with the vehicle at a cloud end as a unique identifier ID. However, if the vehicle is far from the camera, the license plate appears very blurred and difficult to identify, and only the position and type of the vehicle can be detected. Therefore, the digital twin data still has the problem that the license plate of the vehicle is difficult to recognize or cannot be recognized, so that the unique identification ID cannot be endowed to the digital twin system for visual display.
Disclosure of Invention
The embodiment of the application provides a cloud license plate recognition method and device, electronic equipment and storage medium, and aims to realize information visualization in digital twin data through a cloud.
The embodiment of the application adopts the following technical scheme:
in a first aspect, an embodiment of the present application provides a cloud license plate identification method, where the identification method includes:
acquiring a first detection result uploaded by a road side;
obtaining a second detection result sent by the vehicle side; and
and identifying license plate information of the target vehicle according to the first detection result and the second detection result, wherein the second detection result and the first detection result both comprise position information of the target under the same coordinate system.
In some embodiments, the target includes at least a vehicle, the identifying license plate information of the target vehicle according to the first detection result and the second detection result, where each of the second detection result and the first detection result includes location information of the target in the same coordinate system, includes:
judging whether the first characteristic information of the vehicle in the first detection result contains original license plate information or not;
if the first characteristic information of the vehicle in the first detection result contains the original license plate information, directly transmitting the original license plate information to the local terminal so that digital twin data of the local terminal display the first characteristic information of the vehicle;
if the first characteristic information of the vehicle in the first detection result does not contain the original license plate information, continuously identifying whether the original license plate information exists in the second detection result;
if the original license plate information exists in the second detection result, judging whether the second characteristic information of the vehicle and the first characteristic information of the vehicle belong to the same vehicle or not by comparing the second characteristic information of the vehicle in the second detection result with the first characteristic information of the vehicle in the first detection result according to the original license plate information;
If the vehicle is the same vehicle, the second characteristic information of the vehicle in the second detection result is endowed to the vehicle in the first detection result, so that the second characteristic information of the vehicle is displayed in digital twin data of a local terminal, and the local terminal is used for receiving the processing result of the cloud. In some embodiments, the coordinate system includes a world coordinate system, and the obtaining the first detection result uploaded by the road side includes:
acquiring a first vehicle detection result of the road side on the road surface in the current monitoring area range through first sensing equipment, wherein the first sensing equipment obtains the position of the vehicle under the world coordinate system according to a preset calibration relation between the road side sensing equipment and the road surface;
and receiving the position, the vehicle type, the detection time stamp and the vehicle ID of the vehicle in the world coordinate system in the first vehicle detection result.
In some embodiments, the obtaining the second detection result sent by the vehicle side includes:
acquiring a second vehicle detection result of the vehicle side in the surrounding environment through second sensing equipment, wherein the second sensing equipment obtains the position of the vehicle under the world coordinate system according to the self-positioning of the vehicle side and the calibration parameters of the second sensing equipment;
Receiving a picture of the position of the vehicle in the image, which is contained in the second vehicle detection result, and the position of the vehicle under a world coordinate system;
and/or the number of the groups of groups,
and receiving a license plate recognition result of the vehicle in the second vehicle detection result and the position of the vehicle under a world coordinate system.
In some embodiments, after identifying license plate information of the target vehicle according to the first detection result and the second detection result, the method further includes: and taking license plate information of the target vehicle as a unique identification code of the digital twin data, and visually displaying the digital twin data on different terminals.
In some embodiments, the method further comprises:
invoking a history detection result uploaded by the road side;
judging whether the license plate information of the target vehicle is successfully matched or not according to the running track in the history detection result uploaded by the road side;
if the matching is successful, license plate information of the target vehicle in the first detection result uploaded by the road side is directly used;
if the matching is unsuccessful, the first detection result uploaded by the road side and/or the second detection result sent by the vehicle side are synchronously updated to the historical detection result uploaded by the road side.
In some embodiments, the method further comprises:
acquiring a real-time detection result sent by a vehicle side;
judging whether the license plate information of the target vehicle is successfully matched with the target vehicle in the first detection result uploaded by the road side according to the license plate information of the target vehicle in the real-time detection result sent by the vehicle side;
and if the matching is successful, assigning the target vehicle of the first detection result according to the license plate information of the target vehicle.
In a second aspect, an embodiment of the present application further provides a cloud license plate identification device, where the device includes:
the first acquisition module is used for acquiring a first detection result uploaded by the road side;
the second acquisition module is used for acquiring a second detection result sent by the vehicle side; and
the identification module is used for identifying license plate information of the target vehicle according to the first detection result and the second detection result, and the second detection result and the first detection result both comprise position information of the target under the same coordinate system.
In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to perform the above method.
In a fourth aspect, embodiments of the present application also provide a computer-readable storage medium storing one or more programs, which when executed by an electronic device comprising a plurality of application programs, cause the electronic device to perform the above-described method.
The above at least one technical scheme adopted by the embodiment of the application can achieve the following beneficial effects: and acquiring a first detection result uploaded by the road side and a second detection result transmitted by the vehicle side at the cloud end, and identifying license plate information of the target vehicle according to the first detection result and the second detection result. Because the second detection result and the first detection result both comprise the position information of the target under the same coordinate system, the unique identification can be determined through the position information of the target under the same coordinate system, and the unique identification recognized by the cloud is assigned to digital twin data in the digital twin system, such as a vehicle.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a schematic flow chart of a cloud end license plate recognition method in an embodiment of the application;
fig. 2 is a schematic structural diagram of a cloud-end license plate recognition device according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The following describes in detail the technical solutions provided by the embodiments of the present application with reference to the accompanying drawings.
The embodiment of the application provides a cloud license plate recognition method, as shown in fig. 1, and provides a flow diagram of the cloud license plate recognition method in the embodiment of the application, wherein the method at least comprises the following steps of S110 to S130:
step S110, a first detection result uploaded by the road side is obtained.
The roadside terminal includes a plurality of roadside devices, each having a roadside sensing unit therein, including but not limited to, image sensors, radars, and the like. The image sensor mainly comprises a camera, and the camera can be divided into a far view camera, a middle view camera and a near view camera. The radar mainly comprises a laser radar, a millimeter wave radar and the like.
And uploading the road side sensing result by the road side sensing unit as a first detection result. It should be noted that at least the detection result of the vehicle on the road surface is included in the first detection result. The detection result also includes a vehicle position, a vehicle type, a time stamp, a vehicle ID, and the like. The vehicle ID at this time is not a vehicle license plate, but a vehicle ID randomly assigned in the road side perception result.
The road side end detects the road surface vehicle by using equipment in the road side sensing unit, and obtains the position information A of the current vehicle under the world coordinate system according to the joint calibration relation in advance. The position information a here is position information in the world coordinate system, i.e., the absolute position of the vehicle in the real world can be known.
In addition, only the roadside image video stream data may be included in the first detection result. The cloud acquires the road side image video stream data and then can carry out post-processing.
Step S120, obtaining a second detection result sent by the vehicle side.
The vehicle side end can be an intelligent vehicle or an automatic driving vehicle. The vehicle detection system has the functions of detecting and positioning surrounding vehicles and self-vehicles, whether the vehicle is an intelligent vehicle or an automatic driving vehicle.
Because the vehicle-mounted camera at the vehicle side end is clear in image information acquired during detection of surrounding vehicles, the intelligent vehicle and/or the automatic driving vehicle also always move, and therefore detection results of the surrounding vehicles are more accurate.
The second detection result at least comprises detection results of surrounding vehicles, and the detected positions of the vehicles under the world coordinate system can be obtained according to the self-positioning information of the vehicles detected by the self-vehicle and the calibration parameters of the vehicle-mounted camera. The detection result includes position information B of the vehicle in the world coordinate system. The position information B here is also position information in the world coordinate system, i.e. the absolute position of the vehicle in the real world can be known.
The second detection result may further include a location of a surrounding vehicle, a type of the surrounding vehicle, and a time stamp at the time of detection.
It should be noted that, although the above-mentioned "position information B" and "position information a" are both position information of the target in the same coordinate system, the same target cannot be determined because no association match is established between the two. And, it is also impossible to determine whether or not the vehicle is the target vehicle.
Alternatively, license plate information of the vehicle may also be detected in the second detection result. Of course, the license plate information of the vehicle can also be detected in the first detection result, but the association matching of the license plate information and the vehicle cannot be established.
Step S130, identifying license plate information of the target vehicle according to the first detection result and the second detection result, where both the second detection result and the first detection result include position information of the target in the same coordinate system.
For a digital twin system, digital twin vehicle data is obtained after all vehicles traveling on a road are processed. Therefore, after determining the vehicle to be processed in the road, license plate information of the target vehicle can be determined according to the first detection result and the second detection result obtained in the above steps.
Here, the "target vehicle" refers to a vehicle that is present in both detection results and that needs to detect and recognize license plate information. That is, if the vehicle appears in only one detection result, it cannot be the target vehicle. Only when the detection result of the vehicle side and the detection result of the road side match, the vehicle can be regarded as the target vehicle.
The "object" herein refers to an object in image detection, and the type thereof may include a vehicle, an obstacle, a road sign, and the like, and is not particularly limited herein. But the "object (vehicle, obstacle)" is under the same coordinate system in the detection result and has the same positional information.
It will be appreciated that the use of a neural network based image detection model enables the detection of "targets".
When the cloud terminal processes the second detection result and the first detection result, the target vehicle can be accurately matched according to the position information of the target under the same coordinate system and the time stamp information received by the cloud terminal.
It should be noted that, first, it is necessary to determine whether the first detection result includes original license plate information, if the license plate information of the vehicle can be accurately obtained in the first detection result, the license plate information of the vehicle in the first detection result is directly used as the license plate of the target vehicle after the same vehicle is determined by matching.
And then if the first characteristic information of the vehicle in the first detection result does not contain the original license plate information, continuously identifying whether the original license plate information exists in the second detection result, specifically comparing the position information and the time stamp information of the identified vehicle detected in the second detection result with the position and the time stamp information of the target vehicle in the first detection result, and if the position information and the time stamp information are matched, judging that the vehicle is the same vehicle. And the license plate of the vehicle obtained in the second detection result can be assigned to the corresponding target vehicle in the first detection result.
Finally, the cloud can integrate license plate number results at the vehicle side end and the road side end, and visual display is carried out at each terminal, so that monitoring is convenient. By adopting the cloud license plate recognition method, the vehicle side end is used for detecting the position information and the image information of surrounding vehicles, uploading the result to the cloud, analyzing the uploading result of the vehicle side end at the cloud to obtain license plate information, and combining the license plate information with the road side end detection result to endow the license plate information of the vehicles in the road side end detection result.
According to the cloud license plate recognition method, the cloud receives and acquires the first detection result uploaded by the road side and the second detection result sent by the vehicle side, and the target vehicle is determined according to the second detection result and the position information of the target included in the first detection result under the same coordinate system. That is, the two detection results are associated and matched. And then further identifying license plate information of the target vehicle. License plate information is generally provided by the second detection result and assigned to the first detection result. The license plate information of the target vehicle can be visualized at multiple ends, so that the monitoring by the relevant manager is facilitated.
The cloud license plate recognition method is used for displaying a visual license plate information result in a digital twin system and is used as a unique identification ID of digital twin vehicle data. According to the visualized unique identification ID, the running track of the digital twin vehicle data can be continuously tracked, the historical track and the real-time running track of the digital twin vehicle data are obtained, and related information recommendation can be carried out on vehicles in the real world based on the unique identification ID, or road condition information is estimated and issued to the vehicles in the real world through the unique identification ID.
In the related art, the method is different from the related art, the vehicle information on the road is detected only through the road side end, and when the vehicle information is identified by the cloud, the license plate information is accurately identified or is easy to be blocked. By the method, the problems that license plate information is blocked and license plate information cannot be accurately identified can be solved, and the target vehicle can be determined because the second detection result and the first detection result both comprise the position information of the target under the same coordinate system. And then, the license plate information of the target vehicle is assigned, and the first detection result or the second detection result can be used, so that the accuracy of the digital twin vehicle data generation is improved.
Unlike the related art, the digital twin vehicle data visualization method and the system thereof have the problem that the visualized information of the digital twin vehicle data in the digital twin system is inaccurate. By the method, the generation efficiency of the visual information can be improved. And identifying license plate information of the target vehicle according to the first detection result and the second detection result, and taking the license plate information as a unique identity mark, wherein the unique identity mark is verified and can be continuously used as digital twin vehicle data in the cloud. For example, when a new target enters and is the same as the recorded license plate information after detection, the target vehicle with the same license plate is considered to reappear, and the unique identity of the license plate information can be continuously used.
In one embodiment of the present application, the target includes at least a vehicle, the license plate information of the target vehicle is identified according to the first detection result and the second detection result, and the second detection result and the first detection result each include position information of the target under the same coordinate system, including: judging whether the first characteristic information of the vehicle in the first detection result contains original license plate information or not; if the first characteristic information of the vehicle in the first detection result contains the original license plate information, directly transmitting the original license plate information to the local terminal so that digital twin data of the local terminal display the first characteristic information of the vehicle; if the first characteristic information of the vehicle in the first detection result does not contain the original license plate information, continuously identifying whether the original license plate information exists in the second detection result; if the original license plate information exists in the second detection result, judging whether the second characteristic information of the vehicle and the first characteristic information of the vehicle belong to the same vehicle or not by comparing the second characteristic information of the vehicle in the second detection result with the first characteristic information of the vehicle in the first detection result according to the original license plate information; if the vehicle is the same vehicle, the second characteristic information of the vehicle in the second detection result is endowed to the vehicle in the first detection result, so that the second characteristic information of the vehicle is displayed in digital twin data of a local terminal, and the local terminal is used for receiving the processing result of the cloud.
Case one: it can be understood that if the road side end itself can already provide the license plate number of the vehicle, the license plate detection result of the vehicle side is not needed.
The "second feature information" and "first feature information" include, but are not limited to, a vehicle ID (may not include a vehicle identification such as license plate information), location information, type information, and detected time stamp information, etc., and are not particularly limited in the embodiment of the present application.
If the first characteristic information of the vehicle in the first detection result contains the original license plate information, the first characteristic information is directly issued to the local terminal, so that the digital twin data of the local terminal display the first characteristic information of the vehicle. The original license plate information is contained in the first characteristic information of the vehicle in the first detection result, the first characteristic information of the vehicle can be issued to a local terminal, and digital twin data of the local terminal are used for displaying the first characteristic information of the vehicle.
And a second case: if the first characteristic information of the vehicle in the first detection result does not contain the original license plate information, continuing to identify whether the original license plate information exists in the second detection result.
The cloud end acquires a road side detection result and acquires a detection result uploaded by the vehicle side, if the vehicle side is uploading the picture, license plate identification is carried out on the picture, comparison is carried out according to the vehicle position and the time stamp detected by the vehicle side and the vehicle position and the time stamp of the road side, and if the vehicle is the same vehicle, the license plate result of the vehicle side is adopted to endow the vehicle of the road side detected by the road side with display.
And a third case: and if the original license plate information exists in the second detection result, carrying out association according to the original license plate information.
When the original license plate information in the second detection result is identified, the image information is mainly processed, and whether the second characteristic information of the vehicle and the first characteristic information of the vehicle belong to the same vehicle is further determined by comparing the second characteristic information of the vehicle in the second detection result with the first characteristic information of the vehicle in the first detection result through the original license plate information obtained from the image information.
If the vehicle is the same vehicle, the second characteristic information of the vehicle in the second detection result is endowed to the vehicle in the first detection result, so that the second characteristic information of the vehicle is displayed in digital twin data of a local terminal, and the local terminal is used for receiving the processing result of the cloud.
After the second characteristic information of the vehicle in the second detection result is assigned to the vehicle in the first detection result, the second characteristic information can be directly visually displayed in the digital twin data of the local terminal. The "local terminal" includes, but is not limited to, a local vehicle end, a local traffic management platform, and a local mobile terminal.
In one embodiment of the present application, the coordinate system includes a world coordinate system, and the obtaining the first detection result uploaded by the road side includes: acquiring a first vehicle detection result of the road side on the road surface in the current monitoring area range through first sensing equipment, wherein the first sensing equipment obtains the position of the vehicle under the world coordinate system according to a preset calibration relation between the road side sensing equipment and the road surface; and receiving the position, the vehicle type, the detection time stamp and the vehicle ID of the vehicle in the world coordinate system in the first vehicle detection result.
The road side end detects vehicles on the road surface by using road side sensing equipment, obtains the positions of the vehicles under the world coordinate system according to a preset sensor calibration relation, and uploads the detected positions, types, time stamps and vehicle IDs to the cloud. The vehicle ID is a unique value given to the same vehicle by continuous detection and tracking of a plurality of frames, and is not license plate information.
The cloud end further matches the detection result of the vehicle side according to the position of the vehicle in the world coordinate system, the type of the vehicle, the detection timestamp and the vehicle ID in the received first vehicle detection result.
In one embodiment of the present application, the obtaining the second detection result sent by the vehicle side includes: acquiring a second vehicle detection result of the vehicle side in the surrounding environment through second sensing equipment, wherein the second sensing equipment obtains the position of the vehicle under the world coordinate system according to the self-positioning of the vehicle side and the calibration parameters of the second sensing equipment; receiving a picture of the position of the vehicle in the image, which is contained in the second vehicle detection result, and the position of the vehicle under a world coordinate system; and/or receiving a license plate recognition result of the vehicle in the second vehicle detection result and the position of the vehicle under a world coordinate system.
The vehicle side end includes, but is not limited to, an intelligent vehicle or an autonomous vehicle, and detects the vehicles around itself by using its own looking-around camera/front-rear-view camera. The position of the vehicle under the world coordinate system can be obtained according to the self-positioning and camera calibration parameters of the vehicle detected on the road surface, the position of the vehicle on the image is cut off to obtain the pixel coordinates of the vehicle in the image, and meanwhile, the cut-off picture and the position of the vehicle under the world coordinate system are sent to the cloud.
The detection function of the vehicle side end is that the intelligent vehicle or the automatic driving vehicle can take the vehicle, and only the additional picture cutting and uploading are needed, and license plate information recognition is carried out at the cloud end, so that the computing resource is not occupied.
It can be understood that if the computing resources of the vehicle side end are sufficient, license plate recognition can be further performed on the image containing the vehicle directly, and finally, the license plate recognition result and the vehicle position are sent to the cloud.
In one embodiment of the present application, after identifying license plate information of the target vehicle according to the first detection result and the second detection result, the method further includes: and taking license plate information of the target vehicle as a unique identification code of the digital twin data, and visually displaying the digital twin data on different terminals.
The license plate information of the target vehicle can be used as the unique identification information of the vehicle by taking the unique identification code of the digital twin data, so that the digital twin data can be visually displayed on different terminals. In addition, according to license plate information of the target vehicle, the cloud information can be issued or correlated with OBU (On board Unit) of the target vehicle at the cloud, time periods of congestion in the process of driving the corresponding target vehicle on a road are recorded, and then traffic control is conducted at the cloud according to the time periods of congestion. The OBU of the target vehicle can communicate with the road side end, and the information issued by the cloud end is issued to the OBU of the target vehicle through the road side end.
In one embodiment of the application, the method further comprises: invoking a history detection result uploaded by the road side; judging whether the license plate information of the target vehicle is successfully matched or not according to the running track in the history detection result uploaded by the road side; if the matching is successful, license plate information of the target vehicle in the first detection result uploaded by the road side is directly used; if the matching is unsuccessful, the first detection result uploaded by the road side and/or the second detection result sent by the vehicle side are synchronously updated to the historical detection result uploaded by the road side.
Based on the expansion of the scheme: and judging whether the license plate information of the target vehicle is successfully matched or not according to the running track in the history detection result uploaded by the road side by calling the history detection result uploaded by the road side.
If the matching is successful, the cloud directly uses license plate information of the target vehicle in the first detection result uploaded by the road side to generate digital twin vehicle data, and if the matching is unsuccessful, the cloud synchronously updates the first detection result uploaded by the road side and/or the second detection result sent by the vehicle side to the historical detection result uploaded by the road side.
In the above scheme, the license plate information of the target vehicle in the first detection result can be continuously updated through asynchronous synchronization, and the license plate information can be actively synchronized without more manual monitoring.
In one embodiment of the application, the method further comprises: acquiring a real-time detection result sent by a vehicle side; judging whether the license plate information of the target vehicle is successfully matched with the target vehicle in the first detection result uploaded by the road side according to the license plate information of the target vehicle in the real-time detection result sent by the vehicle side; and if the matching is successful, assigning the target vehicle of the first detection result according to the license plate information of the target vehicle.
Based on the expansion of the scheme: and judging whether the first detection result uploaded on the road side is successfully matched or not according to license plate information of the target vehicle in the real-time detection result sent by the vehicle side by the cloud terminal through acquiring the real-time detection result sent by the vehicle side.
And if the matching is successful, the cloud end carries out assignment on the target vehicle of the first detection result according to license plate information of the target vehicle.
In the scheme, the license plate information of the target vehicle in the first detection result is updated in a real-time synchronous mode.
The embodiment of the application also provides a cloud license plate recognition device 200, as shown in fig. 2, and provides a schematic structural diagram of the cloud license plate recognition device in the embodiment of the application, where the cloud license plate recognition device 200 at least includes: an acquisition module 210, a first determination module 220, and a second determination module 230, wherein:
in one embodiment of the present application, the obtaining module 210 is specifically configured to: and acquiring a first detection result uploaded by the road side.
The roadside terminal includes a plurality of roadside devices, each having a roadside sensing unit therein, including but not limited to, image sensors, radars, and the like. The image sensor mainly comprises a camera, and the camera can be divided into a far view camera, a middle view camera and a near view camera. The radar mainly comprises a laser radar, a millimeter wave radar and the like.
And uploading the road side sensing result by the road side sensing unit as a first detection result. It should be noted that at least the detection result of the vehicle on the road surface is included in the first detection result. The detection result also includes a vehicle position, a vehicle type, a time stamp, a vehicle ID, and the like. The vehicle ID at this time is not a vehicle license plate, but a vehicle ID randomly assigned in the road side perception result.
The road side end detects the road surface vehicle by using equipment in the road side sensing unit, and obtains the position information A of the current vehicle under the world coordinate system according to the joint calibration relation in advance. The position information a here is position information in the world coordinate system, i.e., the absolute position of the vehicle in the real world can be known.
In addition, only the roadside image video stream data may be included in the first detection result. The cloud acquires the road side image video stream data and then can carry out post-processing.
In one embodiment of the present application, the first determining module 220 is specifically configured to: and obtaining a second detection result sent by the vehicle side.
The vehicle side end can be an intelligent vehicle or an automatic driving vehicle. The vehicle detection system has the functions of detecting and positioning surrounding vehicles and self-vehicles, whether the vehicle is an intelligent vehicle or an automatic driving vehicle.
Because the vehicle-mounted camera at the vehicle side end is clear in image information acquired during detection of surrounding vehicles, the intelligent vehicle and/or the automatic driving vehicle also always move, and therefore detection results of the surrounding vehicles are more accurate.
The second detection result at least comprises detection results of surrounding vehicles, and the position of the current vehicle under the world coordinate system can be obtained according to the self-positioning information of the vehicle detected by the self-vehicle and the calibration parameters of the vehicle-mounted camera. The detection result includes position information B of the vehicle in the world coordinate system. The position information B here is also position information in the world coordinate system, i.e. the absolute position of the vehicle in the real world can be known.
The second detection result may further include a location of a surrounding vehicle, a type of the surrounding vehicle, and a time stamp at the time of detection.
It should be noted that, although the above-mentioned "position information B" and "position information a" are both position information of the target in the same coordinate system, the same target cannot be determined because no association match is established between the two. And, it is also impossible to determine whether or not the vehicle is the target vehicle.
In one embodiment of the present application, the second determining module 230 is specifically configured to: and identifying license plate information of the target vehicle according to the first detection result and the second detection result, wherein the second detection result and the first detection result both comprise position information of the target under the same coordinate system.
For a digital twin system, digital twin vehicle data is obtained after all vehicles traveling on a road are processed. Therefore, after determining the vehicle to be processed in the road, license plate information of the target vehicle can be determined according to the first detection result and the second detection result obtained in the above steps.
Here, the "target vehicle" refers to a vehicle that is present in both detection results and that needs to detect and recognize license plate information. That is, if the vehicle appears in only one detection result, it cannot be the target vehicle. Only when the detection result of the vehicle side and the detection result of the road side match, the vehicle can be regarded as the target vehicle.
The "object" herein refers to an object in image detection, and the type thereof may include a vehicle, an obstacle, a road sign, and the like, and is not particularly limited herein. But the "targets" are under the same coordinate system in the detection result and have the same positional information.
It will be appreciated that the use of a neural network based image detection model enables the detection of "targets".
When the cloud terminal processes the second detection result and the first detection result, the target vehicle can be accurately matched according to the position information of the target under the same coordinate system and the time stamp information received by the cloud terminal.
It should be noted that, first, it is necessary to determine whether the first detection result includes original license plate information, if the license plate information of the vehicle can be accurately obtained in the first detection result, the license plate information of the vehicle in the first detection result is directly used as the license plate of the target vehicle after the same vehicle is determined by matching.
And then if the first characteristic information of the vehicle in the first detection result does not contain the original license plate information, continuously identifying whether the original license plate information exists in the second detection result, specifically comparing the position information and the time stamp information of the identified vehicle detected in the second detection result with the position and the time stamp information of the target vehicle in the first detection result, and if the position information and the time stamp information are matched, judging that the vehicle is the same vehicle. And the license plate of the vehicle obtained in the second detection result can be assigned to the corresponding target vehicle in the first detection result.
Finally, the cloud can integrate license plate number results at the vehicle side end and the road side end, and visual display is carried out at each terminal, so that monitoring is convenient. By adopting the cloud license plate recognition method, the vehicle side end is used for detecting the position information and the image information of surrounding vehicles, uploading the result to the cloud, analyzing the uploading result of the vehicle side end at the cloud to obtain license plate information, and combining the license plate information with the road side end detection result to endow the license plate information of the vehicles in the road side end detection result.
It can be appreciated that the above-mentioned cloud license plate recognition device can implement each step of the cloud license plate recognition method provided in the foregoing embodiment, and the relevant explanation about the cloud license plate recognition method is applicable to the cloud license plate recognition device, and is not repeated here.
Fig. 3 is a schematic structural view of an electronic device according to an embodiment of the present application. Referring to fig. 3, at the hardware level, the electronic device includes a processor, and optionally an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, network interface, and memory may be interconnected by an internal bus, which may be an ISA (Industry Standard Architecture ) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus, or EISA (Extended Industry Standard Architecture ) bus, among others. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 3, but not only one bus or type of bus.
And the memory is used for storing programs. In particular, the program may include program code including computer-operating instructions. The memory may include memory and non-volatile storage and provide instructions and data to the processor.
The processor reads the corresponding computer program from the nonvolatile memory to the memory and then operates the computer program to form the cloud license plate recognition device on the logic level. The processor is used for executing the programs stored in the memory and is specifically used for executing the following operations:
acquiring a first detection result uploaded by a road side;
obtaining a second detection result sent by the vehicle side; and
And identifying license plate information of the target vehicle according to the first detection result and the second detection result, wherein the second detection result and the first detection result both comprise position information of the target under the same coordinate system.
The method executed by the cloud license plate recognition device disclosed in the embodiment of fig. 1 of the present application may be applied to a processor or implemented by the processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.
The electronic device may further execute the method executed by the cloud-end license plate recognition device in fig. 1, and implement the function of the cloud-end license plate recognition device in the embodiment shown in fig. 1, which is not described herein.
The embodiment of the application also provides a computer readable storage medium, which stores one or more programs, the one or more programs including instructions, which when executed by an electronic device including a plurality of application programs, enable the electronic device to execute the method executed by the cloud-end license plate recognition device in the embodiment shown in fig. 1, and specifically is used for executing:
acquiring a first detection result uploaded by a road side;
obtaining a second detection result sent by the vehicle side; and
and identifying license plate information of the target vehicle according to the first detection result and the second detection result, wherein the second detection result and the first detection result both comprise position information of the target under the same coordinate system.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.
Claims (10)
1. A cloud license plate recognition method, wherein the recognition method comprises:
acquiring a first detection result uploaded by a road side;
obtaining a second detection result sent by the vehicle side; and
and identifying license plate information of the target vehicle according to the first detection result and the second detection result, wherein the second detection result and the first detection result both comprise position information of the target under the same coordinate system.
2. The method of claim 1, wherein the target includes at least a vehicle, the license plate information of the target vehicle is identified according to the first detection result and the second detection result, and the second detection result and the first detection result each include position information of the target in a same coordinate system, including:
judging whether the first characteristic information of the vehicle in the first detection result contains original license plate information or not;
if the first characteristic information of the vehicle in the first detection result contains the original license plate information, directly transmitting the original license plate information to a local terminal so that digital twin data of the local terminal display the first characteristic information of the vehicle;
if the first characteristic information of the vehicle in the first detection result does not contain the original license plate information, continuously identifying whether the original license plate information exists in the second detection result;
If the original license plate information exists in the second detection result, judging whether the second characteristic information of the vehicle and the first characteristic information of the vehicle belong to the same vehicle or not by comparing the second characteristic information of the vehicle in the second detection result with the first characteristic information of the vehicle in the first detection result according to the original license plate information;
if the vehicle is the same vehicle, the second characteristic information of the vehicle in the second detection result is endowed to the vehicle in the first detection result, so that the second characteristic information of the vehicle is displayed in digital twin data of a local terminal, and the local terminal is used for receiving the processing result of the cloud.
3. The method of claim 2, wherein the coordinate system comprises a world coordinate system, and the obtaining the first detection result uploaded by the road side comprises:
acquiring a first vehicle detection result of the road side on the road surface in the current monitoring area range through first sensing equipment, wherein the first sensing equipment obtains the position of the vehicle under the world coordinate system according to a preset calibration relation between the road side sensing equipment and the road surface;
and receiving the position, the vehicle type, the detection time stamp and the vehicle ID of the vehicle in the world coordinate system in the first vehicle detection result.
4. The method of claim 3, wherein the obtaining the second detection result sent by the vehicle side includes:
acquiring a second vehicle detection result of the vehicle side in the surrounding environment through second sensing equipment, wherein the second sensing equipment obtains the position of the vehicle under the world coordinate system according to the self-positioning of the vehicle side and the calibration parameters of the second sensing equipment;
receiving a picture of the position of the vehicle in the image, which is contained in the second vehicle detection result, and the position of the vehicle under a world coordinate system;
and/or the number of the groups of groups,
and receiving a license plate recognition result of the vehicle in the second vehicle detection result and the position of the vehicle under a world coordinate system.
5. The method of claim 1, wherein the identifying license plate information of the target vehicle based on the first detection result and the second detection result further comprises: and taking license plate information of the target vehicle as a unique identification code of the digital twin data, and visually displaying the digital twin data on different terminals.
6. The method of claim 1, wherein the method further comprises:
invoking a history detection result uploaded by the road side;
judging whether the license plate information of the target vehicle is successfully matched or not according to the running track in the history detection result uploaded by the road side;
If the matching is successful, license plate information of the target vehicle in the first detection result uploaded by the road side is directly used;
if the matching is unsuccessful, the first detection result uploaded by the road side and/or the second detection result sent by the vehicle side are synchronously updated to the historical detection result uploaded by the road side.
7. The method of claim 6, wherein the method further comprises:
acquiring a real-time detection result sent by a vehicle side;
judging whether the license plate information of the target vehicle is successfully matched with the target vehicle in the first detection result uploaded by the road side according to the license plate information of the target vehicle in the real-time detection result sent by the vehicle side;
and if the matching is successful, assigning the target vehicle of the first detection result according to the license plate information of the target vehicle.
8. A cloud license plate recognition device, wherein the device comprises:
the first acquisition module is used for acquiring a first detection result uploaded by the road side;
the second acquisition module is used for acquiring a second detection result sent by the vehicle side; and
the identification module is used for identifying license plate information of the target vehicle according to the first detection result and the second detection result, and the second detection result and the first detection result both comprise position information of the target under the same coordinate system.
9. An electronic device, comprising:
a processor; and
a memory arranged to store computer executable instructions which, when executed, cause the processor to perform the method of any of claims 1 to 7.
10. A computer readable storage medium storing one or more programs, which when executed by an electronic device comprising a plurality of application programs, cause the electronic device to perform the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310819763.8A CN116935370A (en) | 2023-07-05 | 2023-07-05 | Cloud license plate recognition method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310819763.8A CN116935370A (en) | 2023-07-05 | 2023-07-05 | Cloud license plate recognition method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116935370A true CN116935370A (en) | 2023-10-24 |
Family
ID=88383606
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310819763.8A Pending CN116935370A (en) | 2023-07-05 | 2023-07-05 | Cloud license plate recognition method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116935370A (en) |
-
2023
- 2023-07-05 CN CN202310819763.8A patent/CN116935370A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112069643B (en) | Automatic driving simulation scene generation method and device | |
| CN110753892A (en) | Method and system for instant object tagging via cross-modality verification in autonomous vehicles | |
| CN110869559A (en) | Method and system for integrated global and distributed learning in autonomous vehicles | |
| CN110753953A (en) | Method and system for object-centric stereo vision in autonomous vehicles via cross-modality verification | |
| CN110942038A (en) | Traffic scene recognition method, device, medium and electronic equipment based on vision | |
| CN114463984B (en) | Vehicle track display method and related equipment | |
| CN114065876B (en) | Data fusion method, device, system and medium based on roadside multi-sensor | |
| CN115164918A (en) | Semantic point cloud map construction method and device and electronic equipment | |
| CN116990776A (en) | Laser radar point cloud compensation method and device, electronic equipment and storage medium | |
| CN113015243A (en) | Vehicle positioning method and device and vehicle | |
| CN115661014A (en) | Point cloud data processing method and device, electronic equipment and storage medium | |
| CN116935370A (en) | Cloud license plate recognition method and device, electronic equipment and storage medium | |
| CN116164763A (en) | Target course angle determining method and device, electronic equipment and storage medium | |
| CN112598314B (en) | Method, device, equipment and medium for determining perception confidence of intelligent driving automobile | |
| CN113890668A (en) | Multi-sensor time synchronization method and device, electronic equipment and storage medium | |
| CN113611120A (en) | Vehicle position information prediction method, device and equipment | |
| JP2022056153A (en) | Temporary stop detection device, temporary stop detection system, and temporary stop detection program | |
| EP3811349A2 (en) | System and method for automatic calibration of vehicle position determining device in a traffic system | |
| CN116381698B (en) | Road remains detection method and device and electronic equipment | |
| CN115841757A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN116958915B (en) | Target detection method, target detection device, electronic equipment and storage medium | |
| CN115236689B (en) | Method and device for determining relative positions of laser radar and image acquisition equipment | |
| CN113642533B (en) | Lane level positioning method and electronic equipment | |
| CN117079236A (en) | Target detection method and device, electronic equipment and storage medium | |
| CN117765768A (en) | Lane departure early warning method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |