CN113657378B - Vehicle tracking method, vehicle tracking system and computing device - Google Patents

Vehicle tracking method, vehicle tracking system and computing device Download PDF

Info

Publication number
CN113657378B
CN113657378B CN202110858203.4A CN202110858203A CN113657378B CN 113657378 B CN113657378 B CN 113657378B CN 202110858203 A CN202110858203 A CN 202110858203A CN 113657378 B CN113657378 B CN 113657378B
Authority
CN
China
Prior art keywords
vehicle
license plate
result
plate number
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110858203.4A
Other languages
Chinese (zh)
Other versions
CN113657378A (en
Inventor
贾若然
李亚玲
曹玲玲
傅云翔
陈向阳
杨文康
王光新
杨昌东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Iflytek Information Technology Co Ltd
Original Assignee
Iflytek Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Iflytek Information Technology Co Ltd filed Critical Iflytek Information Technology Co Ltd
Priority to CN202110858203.4A priority Critical patent/CN113657378B/en
Publication of CN113657378A publication Critical patent/CN113657378A/en
Application granted granted Critical
Publication of CN113657378B publication Critical patent/CN113657378B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Library & Information Science (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention provides a vehicle tracking method, a vehicle tracking system and a computing device, wherein the method comprises the following steps: receiving at least one image to be identified containing a vehicle to be identified, and detecting the vehicle in the image to be identified to obtain a vehicle image and an angle of the vehicle; extracting vector information of a vehicle corresponding to an angle of the vehicle from a vehicle image; retrieving vector information fields corresponding to the angles of the vehicles in a pre-established vehicle information database based on the vector information of the vehicles to obtain candidate results; and screening the candidate results based on the license plate recognition results to obtain final results, wherein the final results indicate the screened matching images which also contain the vehicle in the vehicle information database. The invention fully utilizes the urban monitoring system, can realize vehicle identification and cross-border tracking by utilizing the multi-mode information of the vehicle, and greatly improves the speed and accuracy of the vehicle identification and cross-border tracking.

Description

Vehicle tracking method, vehicle tracking system and computing device
Technical Field
The present invention relates to the field of vehicle tracking, and more particularly to a vehicle tracking method, a vehicle tracking system, and a computing device.
Background
By 6 months in 2020, the national motor vehicles hold 3.6 hundred million vehicles, wherein 2.7 hundred million vehicles account for 75% of the total motor vehicles. In order to realize intelligent management of a large number of motor vehicles with complex driving tracks in cities, a more intelligent algorithm is needed besides a large number of monitoring systems in urban systems.
In the existing city monitoring system, besides cameras at the positions of the bayonets, most cameras are difficult to realize identification of license plates, intelligent analysis of vehicles is realized by simply relying on license plate identification, the effect of the city monitoring system is often not fully utilized, the vehicle retrieval function realized by the vehicle re-identification technology is often only capable of achieving the function of returning similar picture lists, the characteristic difference of different angles of the same vehicle becomes one of the difficulties that the vehicle re-identification technology is difficult to land, for example, the similar vehicles are retrieved in a database by utilizing the front faces of the vehicles, the rear face images of the vehicles are often difficult to obtain, and therefore a large amount of data of different angles of the same vehicle are missed.
How to fully utilize the urban monitoring system to realize intelligent monitoring of the vehicle running track and vehicle cross-mirror tracking becomes a problem to be solved urgently.
Disclosure of Invention
The present invention has been made to solve the above-described problems. According to an aspect of the present invention, there is provided a vehicle tracking method, the method comprising: receiving at least one image to be identified containing a vehicle to be identified, and detecting the vehicle in the image to be identified to obtain a vehicle image and an angle of the vehicle; extracting vector information of the vehicle corresponding to an angle of the vehicle from the vehicle image; retrieving vector information fields corresponding to the angles of the vehicles in a pre-established vehicle information database based on the vector information of the vehicles to obtain candidate results; and screening the candidate results based on license plate recognition results to obtain final results, wherein the final results indicate the screened matching images which also contain the vehicle in the vehicle information database.
In one embodiment, wherein the license plate recognition result includes a license plate number identifiable and a license plate number unidentifiable, wherein screening the candidate result based on the license plate recognition result includes: when the license plate recognition result is that the license plate number can be recognized, screening the candidate result according to a license plate number matching result to obtain a final result, wherein the license plate number matching result indicates whether the license plate number of the candidate result is consistent with the license plate number of the vehicle; and when the license plate recognition result is that the license plate number is unrecognizable, taking the candidate result as the final result.
In one embodiment, the method further comprises: extracting the vector information from the vehicle image and scalar information of the vehicle, wherein screening the candidate result according to a license plate number matching result comprises the following steps: when the license plate number matching result indicates that the license plate number of the candidate result is consistent with the license plate number of the vehicle, the candidate result is taken as the final result; and when the license plate number matching result indicates that the license plate number of the candidate result is inconsistent with the license plate number of the vehicle, screening the candidate result according to scalar information of the vehicle to obtain the final result.
In one embodiment, wherein screening the candidate results according to scalar information for the vehicle comprises: discarding the candidate result if all scalar information of the candidate result is inconsistent with corresponding scalar information of the vehicle, and not taking the candidate result as the final result; and if one of the scalar information of the candidate result is consistent with the corresponding scalar information of the vehicle, the candidate result is taken as the final result.
In one embodiment, the method further comprises: and if the final result is more than one result and at least two results in the more than one result are not matched, assigning a weight to each final result based on the license plate recognition result and the license plate number matching result, and removing an error result according to the weight.
In one embodiment, the method further comprises: and sequencing the images to be identified and the matched images according to the shooting time sequence of the images to be identified and the matched images so as to synthesize the motion trail of the vehicle.
In one embodiment, wherein the scalar information includes a color, a model, and a brand of the vehicle.
According to another embodiment of the present invention, there is provided a vehicle tracking system including: the detection module is used for detecting the vehicle in at least one image to be identified of the received vehicle to obtain a vehicle image and an angle of the vehicle; an information extraction module for extracting vector information of the vehicle corresponding to an angle of the vehicle from the vehicle image; the retrieval module is used for retrieving vector information fields corresponding to the angles of the vehicles in a pre-established vehicle information database based on the vector information of the vehicles so as to obtain candidate results; and the screening module is used for screening the candidate results based on the license plate recognition results to obtain final results, and the final results indicate the screened matching images which also contain the vehicles in the vehicle information database.
In one embodiment, the system further comprises a track synthesis module, configured to sort the images to be identified and the matching images according to the shooting time sequence of the images to be identified and the matching images, so as to synthesize the motion track of the vehicle.
According to yet another embodiment of the present invention, a computing device is provided, comprising a memory and a processor, the memory having stored thereon a computer program which, when executed by the processor, causes the processor to perform the method as described above.
According to a further embodiment of the present invention, a computer readable medium is provided, having stored thereon a computer program which, when executed, performs the method as described above.
According to the vehicle tracking method, the vehicle tracking system and the computing equipment provided by the embodiment of the invention, the urban monitoring system is fully utilized, the vehicle identification and the cross-border tracking can be realized by utilizing the multi-mode information of the vehicle, and the speed and the accuracy of the vehicle identification and the cross-border tracking are greatly improved.
Drawings
The above and other objects, features and advantages of the present invention will become more apparent from the following more particular description of embodiments of the present invention, as illustrated in the accompanying drawings. The accompanying drawings are included to provide a further understanding of embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, and not constitute a limitation to the invention. In the drawings, like reference numerals generally refer to like parts or steps.
FIG. 1 illustrates an electronic device for implementing a vehicle tracking method, a vehicle tracking system, and a computing device in accordance with an embodiment of the invention.
FIG. 2 shows a flow chart of steps of a vehicle tracking method according to one embodiment of the invention.
FIG. 3 shows a schematic block diagram of a vehicle tracking system according to one embodiment of the invention.
FIG. 4 shows a schematic block diagram of a computing device for implementing vehicle tracking, according to one embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some embodiments of the present invention and not all embodiments of the present invention, and it should be understood that the present invention is not limited by the example embodiments described herein. Based on the embodiments of the invention described in the present application, all other embodiments that a person skilled in the art would have without inventive effort shall fall within the scope of the invention.
As described above, the conventional vehicle recognition technology cannot recognize a vehicle from features of the same vehicle at different angles. Therefore, in order to fully utilize the urban monitoring system to realize intelligent monitoring of the vehicle running track and vehicle cross-mirror tracking, the invention provides a vehicle tracking method, which comprises the following steps: receiving at least one image to be identified containing a vehicle to be identified, and detecting the vehicle in the image to be identified to obtain a vehicle image and an angle of the vehicle; extracting vector information of the vehicle corresponding to an angle of the vehicle from the vehicle image; retrieving vector information fields corresponding to the angles of the vehicles in a pre-established vehicle information database based on the vector information of the vehicles to obtain candidate results; and screening the candidate results based on license plate recognition results to obtain final results, wherein the final results indicate the screened matching images which also contain the vehicle in the vehicle information database.
The invention fully utilizes the urban monitoring system, can realize vehicle identification and cross-border tracking by utilizing the multi-mode information of the vehicle, and greatly improves the speed and accuracy of the vehicle identification and cross-border tracking.
The method, system and computing device for recognizing the stroke order of handwritten Chinese characters in real time according to the invention are described in detail below with reference to specific embodiments.
First, an electronic device 100 for implementing a method, system and computing device for restoring a motion trajectory of a vehicle according to an embodiment of the present invention is described with reference to fig. 1.
In one embodiment, the electronic device 100 may be, for example, a notebook computer, a desktop computer, a tablet computer, a learning machine, a mobile device (such as a smartphone, a phone watch, etc.), an embedded computer, a tower server, a rack server, a blade server, or any other suitable electronic device.
In one embodiment, the electronic device 100 may include at least one processor 102 and at least one memory 104.
The memory 104 may be volatile memory, such as Random Access Memory (RAM), cache memory (cache), dynamic Random Access Memory (DRAM) (including stacked DRAM), or High Bandwidth Memory (HBM), etc., or nonvolatile memory, such as Read Only Memory (ROM), flash memory, 3D Xpoint, etc. In one embodiment, some portions of memory 104 may be volatile memory while other portions may be non-volatile memory (e.g., using a two-level memory hierarchy). The memory 104 is used to store a computer program that, when executed, is capable of performing client functions (implemented by a processor) and/or other desired functions in embodiments of the invention described below.
The processor 102 may be a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a microprocessor, or other processing unit having data processing capabilities and/or instruction execution capabilities. The processor 102 may be communicatively coupled to any suitable number or variety of components, peripheral devices, modules, or devices via a communication bus. In one embodiment, the communication bus may be implemented using any suitable protocol, such as Peripheral Component Interconnect (PCI), peripheral component interconnect express (PCIe), accelerated Graphics Port (AGP), hyperTransport, or any other bus or one or more point-to-point communication protocols.
The electronic device 100 may also include an input device 106 and an output device 108. The input device 106 is a device for receiving user input, and may include a keyboard, a mouse, a touch pad, a microphone, and the like. In addition, the input device 106 may be any interface that receives information. The output device 108 may output various information (e.g., images or sounds) to the outside (e.g., a user), which may include one or more of a display, speakers, etc. The output device 108 may be any other device having an output function, such as a printer.
A flowchart of steps of a vehicle tracking method 200 according to one embodiment of the invention is described below with reference to fig. 2. As shown in fig. 2, the vehicle tracking method 200 may include the steps of:
in step S210, at least one image to be identified including the vehicle to be identified is received, and the vehicle in the image to be identified is detected to obtain a vehicle image of the vehicle and an angle of the vehicle.
In one embodiment, detection of the vehicle in the image to be identified may be accomplished using any object detection and image segmentation algorithm known in the art, and image segmentation may also be performed manually by a user, as the invention is not limited in this regard. By way of example, target detection may include, for example, fast R-CNN, R-FCN, YOLO, SSD, retinaNet, etc., as the invention is not limited in this regard.
In one embodiment, after detecting the vehicle in each image, 6 basic attributes of the vehicle may be output. By way of example, the 6 basic attributes may include xmin, xmax, ymax, score, cls_id, where xmax, ymax may represent coordinates of an upper left corner of the vehicle, xmax, ymax may represent coordinates of a lower right corner of the vehicle, score may represent a confidence score, cls_id may represent a type of vehicle, such as a car, a minibus, an SUV, a truck, and the like.
Because the features of different angles of the same vehicle are different, in order to improve the recognition accuracy, in one embodiment, after detecting the vehicle in the image to be recognized, the angle of the vehicle may also be output. Illustratively, the angle of the vehicle may include: front, front left, front right, left, right, front rear, rear left, rear right. It should be understood that other means of indicating the angle of the vehicle may be used, such as indicating the angle between the vehicle head and the horizontal, and the invention is not limited thereto.
The license plate number of a vehicle is a unique ID of the vehicle, and the distinguishing ability of the vehicle is far greater than other information of the vehicle. Thus, in one embodiment, after obtaining the vehicle image of the vehicle, the license plate number of the vehicle may be identified based on the vehicle image to obtain a license plate identification result, wherein the license plate identification result may include a license plate number identifiable and a license plate number unrecognizable.
In one embodiment, the license plate number of the vehicle may be identified by a trained license plate identification network, or may be identified manually, as the invention is not limited in this regard. For example, when a trained license plate recognition network is employed to recognize the license plate number of a vehicle, the license plate recognition network may include license plate extraction, image preprocessing, feature extraction, license plate character recognition, etc., as is well known in the art and will not be described in detail herein.
If the license plate recognition result is that the license plate number can be recognized, the recognized license plate number is saved; if the license plate number is unrecognizable as a result of the license plate recognition, the license plate number is empty.
The license plate recognition result may also include information such as color of the license plate, which is not limited in the present invention.
In step S220, vector information of the vehicle corresponding to the angle of the vehicle is extracted from the vehicle image.
The vector information may include a feature vector of the vehicle, among others. Because the feature vectors of different angles of the same vehicle are different, in the case that the angles of the vehicle include eight angles of right front, left front, right front, left, right rear, left rear and right rear, the feature vectors of the vehicle may include 16 kinds of feature vectors of eight angles in the case that the license plate number is identifiable and the license plate number is unrecognizable, respectively.
By way of example, feature extraction algorithms known in the art may be employed to extract feature vectors of a vehicle, such as LBP (local binary pattern) feature extraction algorithm, HOG feature extraction algorithm, haar feature extraction algorithm, loG feature extraction algorithm, harris corner feature extraction algorithm, SIFT feature extraction algorithm, SURF feature extraction algorithm, or other feature extraction enabled algorithms, and the like, as the present invention is not limited thereto.
In one embodiment, vector information is extracted from the vehicle image along with scalar information for the vehicle. The scalar information may include information of a color, a brand, a vehicle type, and the like of the vehicle. It should be understood that the license plate number and the angle of the vehicle also belong to scalar information of the vehicle, but the scalar information of the vehicle to be identified extracted in this step is not described herein as including the license plate number and the angle of the vehicle, but this is not a limitation, only for convenience of description.
In step S230, vector information fields corresponding to the angles of the vehicles in the pre-established vehicle information database are retrieved based on the vector information of the vehicles to obtain candidate results.
In one embodiment, the vehicle information database, in particular, may be built up using video files captured by existing monitoring systems:
When a vehicle enters the visual field of a camera of the monitoring system, the monitoring system intelligently records a piece of data in a vehicle information database, and mainly records the following information: license plate number, color of vehicle, brand, vehicle type, angle of vehicle, feature vector, start time, end time. For example, the vehicle information database may include scalar information fields that may include information of license plate numbers, colors of vehicles, brands, vehicle types, angles of vehicles, and the like, and vector information fields that may include feature vectors of vehicles at different angles. The color, brand, license plate number and the like of the vehicle can possibly be unrecognizable due to angles, distances, light rays and the like, and the angles and the feature vectors of the vehicle are necessary fields, so that whether the interference of the factors such as the distances, the light rays and the like of the vehicle exists or not can be obtained.
The system can track and identify the vehicle, when the vehicle information changes, the related information is updated, for example, if the angle of the vehicle changes, for example, the angle changes from the right rear to the left rear, the characteristic vector at the left rear of the vehicle is extracted and written into the database; when the vehicle approaches the camera from a far place, the license plate is changed from unrecognizable to identifiable, license plate information, such as a license plate number, is additionally recorded. When the vehicle is changed from far to near and from unclear to clear, a plurality of information is changed from unrecognizable to identifiable, and then the feature vector of the vehicle at the angle is updated to be the vehicle image in the frame with the most complete scalar information.
Specifically, when the search is performed, only the vector information field in the vehicle information database corresponding to the angle of the vehicle to be identified needs to be searched. For example, if the angle of the vehicle to be identified is the left front, the extracted feature vector of the vehicle to be identified is the left front feature vector, and only the vector information field corresponding to the left front in the vehicle information database is required to be searched to obtain the candidate result. In one embodiment, if multiple candidate results are retrieved, the multiple candidate results may be ranked according to similarity to the feature vector of the vehicle to be identified.
In step S240, candidate results are screened based on license plate recognition results to obtain final results. The final result indicates the screened matching image which also contains the vehicle to be identified in the vehicle information database.
In one embodiment, when candidate results are screened based on license plate recognition results, it can be described that license plate numbers are identifiable and license plate numbers are not identifiable. Specifically:
When the license plate recognition result is that the license plate number can be recognized, the candidate result can be screened according to the license plate number matching result to obtain a final result, wherein the license plate number matching result can indicate whether the license plate number of the candidate result is consistent with the license plate number of the vehicle to be recognized. For example, by way of example, the license plate number matching result being positive may indicate that the license plate number of the candidate result is consistent with the license plate number of the vehicle to be identified; the number plate number matching result is negative, and the number plate number of the candidate result is not consistent with the number plate number of the vehicle to be identified. It should be appreciated that other representation methods may be used for the license plate number matching result, such as the numbers 0 (indicating inconsistency) and 1 (indicating consistency), and the invention is not limited in this regard.
In one embodiment, when the license plate number matching result indicates that the license plate number of the candidate result is consistent with the license plate number of the vehicle to be identified, the candidate result is taken as a final result. Because the license plate number is the unique ID of the vehicle, the distinguishing degree is far greater than other information of the vehicle, and therefore, when the license plate number of the candidate result is consistent with the license plate number of the vehicle to be identified, other comparison is not needed, and the candidate result can be directly used as a final result.
In one embodiment, when the license plate number matching result indicates that the license plate number of the candidate result is inconsistent with the license plate number of the vehicle to be identified, other information of the vehicle needs to be compared, and at this time, the candidate result can be screened according to scalar information of the vehicle to be identified, for example, scalar information such as color, brand and the like of the vehicle can be compared, so as to obtain a final result.
Illustratively, if all scalar information of the candidate result is inconsistent with corresponding scalar information of the vehicle to be identified, such as color, brand, vehicle type, etc., indicating that the candidate result is less likely to be the vehicle to be identified, discarding the candidate result and not taking it as a final result;
For example, if one of the scalar information of the candidate result is consistent with the corresponding scalar information of the vehicle to be identified, indicating that the candidate result is still likely to be the vehicle to be identified, the candidate result is taken as the final result.
When the license plate recognition result is that the license plate number is unrecognizable, other information is not required to be compared, and the candidate result is directly used as a final result. For example, the end result may be ranked by similarity to the vehicle to be identified.
If the final result is more than one result, there is sometimes a case that the results conflict, i.e. at least two of the more than one results do not match, and then erroneous results need to be excluded. For example, 3 images of a vehicle at different angles are obtained from camera a, retrieved from a nearby camera, and found to have two eligible options, where 2 images of camera B can be matched, showing the vehicle traveling 8 km eastward, 3 images of camera C can be matched, showing the vehicle traveling 10 km westward, and time analysis of the vehicle is unlikely to occur at the point at which camera B, C is located, so that 2 images of camera B and 3 images of camera C are conflicting.
At this time, in one embodiment, a weight may be assigned to each final result based on the license plate recognition result and the license plate number matching result, and the error result may be removed according to the weight.
Illustratively, the rule for assigning weights based on license plate recognition results and license plate number matching results may be set as follows:
Whether or not to identify license plates Whether the license plates are matched Weighting of
Is that Is that 3.0
Is that Whether or not 1.0
Whether or not 1.0
After the error result is removed and the correct result is obtained, the images to be recognized and the matched images are sequenced according to the shooting time sequence of the images to be recognized and the matched images so as to synthesize the motion trail of the vehicle to be recognized, namely, the restoration of the motion trail of the vehicle is completed.
The method is used for the vehicle tracking method, fully utilizes the urban monitoring system, can realize vehicle identification and cross-border tracking by utilizing the multi-mode information of the vehicle, and greatly improves the speed and accuracy of the vehicle identification and cross-border tracking.
A schematic block diagram of a vehicle tracking system 300 according to one embodiment of the invention is described below with reference to fig. 3. As shown in fig. 3, the vehicle tracking system 300 may include a detection module 310, an information extraction module 330, a retrieval module 340, a screening module 350, and a trajectory synthesis module 360. The vehicle tracking system 300 may also include an identification module 320.
The detection module 310 is configured to detect a vehicle in at least one image to be identified of the received vehicle to obtain a vehicle image of the vehicle to be identified and an angle of the vehicle.
In one embodiment, detection of the vehicle in the image to be identified may be accomplished using any object detection and image segmentation algorithm known in the art, and image segmentation may also be performed manually by a user, as the invention is not limited in this regard. By way of example, target detection may include, for example, fast R-CNN, R-FCN, YOLO, SSD, retinaNet, etc., as the invention is not limited in this regard.
In one embodiment, after detecting the vehicle in each image, 6 basic attributes of the vehicle may be output. By way of example, the 6 basic attributes may include xmin, xmax, ymax, score, cls_id, where xmax, ymax may represent coordinates of an upper left corner of the vehicle, xmax, ymax may represent coordinates of a lower right corner of the vehicle, score may represent a confidence score, cls_id may represent a type of vehicle, such as a car, a minibus, an SUV, a truck, and the like.
Because the features of different angles of the same vehicle are different, in order to improve the recognition accuracy, in one embodiment, after detecting the vehicle in the image to be recognized, the angle of the vehicle may also be output. Illustratively, the angle of the vehicle may include: front, front left, front right, left, right, front rear, rear left, rear right. It should be understood that other means of indicating the angle of the vehicle may be used, such as indicating the angle between the vehicle head and the horizontal, and the invention is not limited thereto.
The license plate number of a vehicle is a unique ID of the vehicle, and the distinguishing ability of the vehicle is far greater than other information of the vehicle. Accordingly, after obtaining the vehicle image of the vehicle, the license plate number of the vehicle may be identified by the identification module 320 based on the vehicle image to obtain a license plate identification result, wherein the license plate identification result includes a license plate number identifiable and a license plate number unrecognizable.
In one embodiment, the license plate number of the vehicle may be identified by a trained license plate identification network, or may be identified manually, as the invention is not limited in this regard. For example, when a trained license plate recognition network is employed to recognize the license plate number of a vehicle, the license plate recognition network may include license plate extraction, image preprocessing, feature extraction, license plate character recognition, etc., as is well known in the art and will not be described in detail herein.
If the license plate recognition result is that the license plate number can be recognized, the recognized license plate number is saved; if the license plate number is unrecognizable as a result of the license plate recognition, the license plate number is empty.
The license plate recognition result may also include information such as color of the license plate, which is not limited in the present invention.
The information extraction module 330 is used to extract vector information of the vehicle corresponding to the angle of the vehicle from the vehicle image.
The vector information may include a feature vector of the vehicle, among others. Because the feature vectors of different angles of the same vehicle are different, in the case that the angles of the vehicle include eight angles of right front, left front, right front, left, right rear, left rear and right rear, the feature vectors of the vehicle may include 16 kinds of feature vectors of eight angles in the case that the license plate number is identifiable and the license plate number is unrecognizable, respectively.
By way of example, feature extraction algorithms known in the art may be employed to extract feature vectors of a vehicle, such as LBP (local binary pattern) feature extraction algorithm, HOG feature extraction algorithm, haar feature extraction algorithm, loG feature extraction algorithm, harris corner feature extraction algorithm, SIFT feature extraction algorithm, SURF feature extraction algorithm, or other feature extraction enabled algorithms, and the like, as the present invention is not limited thereto.
In one embodiment, vector information is extracted from the vehicle image along with scalar information for the vehicle. The scalar information may include information of a color, a brand, a vehicle type, and the like of the vehicle. It should be understood that the license plate number and the angle of the vehicle also belong to scalar information of the vehicle, but the scalar information of the vehicle to be identified extracted in this step is not described herein as including the license plate number and the angle of the vehicle, but this is not a limitation, only for convenience of description.
The retrieving module 340 is configured to retrieve vector information fields corresponding to angles of vehicles in a pre-established vehicle information database based on vector information of the vehicles, so as to obtain candidate results.
In one embodiment, the vehicle information database, in particular, may be built up using video files captured by existing monitoring systems:
When a vehicle enters the visual field of a camera of the monitoring system, the monitoring system intelligently records a piece of data in a vehicle information database, and mainly records the following information: license plate number, color of vehicle, brand, vehicle type, angle of vehicle, feature vector, start time, end time. For example, the vehicle information database may include scalar information fields that may include information of license plate numbers, colors of vehicles, brands, vehicle types, angles of vehicles, and the like, and vector information fields that may include feature vectors of vehicles at different angles. The color, brand, license plate number and the like of the vehicle can possibly be unrecognizable due to angles, distances, light rays and the like, and the angles and the feature vectors of the vehicle are necessary fields, so that whether the interference of the factors such as the distances, the light rays and the like of the vehicle exists or not can be obtained.
The system can track and identify the vehicle, when the vehicle information changes, the related information is updated, for example, if the angle of the vehicle changes, for example, the angle changes from the right rear to the left rear, the characteristic vector at the left rear of the vehicle is extracted and written into the database; when the vehicle approaches the camera from a far place, the license plate is changed from unrecognizable to identifiable, license plate information, such as a license plate number, is additionally recorded. When the vehicle is changed from far to near and from unclear to clear, a plurality of information is changed from unrecognizable to identifiable, and then the feature vector of the vehicle at the angle is updated to be the vehicle image in the frame with the most complete scalar information.
Specifically, when the search is performed, only the vector information field in the vehicle information database corresponding to the angle of the vehicle to be identified needs to be searched. For example, if the angle of the vehicle to be identified is the left front, the extracted feature vector of the vehicle to be identified is the left front feature vector, and only the vector information field corresponding to the left front in the vehicle information database is required to be searched to obtain the candidate result. In one embodiment, if multiple candidate results are retrieved, the multiple candidate results may be ranked according to similarity to the feature vector of the vehicle to be identified.
The screening module 350 is configured to screen the candidate result based on the license plate recognition result to obtain a final result. The final result indicates the screened matching image which also contains the vehicle to be identified in the vehicle information database.
In one embodiment, when candidate results are screened based on license plate recognition results, it can be described that license plate numbers are identifiable and license plate numbers are not identifiable. Specifically:
When the license plate recognition result is that the license plate number can be recognized, the candidate result can be screened according to the license plate number matching result to obtain a final result, wherein the license plate number matching result can indicate whether the license plate number of the candidate result is consistent with the license plate number of the vehicle to be recognized. For example, by way of example, the license plate number matching result being positive may indicate that the license plate number of the candidate result is consistent with the license plate number of the vehicle to be identified; the number plate number matching result is negative, and the number plate number of the candidate result is not consistent with the number plate number of the vehicle to be identified. It should be appreciated that other representation methods may be used for the license plate number matching result, such as the numbers 0 (indicating inconsistency) and 1 (indicating consistency), and the invention is not limited in this regard.
In one embodiment, when the license plate number matching result indicates that the license plate number of the candidate result is consistent with the license plate number of the vehicle to be identified, the candidate result is taken as a final result. Because the license plate number is the unique ID of the vehicle, the distinguishing degree is far greater than other information of the vehicle, and therefore, when the license plate number of the candidate result is consistent with the license plate number of the vehicle to be identified, other comparison is not needed, and the candidate result can be directly used as a final result.
In one embodiment, when the license plate number matching result indicates that the license plate number of the candidate result is inconsistent with the license plate number of the vehicle to be identified, other information of the vehicle needs to be compared, and at this time, the candidate result can be screened according to scalar information of the vehicle to be identified, for example, scalar information such as color, brand and the like of the vehicle can be compared, so as to obtain a final result.
Illustratively, if all scalar information of the candidate result is inconsistent with corresponding scalar information of the vehicle to be identified, such as color, brand, vehicle type, etc., indicating that the candidate result is less likely to be the vehicle to be identified, discarding the candidate result and not taking it as a final result;
For example, if one of the scalar information of the candidate result is consistent with the corresponding scalar information of the vehicle to be identified, indicating that the candidate result is still likely to be the vehicle to be identified, the candidate result is taken as the final result.
When the license plate recognition result is that the license plate number is unrecognizable, other information is not required to be compared, and the candidate result is directly used as a final result. For example, the end result may be ranked by similarity to the vehicle to be identified.
If the final result is more than one result, there is sometimes a case that the results conflict, i.e. at least two of the more than one results do not match, and then erroneous results need to be excluded.
At this time, in one embodiment, a weight may be assigned to each final result based on the license plate recognition result and the license plate number matching result, and the error result may be removed according to the weight.
For example, the rule for assigning weights based on the license plate recognition result and the license plate number matching result may be set as described above, and will not be described in detail herein.
After the error result is removed and the correct result is obtained, the images to be identified and the matching images can be sequenced by utilizing the track synthesis module 360 according to the shooting time sequence of the images to be identified and the matching images so as to synthesize the motion track of the vehicle to be identified, namely, the restoration of the motion track of the vehicle is completed.
The vehicle tracking system fully utilizes the urban monitoring system, can realize vehicle identification and cross-border tracking by utilizing the multi-mode information of the vehicle, and greatly improves the speed and accuracy of the vehicle identification and cross-border tracking.
Referring now to FIG. 4, FIG. 4 shows a schematic block diagram of a computing device 400 for restoring a motion profile of a vehicle in accordance with one embodiment of the invention. As shown in fig. 4, computing device 400 may include a memory 410 and a processor 420, wherein memory 410 has stored thereon a computer program that, when executed by the processor 420, causes the processor 420 to perform the vehicle tracking method as described above.
Those skilled in the art will understand the specific operation of the computing device 400 according to embodiments of the present invention in conjunction with the foregoing description, and for brevity, only some of the main operations of the processor 420 will be described below without further elaboration:
receiving at least one image to be identified containing a vehicle to be identified, and detecting the vehicle in the image to be identified to obtain a vehicle image and an angle of the vehicle;
Extracting vector information of the vehicle corresponding to an angle of the vehicle from the vehicle image;
Retrieving vector information fields corresponding to the angles of the vehicles in a pre-established vehicle information database based on the vector information of the vehicles to obtain candidate results;
And screening the candidate results based on license plate recognition results to obtain final results, wherein the final results indicate the screened matching images which also contain the vehicle in the vehicle information database.
The computing equipment for implementing vehicle tracking fully utilizes the urban monitoring system, can realize vehicle identification and cross-border tracking by utilizing the multi-mode information of the vehicle, and greatly improves the speed and accuracy of vehicle identification and cross-border tracking.
The invention also provides a computer readable medium having stored thereon a computer program which, when run, performs a method as described in the above embodiments. Any tangible, non-transitory computer readable medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, blu-ray discs, etc.), flash memory, and/or the like. These computer program instructions may be loaded onto a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions which execute on the computer or other programmable data processing apparatus create means for implementing the functions specified. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including means which implement the function specified. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified.
Although the illustrative embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the above illustrative embodiments are merely illustrative and are not intended to limit the scope of the present invention thereto. Various changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the present invention as set forth in the appended claims.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in order to streamline the invention and aid in understanding one or more of the various inventive aspects, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, the method of the present invention should not be construed as reflecting the following intent: i.e., the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
It will be understood by those skilled in the art that all of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be combined in any combination, except combinations where the features are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names.
The foregoing description is merely illustrative of specific embodiments of the present invention and the scope of the present invention is not limited thereto, and any person skilled in the art can easily think about variations or substitutions within the scope of the present invention. The protection scope of the invention is subject to the protection scope of the claims.

Claims (11)

1. A method of vehicle tracking, the method comprising:
receiving at least one image to be identified containing a vehicle to be identified, and detecting the vehicle in the image to be identified to obtain a vehicle image and an angle of the vehicle;
extracting vector information and scalar information of the vehicle corresponding to an angle of the vehicle from the vehicle image, the vector information including a feature vector of the vehicle;
Retrieving vector information fields corresponding to the angles of the vehicles in a pre-established vehicle information database based on the vector information of the vehicles to obtain candidate results, wherein the angles of the vehicles and the feature vectors are necessary fields in the vehicle information database;
and screening the candidate result based on the license plate recognition result of the vehicle to obtain a final result, wherein the final result indicates the screened matching image which also contains the vehicle in the vehicle information database.
2. The method of claim 1, wherein the license plate recognition result comprises a license plate number identifiable and a license plate number unidentifiable, wherein screening the candidate result based on the license plate recognition result comprises:
When the license plate recognition result indicates that the license plate number can be recognized, screening the candidate result according to a license plate number matching result to obtain a final result, wherein the license plate number matching result indicates whether the license plate number of the candidate result is consistent with the license plate number of the vehicle;
And when the license plate recognition result is that the license plate number is unrecognizable, taking the candidate result as the final result.
3. The method of claim 2, wherein screening the candidate results based on the license plate number matching results comprises:
When the license plate number matching result indicates that the license plate number of the candidate result is consistent with the license plate number of the vehicle, the candidate result is taken as the final result;
and when the license plate number matching result indicates that the license plate number of the candidate result is inconsistent with the license plate number of the vehicle, screening the candidate result according to scalar information of the vehicle to obtain the final result.
4. The method of claim 3, wherein screening the candidate results based on scalar information for the vehicle comprises:
Discarding the candidate result if all scalar information of the candidate result is inconsistent with corresponding scalar information of the vehicle, and not taking the candidate result as the final result;
and if one of the scalar information of the candidate result is consistent with the corresponding scalar information of the vehicle, the candidate result is taken as the final result.
5. The method of claim 1, wherein the method further comprises:
And if the final result is more than one result and at least two results in the more than one result are not matched, assigning a weight to each final result based on the license plate recognition result and the license plate number matching result, and removing an error result according to the weight.
6. The method of claim 1, wherein the method further comprises:
and sequencing the images to be identified and the matched images according to the shooting time sequence of the images to be identified and the matched images so as to synthesize the motion trail of the vehicle.
7. A method according to claim 3, wherein the scalar information includes a color, a model and a brand of the vehicle.
8. A vehicle tracking system, the system comprising:
The detection module is used for detecting the vehicle in at least one image to be identified of the received vehicle to obtain a vehicle image and an angle of the vehicle;
An information extraction module for extracting vector information and scalar information of the vehicle corresponding to an angle of the vehicle from the vehicle image, the vector information including a feature vector of the vehicle;
The retrieval module is used for retrieving vector information fields corresponding to the angles of the vehicles in a pre-established vehicle information database based on the vector information of the vehicles to obtain candidate results, wherein the angles and the feature vectors of the vehicles are necessary fields in the vehicle information database;
And the screening module is used for screening the candidate result based on the license plate recognition result of the vehicle to obtain a final result, and the final result indicates the screened matching image which also contains the vehicle in the vehicle information database.
9. The system of claim 8, further comprising a trajectory synthesis module for ordering the images to be identified and the matching images in chronological order of their capture to synthesize a motion trajectory of the vehicle.
10. A computing device comprising a memory and a processor, the memory having stored thereon a computer program which, when executed by the processor, causes the processor to perform the method of any of claims 1-7.
11. A computer readable medium, characterized in that it has stored thereon a computer program which, when executed, performs the method according to any of claims 1-7.
CN202110858203.4A 2021-07-28 2021-07-28 Vehicle tracking method, vehicle tracking system and computing device Active CN113657378B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110858203.4A CN113657378B (en) 2021-07-28 2021-07-28 Vehicle tracking method, vehicle tracking system and computing device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110858203.4A CN113657378B (en) 2021-07-28 2021-07-28 Vehicle tracking method, vehicle tracking system and computing device

Publications (2)

Publication Number Publication Date
CN113657378A CN113657378A (en) 2021-11-16
CN113657378B true CN113657378B (en) 2024-04-26

Family

ID=78478856

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110858203.4A Active CN113657378B (en) 2021-07-28 2021-07-28 Vehicle tracking method, vehicle tracking system and computing device

Country Status (1)

Country Link
CN (1) CN113657378B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114219836B (en) * 2021-12-15 2022-06-03 北京建筑大学 Unmanned aerial vehicle video vehicle tracking method based on space-time information assistance

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106682087A (en) * 2016-11-28 2017-05-17 东南大学 Method for retrieving vehicles on basis of sparse codes of features of vehicular ornaments
CN106778648A (en) * 2016-12-26 2017-05-31 江苏盛世华安智能科技股份有限公司 Vehicle tracing and Vehicle License Plate Recognition System and recognition methods
CN107729818A (en) * 2017-09-21 2018-02-23 北京航空航天大学 A kind of multiple features fusion vehicle recognition methods again based on deep learning
CN109409208A (en) * 2018-09-10 2019-03-01 东南大学 A kind of vehicle characteristics extraction and matching process based on video
CN110766720A (en) * 2019-09-23 2020-02-07 盐城吉大智能终端产业研究院有限公司 Multi-camera vehicle tracking system based on deep learning
CN111932901A (en) * 2019-05-13 2020-11-13 阿里巴巴集团控股有限公司 Road vehicle tracking detection apparatus, method and storage medium
CN112069969A (en) * 2020-08-31 2020-12-11 河北省交通规划设计院 Method and system for tracking highway monitoring video mirror-crossing vehicle

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8620026B2 (en) * 2011-04-13 2013-12-31 International Business Machines Corporation Video-based detection of multiple object types under varying poses
US8698896B2 (en) * 2012-08-06 2014-04-15 Cloudparc, Inc. Controlling vehicle use of parking spaces and parking violations within the parking spaces using multiple cameras
CN110245565A (en) * 2019-05-14 2019-09-17 东软集团股份有限公司 Wireless vehicle tracking, device, computer readable storage medium and electronic equipment

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106682087A (en) * 2016-11-28 2017-05-17 东南大学 Method for retrieving vehicles on basis of sparse codes of features of vehicular ornaments
CN106778648A (en) * 2016-12-26 2017-05-31 江苏盛世华安智能科技股份有限公司 Vehicle tracing and Vehicle License Plate Recognition System and recognition methods
CN107729818A (en) * 2017-09-21 2018-02-23 北京航空航天大学 A kind of multiple features fusion vehicle recognition methods again based on deep learning
CN109409208A (en) * 2018-09-10 2019-03-01 东南大学 A kind of vehicle characteristics extraction and matching process based on video
CN111932901A (en) * 2019-05-13 2020-11-13 阿里巴巴集团控股有限公司 Road vehicle tracking detection apparatus, method and storage medium
CN110766720A (en) * 2019-09-23 2020-02-07 盐城吉大智能终端产业研究院有限公司 Multi-camera vehicle tracking system based on deep learning
CN112069969A (en) * 2020-08-31 2020-12-11 河北省交通规划设计院 Method and system for tracking highway monitoring video mirror-crossing vehicle

Also Published As

Publication number Publication date
CN113657378A (en) 2021-11-16

Similar Documents

Publication Publication Date Title
CN106446816B (en) Face recognition method and device
JP5775225B2 (en) Text detection using multi-layer connected components with histograms
Wang et al. An effective method for plate number recognition
Li et al. An anti-fraud system for car insurance claim based on visual evidence
CN111353491B (en) Text direction determining method, device, equipment and storage medium
US10354134B1 (en) Feature classification with spatial analysis
CN103699905A (en) Method and device for positioning license plate
Molina-Moreno et al. Efficient scale-adaptive license plate detection system
CN104615986A (en) Method for utilizing multiple detectors to conduct pedestrian detection on video images of scene change
US10423817B2 (en) Latent fingerprint ridge flow map improvement
CN111814690B (en) Target re-identification method, device and computer readable storage medium
US8027978B2 (en) Image search method, apparatus, and program
US20190205693A1 (en) Scale-Invariant Feature Point Extraction in Edge Map
CN110826415A (en) Method and device for re-identifying vehicles in scene image
He et al. Aggregating local context for accurate scene text detection
CN111767908A (en) Character detection method, device, detection equipment and storage medium
CN106384071A (en) Two-dimensional code scanning recognition processing method and device
CN113657378B (en) Vehicle tracking method, vehicle tracking system and computing device
CN115171125A (en) Data anomaly detection method
Ganapathy et al. A Malaysian vehicle license plate localization and recognition system
CN114168768A (en) Image retrieval method and related equipment
CN114783042A (en) Face recognition method, device, equipment and storage medium based on multiple moving targets
CN110717406B (en) Face detection method and device and terminal equipment
Shekar Skeleton matching based approach for text localization in scene images
Chen et al. Trademark image retrieval system based on SIFT algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant