CN113806576B - Image-based vehicle retrieval method and device, electronic equipment and storage medium - Google Patents
Image-based vehicle retrieval method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113806576B CN113806576B CN202110913283.9A CN202110913283A CN113806576B CN 113806576 B CN113806576 B CN 113806576B CN 202110913283 A CN202110913283 A CN 202110913283A CN 113806576 B CN113806576 B CN 113806576B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- queried
- image
- license plate
- apparent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/55—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Traffic Control Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a vehicle retrieval method and device based on images, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring apparent characteristics of a vehicle to be queried; acquiring license plate characteristics of a license plate to be queried; calculating the space-time distance between the vehicle to be queried and the standard image in the vehicle image database according to the principle from the near to the far; according to the apparent features, the license plate features and the space-time distance, calculating the similarity between the image to be queried and each standard image in a vehicle image database to obtain a first similarity sequence; determining a standard image with highest similarity according to the first similarity sequence to obtain a retrieval result of the vehicle to be queried; the embodiment of the application simultaneously combines the apparent characteristics and license plate characteristics of the vehicle to search, improves the search accuracy of the target vehicle, and can be widely applied to the technical field of image processing.
Description
Technical Field
The application relates to the technical field of image processing, in particular to a vehicle retrieval method and device based on images, electronic equipment and a storage medium.
Background
The image search technology is an image-based vehicle search technology, and is widely applied to the fields of video monitoring, intelligent transportation, smart cities and the like.
Along with DCNN (Deep Convolutional Neural Network) in many application fields, such as pedestrian re-identification, face recognition, fine granularity classification and the like, great breakthroughs are made, and the application of DCNN (Deep Convolutional Neural Network) in the field of vehicle searching in a drawing is promoted. Currently, a DCNN (Deep Convolutional Neural Network) framework is used for searching the vehicle field in a graph to extract global apparent features and local apparent features of the vehicle, so that the feature distances of the same target vehicle are close, and the feature distances of different target vehicles are far away, so that the aim that the target vehicles can be ranked in front during searching is fulfilled. However, in practical applications, the apparent features of the same vehicle have large intra-class differences under different cameras, and the inter-class differences of the apparent features of different vehicles in the same camera are very small, so that the method based on the apparent features is difficult to obtain optimal results.
Disclosure of Invention
In view of this, the embodiments of the present application provide a vehicle searching method, apparatus, electronic device and storage medium based on images, so as to improve the searching accuracy of the target vehicle.
An aspect of the present application provides an image-based vehicle retrieval method, including:
acquiring apparent characteristics of a vehicle to be queried;
acquiring license plate characteristics of a license plate to be queried;
calculating the space-time distance between the vehicle to be queried and the standard image in the vehicle image database according to the principle from the near to the far;
according to the apparent features, the license plate features and the space-time distance, calculating the similarity between the image to be queried and each standard image in a vehicle image database to obtain a first similarity sequence;
and determining a standard image with highest similarity according to the first similarity sequence to obtain a retrieval result of the vehicle to be queried.
Optionally, the obtaining the apparent characteristics of the vehicle to be queried includes:
acquiring a vehicle image set, classifying images belonging to the same vehicle in the vehicle image set into the same category, and obtaining a training set;
establishing a feature extraction network of apparent features of the vehicle;
inputting the training set into the feature extraction network to obtain a vehicle apparent feature model;
and extracting global apparent features and local apparent features of the vehicle to be queried through the vehicle apparent feature model.
Optionally, the obtaining the apparent characteristics of the vehicle to be queried further includes:
calculating the similarity between the apparent features of the vehicles to be queried to obtain a second similarity sequence of the apparent features;
determining a first filtering threshold according to the second similarity sequence;
and filtering the apparent characteristics of the vehicle to be queried according to the first filtering threshold.
Optionally, the obtaining the license plate feature of the license plate to be queried includes:
acquiring a vehicle image set, classifying images belonging to the same vehicle in the vehicle image set into the same category, and obtaining a training set;
establishing a license plate verification network;
and inputting the training set into the license plate verification network, and extracting to obtain license plate features.
Optionally, the obtaining the license plate feature of the license plate to be queried further includes:
calculating the similarity between license plate features of the vehicle to be queried to obtain a third similarity sequence of the license plate features;
determining a second filtering threshold according to the third similarity sequence and the second similarity sequence;
and filtering the license plate features according to the second filtering threshold.
Optionally, the calculating the space-time distance between the vehicle to be queried and the standard image in the vehicle image database according to the principle from the near to the far comprises:
acquiring the image of the vehicle to be queried and the timestamp information of a standard image in a vehicle image database;
acquiring the maximum time difference between the image of the vehicle to be queried and the standard image;
acquiring the shortest distance between cameras corresponding to the images of the vehicle to be queried;
obtaining the maximum distance between all cameras;
and determining the space-time distance according to the timestamp information, the maximum time difference, the shortest distance and the maximum distance.
Optionally, the calculation formula of the space-time distance is:
wherein T is i And T j Time stamps respectively representing an image i of the vehicle to be queried and a standard image j in a vehicle image database, T max Representing the maximum time difference between the images of all vehicles to be queried and the standard images in the vehicle image database; delta (C) i ,C j ) Is a camera C for shooting an image i of the vehicle to be queried i With a camera C which shoots an image j of the vehicle to be queried j Shortest distance between D max Is the maximum distance between all cameras.
Another aspect of an embodiment of the present application provides an image-based vehicle retrieval apparatus, including:
the first acquisition module is used for acquiring apparent characteristics of the vehicle to be queried;
the second acquisition module is used for acquiring license plate characteristics of the license plate to be inquired;
the first calculation module is used for calculating the space-time distance between the vehicle to be queried and the standard image in the vehicle image database according to the principle from the near to the far;
the second calculation module is used for calculating the similarity between the image to be queried and each standard image in the vehicle image database according to the apparent characteristics, the license plate characteristics and the space-time distance to obtain a first similarity sequence;
and the determining module is used for determining a standard image with highest similarity according to the first similarity sequence to obtain a retrieval result of the vehicle to be queried.
Another aspect of an embodiment of the present application provides an electronic device, including a processor and a memory;
the memory is used for storing programs;
the processor executes the program to implement the method as described above.
Another aspect of the embodiments of the present application provides a computer-readable storage medium storing a program that is executed by a processor to implement a method as described above.
Embodiments of the present application also disclose a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions may be read from a computer-readable storage medium by a processor of a computer device, and executed by the processor, to cause the computer device to perform the foregoing method.
The method comprises the steps of firstly obtaining apparent characteristics of a vehicle to be queried; then obtaining license plate characteristics of the license plate to be inquired; then calculating the space-time distance between the vehicle to be queried and the standard image in the vehicle image database according to the principle from the near to the far; calculating the similarity between the image to be queried and each standard image in a vehicle image database according to the apparent features, the license plate features and the space-time distance to obtain a first similarity sequence; finally, determining a standard image with highest similarity according to the first similarity sequence to obtain a retrieval result of the vehicle to be queried; according to the embodiment of the application, the apparent features and license plate features of the vehicle are combined for searching, so that the searching accuracy of the target vehicle is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of an image-based vehicle retrieval method provided by an embodiment of the present application;
FIG. 2 is a schematic flow chart of extraction, sorting and filtering of apparent features of a vehicle according to an embodiment of the present application;
fig. 3 is a schematic flow chart of license plate feature extraction, sorting and filtering according to an embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
Aiming at the problems existing in the prior art, the application provides a vehicle retrieval method based on images, which comprises the following steps:
acquiring apparent characteristics of a vehicle to be queried;
acquiring license plate characteristics of a license plate to be queried;
calculating the space-time distance between the vehicle to be queried and the standard image in the vehicle image database according to the principle from the near to the far;
according to the apparent features, the license plate features and the space-time distance, calculating the similarity between the image to be queried and each standard image in a vehicle image database to obtain a first similarity sequence;
and determining a standard image with highest similarity according to the first similarity sequence to obtain a retrieval result of the vehicle to be queried.
Optionally, the obtaining the apparent characteristics of the vehicle to be queried includes:
acquiring a vehicle image set, classifying images belonging to the same vehicle in the vehicle image set into the same category, and obtaining a training set;
establishing a feature extraction network of apparent features of the vehicle;
inputting the training set into the feature extraction network to obtain a vehicle apparent feature model;
and extracting global apparent features and local apparent features of the vehicle to be queried through the vehicle apparent feature model.
Optionally, the obtaining the apparent characteristics of the vehicle to be queried further includes:
calculating the similarity between the apparent features of the vehicles to be queried to obtain a second similarity sequence of the apparent features;
determining a first filtering threshold according to the second similarity sequence;
and filtering the apparent characteristics of the vehicle to be queried according to the first filtering threshold.
Optionally, the obtaining the license plate feature of the license plate to be queried includes:
acquiring a vehicle image set, classifying images belonging to the same vehicle in the vehicle image set into the same category, and obtaining a training set;
establishing a license plate verification network;
and inputting the training set into the license plate verification network, and extracting to obtain license plate features.
Optionally, the obtaining the license plate feature of the license plate to be queried further includes:
calculating the similarity between license plate features of the vehicle to be queried to obtain a third similarity sequence of the license plate features;
determining a second filtering threshold according to the third similarity sequence and the second similarity sequence;
and filtering the license plate features according to the second filtering threshold.
Optionally, the calculating the space-time distance between the vehicle to be queried and the standard image in the vehicle image database according to the principle from the near to the far comprises:
acquiring the image of the vehicle to be queried and the timestamp information of a standard image in a vehicle image database;
acquiring the maximum time difference between the image of the vehicle to be queried and the standard image;
acquiring the shortest distance between cameras corresponding to the images of the vehicle to be queried;
obtaining the maximum distance between all cameras;
and determining the space-time distance according to the timestamp information, the maximum time difference, the shortest distance and the maximum distance.
Optionally, the calculation formula of the space-time distance is:
wherein T is i And T j Time stamps respectively representing an image i of the vehicle to be queried and a standard image j in a vehicle image database, T max Representing the maximum time difference between the images of all vehicles to be queried and the standard images in the vehicle image database; delta (C) i ,C j ) Is a camera C for shooting an image i of the vehicle to be queried i With a camera C which shoots an image j of the vehicle to be queried j Shortest distance between D max Is the maximum distance between all cameras.
The implementation of the method of the present application will be described in detail below with reference to the accompanying drawings.
The embodiment of the application provides a gradual graph searching method, which particularly adopts a rough-to-fine mode to gradually improve the graph searching effect. The method comprises the steps of firstly carrying out rough-to-fine search in a characteristic domain, namely firstly extracting apparent characteristics (colors, semantic attributes and the like) of a vehicle to carry out rough screening, then carrying out accurate matching by using vehicle license plate information, secondly calculating space-time distances according to a principle from near to far, and finally carrying out reordering by utilizing weighted sum of the apparent characteristic similarity, the license plate characteristic similarity and the space-time distances. The overall scheme of the progressive graphic vehicle searching technology is shown in fig. 1, and the specific implementation mode is as follows:
referring to fig. 2, S1: and extracting, sequencing and filtering apparent features of the vehicle.
S11, collecting vehicle images, classifying the images of the same vehicle into one type, and establishing a training set;
s12, designing a vehicle apparent feature extraction network structure by adopting a convolutional neural network;
s13, inputting the training set sample established in the S11 into a feature extraction network in the S12 to obtain a vehicle apparent feature model so as to extract global apparent features and local apparent features of a vehicle image;
s14, calculating the apparent feature similarity of the vehicle by adopting the method including but not limited to cosine distance and arranging the apparent feature similarity;
the cosine distance is as follows:
wherein, I I.II I is L-2 norm of the vector, X and Y are feature vectors of the query image and the search image respectively, and theta is an included angle between the two feature vectors.
S15, adaptively determining a filtering threshold value by utilizing the apparent feature similarity and the size of the search database;
the adaptive filtering threshold is obtained by the following formula, where n t For the number of test set samples; n is n g To retrieve the number of samples in the library; t is t max Is constant, represents a maximum threshold value, and is specified empirically; t is t test Constant, optimal threshold for measurement in test set or empirically specified; t is an adaptive threshold.
S16, vehicles with very dissimilar apparent features are filtered according to the filtering threshold, and vehicle image data with similar apparent features and apparent feature similarity are stored.
Referring to fig. 3, S2: license plate feature extraction, sequencing and filtering.
S21, acquiring license plate images, classifying license plates of the same vehicle into one type, and establishing a training set;
s22, designing a license plate verification network structure by adopting a license plate verification method based on a dual neural network (Siamese neural network, SNN);
s23, inputting the training set sample established in the S21 into a license plate verification network established in the S22 to obtain a license plate verification model so as to extract license plate features;
s24, calculating the similarity of license plate features by adopting a cosine distance and arranging the similarity;
the cosine distance is as follows:
wherein, I I.II I is L-2 norm of the vector, X and Y are feature vectors of the query image and the search image respectively, and theta is an included angle between the two feature vectors.
S25, self-adaptively determining a filtering threshold value by utilizing the weighting of the license plate feature similarity and the apparent feature similarity and the size of the search data set;
the weighted score of the license plate feature similarity and the apparent feature similarity is obtained by s plate Is the feature similarity of license plates, s apparence For apparent feature similarity, ω is a weighted weight, and the value of ω is obtained according to the performance of the present solution on the self-built test set.
score＝ω×s plate +(1-ω)×s apparence ,(0≤ω≤1.0)
The adaptive filtering threshold is obtained by the following formula, where n t For the number of test set samples; n is n g To retrieve the number of samples in the library; t is t max Is constant, represents a maximum threshold value, and is specified empirically; t is t test Constant, optimal threshold for measurement in test set or empirically specified; t is an adaptive threshold.
S26, filtering vehicles with weighted sums of license plate features and apparent feature similarity smaller than the threshold according to the filtering threshold, and storing vehicle image data, the apparent feature similarity and the license plate feature similarity.
S3: and calculating the space-time distance.
S31, calculating the space-time distance according to the principle from near to far, wherein the specific calculation method comprises the following steps:
wherein T is i And T j Time stamps, T, representing query image i and database image j, respectively max Representing the maximum time difference between all query images and the database image. Delta (C) i ,C j ) Is a camera C i And camera C j Shortest distance between D max Is the maximum distance between all cameras.
S4: and calculating and sequencing the overall similarity.
S41, determining the overall similarity according to the apparent feature similarity, license plate feature similarity and the weighting of the time space distance;
s42, reordering by utilizing the overall similarity;
the application realizes the continuous optimization process of the similarity sequencing, can better distinguish vehicles with the same colors and types under the same angle, and can better give the same vehicle higher similarity under different angles.
Another aspect of an embodiment of the present application provides an image-based vehicle retrieval apparatus, including:
the first acquisition module is used for acquiring apparent characteristics of the vehicle to be queried;
the second acquisition module is used for acquiring license plate characteristics of the license plate to be inquired;
the first calculation module is used for calculating the space-time distance between the vehicle to be queried and the standard image in the vehicle image database according to the principle from the near to the far;
the second calculation module is used for calculating the similarity between the image to be queried and each standard image in the vehicle image database according to the apparent characteristics, the license plate characteristics and the space-time distance to obtain a first similarity sequence;
and the determining module is used for determining a standard image with highest similarity according to the first similarity sequence to obtain a retrieval result of the vehicle to be queried.
Another aspect of an embodiment of the present application provides an electronic device, including a processor and a memory;
the memory is used for storing programs;
the processor executes the program to implement the method as described above.
Another aspect of the embodiments of the present application provides a computer-readable storage medium storing a program that is executed by a processor to implement a method as described above.
Embodiments of the present application also disclose a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions may be read from a computer-readable storage medium by a processor of a computer device, and executed by the processor, to cause the computer device to perform the foregoing method.
In some alternative embodiments, the functions/acts noted in the block diagrams may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Furthermore, the embodiments presented and described in the flowcharts of the present application are provided by way of example in order to provide a more thorough understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed, and in which sub-operations described as part of a larger operation are performed independently.
Furthermore, while the application is described in the context of functional modules, it should be appreciated that, unless otherwise indicated, one or more of the described functions and/or features may be integrated in a single physical device and/or software module or one or more functions and/or features may be implemented in separate physical devices or software modules. It will also be appreciated that a detailed discussion of the actual implementation of each module is not necessary to an understanding of the present application. Rather, the actual implementation of the various functional modules in the apparatus disclosed herein will be apparent to those skilled in the art from consideration of their attributes, functions and internal relationships. Accordingly, one of ordinary skill in the art can implement the application as set forth in the claims without undue experimentation. It is also to be understood that the specific concepts disclosed are merely illustrative and are not intended to be limiting upon the scope of the application, which is to be defined in the appended claims and their full scope of equivalents.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the spirit and principles of the application, the scope of which is defined by the claims and their equivalents.
While the preferred embodiment of the present application has been described in detail, the present application is not limited to the embodiments described above, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present application, and these equivalent modifications or substitutions are included in the scope of the present application as defined in the appended claims.
Claims (8)
1. An image-based vehicle retrieval method, comprising:
acquiring apparent characteristics of a vehicle to be queried;
acquiring license plate characteristics of a license plate to be queried;
calculating the space-time distance between the vehicle to be queried and the standard image in the vehicle image database according to the principle from the near to the far;
according to the apparent features, the license plate features and the space-time distance, calculating the similarity between the image to be queried and each standard image in a vehicle image database to obtain a first similarity sequence;
determining a standard image with highest similarity according to the first similarity sequence to obtain a retrieval result of the vehicle to be queried;
the calculating the space-time distance between the vehicle to be queried and the standard image in the vehicle image database according to the principle from the near to the far comprises the following steps:
acquiring the image of the vehicle to be queried and the timestamp information of a standard image in a vehicle image database;
acquiring the maximum time difference between the image of the vehicle to be queried and the standard image;
acquiring the shortest distance between cameras corresponding to the images of the vehicle to be queried;
obtaining the maximum distance between all cameras;
determining the space-time distance according to the timestamp information, the maximum time difference, the shortest distance and the maximum distance;
the calculation formula of the space-time distance is as follows:
wherein T is i And T j Time stamps respectively representing an image i of the vehicle to be queried and a standard image j in a vehicle image database, T max Representing the maximum time difference between the images of all vehicles to be queried and the standard images in the vehicle image database; delta (C) i ,C j ) Is a camera C for shooting an image i of the vehicle to be queried i With a camera C which shoots an image j of the vehicle to be queried j Shortest distance between D max Is the maximum distance between all cameras.
2. The image-based vehicle retrieval method according to claim 1, wherein the acquiring the apparent features of the vehicle to be queried includes:
acquiring a vehicle image set, classifying images belonging to the same vehicle in the vehicle image set into the same category, and obtaining a training set;
establishing a feature extraction network of apparent features of the vehicle;
inputting the training set into the feature extraction network to obtain a vehicle apparent feature model;
and extracting global apparent features and local apparent features of the vehicle to be queried through the vehicle apparent feature model.
3. The image-based vehicle retrieval method according to claim 2, wherein the acquiring the apparent feature of the vehicle to be queried further comprises:
calculating the similarity between the apparent features of the vehicles to be queried to obtain a second similarity sequence of the apparent features;
determining a first filtering threshold according to the second similarity sequence;
and filtering the apparent characteristics of the vehicle to be queried according to the first filtering threshold.
4. The method for retrieving a vehicle based on images as claimed in claim 3, wherein said obtaining license plate features of a license plate to be queried comprises:
acquiring a vehicle image set, classifying images belonging to the same vehicle in the vehicle image set into the same category, and obtaining a training set;
establishing a license plate verification network;
and inputting the training set into the license plate verification network, and extracting to obtain license plate features.
5. The method for retrieving a vehicle based on images as claimed in claim 4, wherein said obtaining license plate features of a license plate to be queried further comprises:
calculating the similarity between license plate features of the vehicle to be queried to obtain a third similarity sequence of the license plate features;
determining a second filtering threshold according to the third similarity sequence and the second similarity sequence;
and filtering the license plate features according to the second filtering threshold.
6. An image-based vehicle search device, comprising:
the first acquisition module is used for acquiring apparent characteristics of the vehicle to be queried;
the second acquisition module is used for acquiring license plate characteristics of the license plate to be inquired;
the first calculation module is used for calculating the space-time distance between the vehicle to be queried and the standard image in the vehicle image database according to the principle from the near to the far;
the second calculation module is used for calculating the similarity between the image to be queried and each standard image in the vehicle image database according to the apparent characteristics, the license plate characteristics and the space-time distance to obtain a first similarity sequence;
the determining module is used for determining a standard image with highest similarity according to the first similarity sequence to obtain a retrieval result of the vehicle to be queried;
the first computing module is specifically configured to:
acquiring the image of the vehicle to be queried and the timestamp information of a standard image in a vehicle image database;
acquiring the maximum time difference between the image of the vehicle to be queried and the standard image;
acquiring the shortest distance between cameras corresponding to the images of the vehicle to be queried;
obtaining the maximum distance between all cameras;
determining the space-time distance according to the timestamp information, the maximum time difference, the shortest distance and the maximum distance;
the calculation formula of the space-time distance is as follows:
wherein T is i And T j Time stamps respectively representing an image i of the vehicle to be queried and a standard image j in a vehicle image database, T max Representing the maximum time difference between the images of all vehicles to be queried and the standard images in the vehicle image database; delta (C) i ,C j ) Is a camera C for shooting an image i of the vehicle to be queried i With a camera C which shoots an image j of the vehicle to be queried j Shortest distance between D max Is the maximum distance between all cameras.
7. An electronic device comprising a processor and a memory;
the memory is used for storing programs;
the processor executing the program to implement the method of any one of claims 1-5.
8. A computer readable storage medium, characterized in that the storage medium stores a program, which is executed by a processor to implement the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110913283.9A CN113806576B (en) | 2021-08-10 | 2021-08-10 | Image-based vehicle retrieval method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110913283.9A CN113806576B (en) | 2021-08-10 | 2021-08-10 | Image-based vehicle retrieval method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113806576A CN113806576A (en) | 2021-12-17 |
| CN113806576B true CN113806576B (en) | 2023-08-18 |
Family
ID=78943010
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110913283.9A Active CN113806576B (en) | 2021-08-10 | 2021-08-10 | Image-based vehicle retrieval method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113806576B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106599905A (en) * | 2016-11-25 | 2017-04-26 | 杭州中奥科技有限公司 | Fake-licensed vehicle analysis method based on deep learning |
| CN108229468A (en) * | 2017-06-28 | 2018-06-29 | 北京市商汤科技开发有限公司 | Vehicle appearance feature recognition and vehicle retrieval method, apparatus, storage medium, electronic equipment |
| CN109740479A (en) * | 2018-12-25 | 2019-05-10 | 苏州科达科技股份有限公司 | A kind of vehicle recognition methods, device, equipment and readable storage medium storing program for executing again |
| CN109740653A (en) * | 2018-12-25 | 2019-05-10 | 北京航空航天大学 | A kind of vehicle recognition methods again for merging visual appearance and space-time restriction |
| CN110874934A (en) * | 2018-08-29 | 2020-03-10 | 北京万集科技股份有限公司 | License plate recognition method, system and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10423982B2 (en) * | 2014-05-19 | 2019-09-24 | Allstate Insurance Company | Content output systems using vehicle-based data |
| CN109686126B (en) * | 2019-01-22 | 2021-04-27 | 京东方科技集团股份有限公司 | Road screen networking system and vehicle-mounted unit |
-
2021
- 2021-08-10 CN CN202110913283.9A patent/CN113806576B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106599905A (en) * | 2016-11-25 | 2017-04-26 | 杭州中奥科技有限公司 | Fake-licensed vehicle analysis method based on deep learning |
| CN108229468A (en) * | 2017-06-28 | 2018-06-29 | 北京市商汤科技开发有限公司 | Vehicle appearance feature recognition and vehicle retrieval method, apparatus, storage medium, electronic equipment |
| CN110874934A (en) * | 2018-08-29 | 2020-03-10 | 北京万集科技股份有限公司 | License plate recognition method, system and device |
| CN109740479A (en) * | 2018-12-25 | 2019-05-10 | 苏州科达科技股份有限公司 | A kind of vehicle recognition methods, device, equipment and readable storage medium storing program for executing again |
| CN109740653A (en) * | 2018-12-25 | 2019-05-10 | 北京航空航天大学 | A kind of vehicle recognition methods again for merging visual appearance and space-time restriction |
Non-Patent Citations (1)
| Title |
|---|
| 面向智能车的前方车辆识别技术研究及视觉感知系统设计;马要娟;《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》(第12期);C035-111 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113806576A (en) | 2021-12-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Garcia-Fidalgo et al. | Hierarchical place recognition for topological mapping | |
| Sivic et al. | Video Google: Efficient visual search of videos | |
| US8892542B2 (en) | Contextual weighting and efficient re-ranking for vocabulary tree based image retrieval | |
| CN110209866A (en) | A kind of image search method, device, equipment and computer readable storage medium | |
| Derpanis et al. | Classification of traffic video based on a spatiotemporal orientation analysis | |
| CN113177518B (en) | Vehicle re-identification method based on weak supervision area recommendation | |
| CN111104867A (en) | Recognition model training and vehicle heavy recognition method and device based on component segmentation | |
| Inoue et al. | TokyoTechCanon at TRECVID 2012. | |
| Naeem et al. | Deep learned vectors’ formation using auto-correlation, scaling, and derivations with CNN for complex and huge image retrieval | |
| Li et al. | 3D-DETNet: a single stage video-based vehicle detector | |
| CN110826415A (en) | Method and device for re-identifying vehicles in scene image | |
| Bashir et al. | DUPL-VR: Deep unsupervised progressive learning for vehicle re-identification | |
| Yang et al. | An object detection and tracking system for unmanned surface vehicles | |
| Le et al. | Document retrieval based on logo spotting using key-point matching | |
| CN103136292B (en) | A kind of image search method based on characteristics of image and device | |
| Kozlov et al. | Development of real-time ADAS object detector for deployment on CPU | |
| CN113806576B (en) | Image-based vehicle retrieval method and device, electronic equipment and storage medium | |
| Farkaš | Object tracking and detection with YOLOv8 and StrongSORT algorithms captured by drone | |
| Li et al. | Evaluation of Global Descriptor Methods for Appearance‐Based Visual Place Recognition | |
| Nguyen et al. | Video instance search via spatial fusion of visual words and object proposals | |
| Zhou et al. | Attention-aware network and multi-loss joint training method for vehicle re-identification | |
| CN114387571A (en) | Unsupervised vehicle re-identification method and unsupervised vehicle re-identification device based on hierarchical matching | |
| Ali et al. | Recognition of human action and identification based on SIFT and watermark | |
| Konlambigue et al. | Performance evaluation of state-of-the-art filtering criteria applied to sift features | |
| Kovbasiuk et al. | Detection of vehicles on images obtained from unmanned aerial vehicles using instance segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |