CN112990152B - Vehicle weight identification method based on key point detection and local feature alignment - Google Patents
Vehicle weight identification method based on key point detection and local feature alignment Download PDFInfo
- Publication number
- CN112990152B CN112990152B CN202110504848.8A CN202110504848A CN112990152B CN 112990152 B CN112990152 B CN 112990152B CN 202110504848 A CN202110504848 A CN 202110504848A CN 112990152 B CN112990152 B CN 112990152B
- Authority
- CN
- China
- Prior art keywords
- image
- vehicle
- local
- features
- identified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 79
- 238000001514 detection method Methods 0.000 title claims abstract description 33
- 238000012163 sequencing technique Methods 0.000 claims abstract description 10
- 238000000605 extraction Methods 0.000 claims description 23
- 239000000126 substance Substances 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 8
- 238000013135 deep learning Methods 0.000 claims description 6
- 230000036961 partial effect Effects 0.000 claims description 6
- 230000036544 posture Effects 0.000 claims description 6
- 210000000988 bone and bone Anatomy 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 abstract description 8
- 230000008859 change Effects 0.000 abstract description 3
- 238000003909 pattern recognition Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 238000004590 computer program Methods 0.000 description 7
- 239000013598 vector Substances 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 230000003044 adaptive effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 239000000284 extract Substances 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 230000000873 masking effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 241000287196 Asthenes Species 0.000 description 1
- 230000003042 antagnostic effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Abstract
The invention belongs to the field of computer vision and pattern recognition, and particularly relates to a vehicle weight recognition method based on key point detection and local feature alignment, aiming at solving the problem that the existing vehicle weight recognition method has poor feature consistency when the visual angle change of a vehicle is eliminated, and further causes poor robustness of vehicle weight recognition. The method comprises the steps of obtaining a vehicle image to be identified as an input image; performing key point detection on an input image, acquiring key points of a vehicle to be identified and a corresponding confidence coefficient of the key points, and dividing the vehicle to be identified in the input image into N parts as local images; extracting the characteristics of the input image and each local image to be used as global characteristics and local characteristics, and splicing each local characteristic and the global characteristics to be used as comprehensive characteristics; and calculating the distance between the comprehensive features and the corresponding features of the images in the vehicle image library, sequencing the distances, and outputting the sequencing result as a re-identification result. The invention improves the robustness of vehicle weight identification.
Description
Technical Field
The invention belongs to the field of computer vision and pattern recognition, and particularly relates to a vehicle weight recognition method, system and device based on key point detection and local feature alignment.
Background
Vehicle weight recognition is a sub-problem in the field of image retrieval. Given a query vehicle image, the vehicle re-identification task aims to find images of the same vehicle in other scenes. The essence of the task is to learn a vehicle feature vector generation method, which is used for representing unique features of one vehicle, is robust to various external changes (such as visual angle, shielding, light rays and license plate replacement) of the same vehicle, and can have sufficient discriminability for two vehicles with similar appearances. The nature of the changes of occlusion, view angle and the like is to cause the misalignment of the same semantic area in different images of the same vehicle in the spatial position, for example, the semantic components between the head view image and the tail view image of the vehicle are misaligned in the pixel space. A good vehicle feature vector needs to break through the misalignment between images, and the alignment of target semantic components is realized on a feature level, so that the robustness of variables such as visual angles, shelters and the like is realized.
The existing vehicle weight identification methods for aligning vehicle parts are roughly classified into two types: and (3) completing invisible parts in the pictures based on a method for generating a network by confrontation, and finding out a common area in the two pictures based on the idea of semantic segmentation. Among them, the first method is represented by VAMI (visual-aware Multi-view introduction for Vehicle Re-identification). Given a vehicle image at any perspective, the VAMI will extract single-view features for each input image and aim to convert the features into a global multi-view feature representation so that pairwise distance metric learning can be better optimized in this perspective-invariant feature space. The VAMI adopts a visual angle perception attention model to select core areas of different viewpoints, and realizes effective multi-visual angle feature inference through an antagonistic training framework. However, the method has obvious disadvantages, and it is obviously not robust to generate information of other views by only using information under one view. The second method, for example SPAN (organization-aware Vehicle Re-identification with continuous-defined Part-Orientation Network), which uses an unsupervised method to generate masks for individual parts of the Vehicle, extracts discriminative features in the individual regions with the aid of these partial masks, and then, when comparing the images, emphatically compares the shared regions. However, masks generated by the unsupervised method are often unreliable, and can cause a lot of influences on subsequent feature extraction. Based on the method, the invention provides a vehicle weight identification method based on key point detection and local feature alignment.
Disclosure of Invention
In order to solve the above problems in the prior art, that is, to solve the problem that the existing vehicle heavy identification method has poor feature consistency when the vehicle view angle change is eliminated, and thus the vehicle heavy identification robustness is poor, a first aspect of the present invention provides a vehicle heavy identification method based on key point detection and local feature alignment, the method including:
s10, acquiring a vehicle image to be recognized as an input image;
s20, performing key point detection on the input image, and acquiring key points of the vehicle to be identified and corresponding confidence coefficients of the key points; dividing the vehicles to be identified in the input image into N parts as local images according to the detected key points and the corresponding confidence coefficients thereof; n is a natural number;
s30, enlarging and reducing the local image by a preset magnification, and merging the local image with the original local image to obtain a preprocessed local image; inputting the input image and each preprocessed local image into a pre-constructed vehicle weight recognition model, performing convolution processing on each preprocessed image, and extracting the characteristics of the input image and each preprocessed local image after the convolution processing through a characteristic extraction layer of the pre-constructed vehicle weight recognition model to be used as global characteristics and local characteristics; after extraction, splicing each local feature and the global feature to serve as a comprehensive feature corresponding to the vehicle to be identified;
s40, combining the local features and the global features of the vehicle to be recognized, calculating the distance between the comprehensive features and the corresponding features of the images in the vehicle image library through a preset self-adaptive region weighted alignment method, sequencing the distances, and outputting the sequencing result as a re-recognition result;
the vehicle re-identification model is constructed on the basis of a one-dimensional convolutional layer and a deep learning network.
In some preferred embodiments, the method of "dividing the vehicle to be recognized in the input image into N parts" is:
acquiring the posture of the vehicle based on the detected key points and the corresponding confidence degrees thereof, and judging whether the set vehicle semantic region is visible or not;
if the set vehicle semantic area is visible, key points of each vehicle semantic area are connected by combining the postures of the vehicles, and the vehicles to be recognized in the input images are divided into four parts, namely a roof, a side, a head and a parking space.
In some preferred embodiments, the distance between the comprehensive feature and the corresponding feature of each image in the vehicle image library is calculated by a preset adaptive region weighted alignment method, which includes:
wherein the content of the first and second substances,
representing integrated characteristics of a vehicle to be identified
Corresponding comprehensive characteristics to each image in vehicle image library
Distance of euc: (
) The expression of the euclidean distance,
、
respectively, of the vehicle to be identified
The local features and the first image corresponding to each image in the vehicle image library
The local characteristics of the image are measured,
、
representing an image of a vehicle to be recognized
Each local image and each image in the vehicle image library correspond to the first image
The degree of saliency of the individual partial images,
。
in some preferred embodiments, the method for calculating the saliency of the local image is as follows:
for each local image, obtaining the confidence coefficient of the corresponding key point and the number of pixel points of the part which is not covered by the mask, and calculating the corresponding significance of each local image by combining the total number of pixel points of the input image; the method specifically comprises the following steps:
wherein the content of the first and second substances,
representing local image correspondences
The confidence level of each of the key points,
the number of pixel points representing the portion not masked by the mask,
representing the total number of pixel points of the input image.
In some preferred embodiments, the vehicle weight recognition model, whose loss function in training is:
wherein the content of the first and second substances,
a loss value representing a vehicle weight recognition model,
representing the number of training sample images of a batch of the vehicle heavy identification model during training,
the number of the batches is represented by,
any image in a batch of training sample images representing the vehicle re-identification model during training,
representing image features in image set A
One training sample image with the largest euclidean distance of the features of (a),
representing image features in image set B
One training sample image with the smallest euclidean distance of the features of (a),
indicating a preset distance interval between the first and second electrodes,
representation comprises and
a set of images of all images of the same ID,
indicating the current batch except
All images except the image contained in (1) construct an image set,
representing the euclidean distance.
In some of the preferred embodiments of the present invention,
is 0.1.
In a second aspect of the present invention, a vehicle weight recognition system based on key point detection and local feature alignment is provided, the system comprising: the device comprises an image acquisition module, an image division module, a feature extraction module and a re-identification module;
the image acquisition module is configured to acquire a vehicle image to be identified as an input image;
the image dividing module is configured to detect key points of the input image, and acquire the key points of the vehicle to be identified and the corresponding confidence coefficients of the key points; dividing the vehicles to be identified in the input image into N parts as local images according to the detected key points and the corresponding confidence coefficients thereof; n is a natural number;
the characteristic extraction module is configured to enlarge and reduce the local image by a set time, and then combine the local image with the original local image to be used as a preprocessed local image; inputting the input image and each preprocessed local image into a pre-constructed vehicle weight recognition model, performing convolution processing on each preprocessed image, and extracting the characteristics of the input image and each preprocessed local image after the convolution processing through a characteristic extraction layer of the pre-constructed vehicle weight recognition model to be used as global characteristics and local characteristics; after extraction, splicing each local feature and the global feature to serve as a comprehensive feature corresponding to the vehicle to be identified;
the re-recognition module is configured to calculate and sort the distances between the comprehensive features and the corresponding features of the images in the vehicle image library by combining the local features and the global features of the vehicle to be recognized through a preset self-adaptive region weighted alignment method, and output a sorting result as a re-recognition result;
the vehicle re-identification model is constructed on the basis of a one-dimensional convolutional layer and a deep learning network.
In a third aspect of the invention, an apparatus is presented, at least one processor; and a memory communicatively coupled to at least one of the processors; wherein the memory stores instructions executable by the processor for execution by the processor to implement the method for vehicle weight identification based on keypoint detection and local feature alignment of claims above.
In a fourth aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for being executed by the computer to implement the method for recognizing vehicle weight based on keypoint detection and local feature alignment as claimed above.
The invention has the beneficial effects that:
the method and the device eliminate the influence of the background on the foreground and align the features, thereby eliminating the influence of the change of the visual angle on the extraction of the vehicle features and improving the robustness of vehicle weight identification.
According to the method, the vehicle in the picture is divided into component areas with different semantics through the key point detection model, so that local features with discrimination are obtained, and meanwhile, the influence of the background on the re-identification is eliminated. A method for matching regional features of corresponding semantic components between two vehicle pictures is designed, and is used for eliminating visual region differences caused by visual angle transformation, so that robustness of characteristic vectors and a re-recognition model on visual angle changes and background interference is enhanced.
The invention obtains better component matching effect by carrying out scaling processing on the local input picture and learning the relation among different scales through one-dimensional convolution.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings.
FIG. 1 is a schematic flow chart diagram of a vehicle re-identification method based on keypoint detection and local feature alignment according to an embodiment of the present invention;
FIG. 2 is a block diagram of a vehicle weight recognition system based on keypoint detection and local feature alignment according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a training process of a vehicle re-identification model according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating a method for adaptive area weighted alignment according to an embodiment of the present invention;
FIG. 5 is a flow chart illustrating local weight calculation during feature alignment according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
The invention discloses a vehicle weight identification method based on key point detection and local feature alignment, which comprises the following steps of:
s10, acquiring a vehicle image to be recognized as an input image;
s20, performing key point detection on the input image, and acquiring key points of the vehicle to be identified and corresponding confidence coefficients of the key points; dividing the vehicles to be identified in the input image into N parts as local images according to the detected key points and the corresponding confidence coefficients thereof; n is a natural number;
s30, enlarging and reducing the local image by a preset magnification, and merging the local image with the original local image to obtain a preprocessed local image; inputting the input image and each preprocessed local image into a pre-constructed vehicle weight recognition model, performing convolution processing on each preprocessed image, and extracting the characteristics of the input image and each preprocessed local image after the convolution processing through a characteristic extraction layer of the pre-constructed vehicle weight recognition model to be used as global characteristics and local characteristics; after extraction, splicing each local feature and the global feature to serve as a comprehensive feature corresponding to the vehicle to be identified;
s40, combining the local features and the global features of the vehicle to be recognized, calculating the distance between the comprehensive features and the corresponding features of the images in the vehicle image library through a preset self-adaptive region weighted alignment method, sequencing the distances, and outputting the sequencing result as a re-recognition result;
the vehicle re-identification model is constructed on the basis of a one-dimensional convolutional layer and a deep learning network.
In order to more clearly describe the method for recognizing vehicle weight based on key point detection and local feature alignment according to the present invention, the following will describe each step in an embodiment of the method according to the present invention in detail with reference to the accompanying drawings.
S10, acquiring a vehicle image to be recognized as an input image;
in this embodiment, an image of a vehicle to be recognized is acquired first.
S20, performing key point detection on the input image, and acquiring key points of the vehicle to be identified and corresponding confidence coefficients of the key points; dividing the vehicles to be identified in the input image into N parts as local images according to the detected key points and the corresponding confidence coefficients thereof; n is a natural number;
in the embodiment, the pre-trained vehicle key point detection model is used for extracting key points in the vehicle to be recognized and confidence degrees corresponding to the key points, judging the vehicle posture to divide a visible region according to the confidence degrees, and inputting an input image and four local images into the vehicle re-recognition model. Among these, the definition of the keypoints can be referred to in the references "Wang Z, Tang L, Liu X, et al. organization exploration Embedding and Spatial Temporal regulation for Vehicle Re-identification [ C ]// 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017.
As shown in FIG. 3, the key point detection result of the input image is that the vehicle key point detection model will detect 16 key points on the vehicle body and distribute the key points on the vehicleAccording to the confidence corresponding to the key points, the posture of the vehicle and whether the set semantic region of the vehicle is visible or not can be deduced on the outline of the vehicle and the important parts such as wheels, lamps and the like. According to the connecting lines among the key points, the whole vehicle is divided into a vehicle head, a vehicle tail, a vehicle roof and a vehicle side, other areas are covered in the pictures (namely local images) corresponding to the areas respectively in a masking mode, and for invisible areas such as the vehicle tail at a front view angle, the masking covers the whole vehicle image to be identified. Each partial image is recorded as
，
，
，
，
Representing the image of the vehicle to be identified.
S30, enlarging and reducing the local image by a preset magnification, and merging the local image with the original local image to obtain a preprocessed local image; inputting the input image and each preprocessed local image into a pre-constructed vehicle weight recognition model, performing convolution processing on each preprocessed image, and extracting the characteristics of the input image and each preprocessed local image after the convolution processing through a characteristic extraction layer of the pre-constructed vehicle weight recognition model to be used as global characteristics and local characteristics; after extraction, splicing each local feature and the global feature to serve as a comprehensive feature corresponding to the vehicle to be identified;
in the present embodiment, the local image is enlarged and reduced by a predetermined magnification (in the present invention, the magnification is preferably set to
Multiple) is merged with the original local image as a preprocessed local image. And inputting the input image and each preprocessed local image into a pre-constructed vehicle weight recognition model.
The global feature is extracted through a complete ResNet50, the local features are respectively extracted through ResNet18 with four parameters not shared, a one-dimensional convolution layer is added at the head of the model, and four local feature vectors and a global feature vector are obtained after pooling and are respectively expressed as
，
，
，
，
Wherein, the dimension of the local feature is 256 dimensions, the dimension of the global feature is 2048 dimensions, and the above features are fused in a splicing mode to obtain a 3072-dimensional feature
And the complete vehicle feature vector is used as the comprehensive feature corresponding to the vehicle to be identified. Performing convolution processing on each preprocessed image through a one-dimensional convolution layer, and extracting the characteristics of the input image and each preprocessed local image after the convolution processing through a pre-constructed feature extraction layer of the vehicle weight recognition model to be used as global characteristics and local characteristics; and splicing each local feature and the global feature to be used as a comprehensive feature corresponding to the vehicle to be identified.
And S40, combining the local features and the global features of the vehicle to be recognized, calculating the distance between the comprehensive features and the corresponding features of the images in the vehicle image library through a preset self-adaptive region weighted alignment method, sequencing the distances, and outputting the sequencing result as a re-recognition result.
In this embodiment, a self-adaptive region alignment method is designed, and when the similarity degree of two pictures is calculated in the inference stage, the weights occupied by four local features are automatically adjusted, as shown in fig. 4 and 5. The method specifically comprises the following steps: the significance of a corresponding region in a picture is obtained based on an output result of key point detection, namely the confidence of each key point and the number of pixel points of an uncovered part of a mask, wherein each local region is determined by four key points, and the confidences of the four key points are respectively
And the number of pixels in the region is recorded as
The total number of pixels in the original image is recorded as
Then the saliency of the region is defined as:
the distance of F1 and F2 calculated during the inference phase can be defined as:
wherein the content of the first and second substances,
representing integrated characteristics of a vehicle to be identified
Corresponding comprehensive characteristics to each image in vehicle image library
Distance of euc: (
) The expression of the euclidean distance,
、
respectively, of the vehicle to be identified
The local features and the first image corresponding to each image in the vehicle image library
The local characteristics of the image are measured,
、
representing an image of a vehicle to be recognized
Each local image and each image in the vehicle image library correspond to the first image
The degree of saliency of the individual partial images,
. Therefore, the method can automatically measure the weight of the local features in distance calculation, and for the areas which do not appear in the two pictures at the same time, the corresponding display degree
Is 0, does not participate in calculation, is influenced by normalization and simultaneouslyThe area appearing in both pictures plays a higher role.
In addition, the training process of the vehicle re-identification model is shown in fig. 3, the model is supervised by using triple loss during training, and the core idea of the loss is to separate the unmatched vehicle pairs from the matched vehicle pairs by distance intervals to increase the inter-class difference and reduce the intra-class difference, which is specifically shown in formula (3):
wherein the content of the first and second substances,
a loss value representing a vehicle weight recognition model,
representing the number of training sample images of a batch of the vehicle heavy identification model during training,
the number of the batches is represented by,
any image in a batch of training sample images representing the vehicle re-identification model during training,
representing image features in image set A
One training sample image with the largest euclidean distance of the features of (a),
representing image features in image set B
One training sample image with the smallest euclidean distance of the features of (a),
indicating a preset distance interval, is preferably set to 0.1 in the present invention,
representation comprises and
a set of images of all images of the same ID,
indicating the current batch except
All images except the image contained in (1) construct an image set,
representing the euclidean distance.
A vehicle weight recognition system based on key point detection and local feature alignment according to a second embodiment of the present invention, as shown in fig. 2, specifically includes: the image processing system comprises an image acquisition module 100, an image dividing module 200, a feature extraction module 300 and a re-identification module 400;
the image acquisition module 100 is configured to acquire an image of a vehicle to be identified as an input image;
the image dividing module 200 is configured to perform keypoint detection on the input image, and acquire keypoints of the vehicle to be identified and confidence degrees corresponding to the keypoints; dividing the vehicles to be identified in the input image into N parts as local images according to the detected key points and the corresponding confidence coefficients thereof; n is a natural number;
the feature extraction module 300 is configured to enlarge and reduce the local image by a set magnification, and then merge the local image with the original local image to serve as a preprocessed local image; inputting the input image and each preprocessed local image into a pre-constructed vehicle weight recognition model, performing convolution processing on each preprocessed image, and extracting the characteristics of the input image and each preprocessed local image after the convolution processing through a characteristic extraction layer of the pre-constructed vehicle weight recognition model to be used as global characteristics and local characteristics; after extraction, splicing each local feature and the global feature to serve as a comprehensive feature corresponding to the vehicle to be identified;
the re-recognition module 400 is configured to calculate and sort distances between the comprehensive features and the features corresponding to the images in the vehicle image library by a preset adaptive region weighted alignment method in combination with the local features and the global features of the vehicle to be recognized, and output a sorting result as a re-recognition result;
the vehicle re-identification model is constructed on the basis of a one-dimensional convolutional layer and a deep learning network.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the system described above may refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.
It should be noted that, the vehicle weight recognition system based on the key point detection and the local feature alignment provided in the foregoing embodiment is only illustrated by dividing the functional modules, and in practical applications, the above functions may be allocated to different functional modules according to needs, that is, the modules or steps in the embodiments of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiments may be combined into one module, or may be further split into multiple sub-modules, so as to complete all or part of the above described functions. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.
An apparatus of a third embodiment of the invention, at least one processor; and a memory communicatively coupled to at least one of the processors; wherein the memory stores instructions executable by the processor for execution by the processor to implement the method for vehicle weight identification based on keypoint detection and local feature alignment of claims above.
A computer-readable storage medium of a fourth embodiment of the present invention stores computer instructions for execution by the computer to implement the method for vehicle weight identification based on keypoint detection and local feature alignment of the claims above.
It can be clearly understood by those skilled in the art that, for convenience and brevity, the specific working processes and related descriptions of the storage device and the processing device described above may refer to the corresponding processes in the foregoing method examples, and are not described herein again.
Referring now to FIG. 6, there is illustrated a block diagram of a computer system suitable for use as a server in implementing embodiments of the method, system, and apparatus of the present application. The server shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 6, the computer system includes a Central Processing Unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for system operation are also stored. The CPU601, ROM 602, and RAM603 are connected to each other via a bus 604. An Input/Output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output section 607 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a Network interface card such as a LAN (Local Area Network) card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), a compact disc read-only memory (CD-ROM), Optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (7)
1. A vehicle weight identification method based on key point detection and local feature alignment is characterized by comprising the following steps:
s10, acquiring a vehicle image to be recognized as an input image;
s20, performing key point detection on the input image, and acquiring key points of the vehicle to be identified and corresponding confidence coefficients of the key points; dividing the vehicles to be identified in the input image into N parts as local images according to the detected key points and the corresponding confidence coefficients thereof; n is a natural number;
s30, enlarging and reducing the local image by a preset magnification, and merging the local image with the original local image to obtain a preprocessed local image; inputting the input image and each preprocessed local image into a pre-constructed vehicle weight recognition model, performing convolution processing on each preprocessed image, and extracting the characteristics of the input image and each preprocessed local image after the convolution processing through a characteristic extraction layer of the pre-constructed vehicle weight recognition model to be used as global characteristics and local characteristics; after extraction, splicing each local feature and the global feature to serve as a comprehensive feature corresponding to the vehicle to be identified;
s40, combining the local features and the global features of the vehicle to be recognized, calculating the distance between the comprehensive features and the corresponding features of the images in the vehicle image library through a preset self-adaptive region weighted alignment method, sequencing the distances, and outputting the sequencing result as a re-recognition result;
the method comprises the following steps of calculating the distance between the comprehensive characteristic and the corresponding characteristic of each image in the vehicle image library by a preset self-adaptive region weighted alignment method, wherein the method comprises the following steps:
wherein the content of the first and second substances,
representing integrated characteristics of a vehicle to be identified
Corresponding comprehensive characteristics to each image in vehicle image library
Distance of euc: (
) The expression of the euclidean distance,
、
respectively, of the vehicle to be identified
The local features and the first image corresponding to each image in the vehicle image library
The local characteristics of the image are measured,
、
representing an image of a vehicle to be recognized
Each local image and each image in the vehicle image library correspond to the first image
The degree of saliency of the individual partial images,
；
the method for calculating the saliency of the local image comprises the following steps:
for each local image, obtaining the confidence coefficient of the corresponding key point and the number of pixel points of the part which is not covered by the mask, and calculating the corresponding significance of each local image by combining the total number of pixel points of the input image; the method specifically comprises the following steps:
wherein the content of the first and second substances,
representing local image correspondences
The confidence level of each of the key points,
the number of pixel points representing the portion not masked by the mask,
representing a total number of pixel points of the input image;
the vehicle weight recognition model is constructed on the basis of a one-dimensional convolutional layer and a deep learning network.
2. The method of claim 1, wherein the method of dividing the vehicle to be identified in the input image into N parts comprises:
acquiring the posture of the vehicle based on the detected key points and the corresponding confidence degrees thereof, and judging whether the set vehicle semantic region is visible or not;
if the set vehicle semantic area is visible, key points of each vehicle semantic area are connected by combining the postures of the vehicles, and the vehicles to be recognized in the input images are divided into four parts, namely a roof, a side, a head and a parking space.
3. The method of claim 1, wherein the vehicle re-recognition model has a loss function during training as follows:
wherein the content of the first and second substances,
a loss value representing a vehicle weight recognition model,
representing the number of training sample images of a batch of the vehicle heavy identification model during training,
the number of the batches is represented by,
any image in a batch of training sample images representing the vehicle re-identification model during training,
representing image features in image set A
One training sample image with the largest euclidean distance of the features of (a),
representing image features in image set B
One training sample image with the smallest euclidean distance of the features of (a),
indicating a preset distance interval between the first and second electrodes,
representation comprises and
a set of images of all images of the same ID,
indicating the current batch except
All images except the image contained in (1) construct an image set,
representing the euclidean distance.
4. The method of claim 3, wherein the vehicle weight recognition based on keypoint detection and local feature alignment,
is 0.1.
5. A vehicle weight recognition system based on keypoint detection and local feature alignment, the system comprising: the device comprises an image acquisition module, an image division module, a feature extraction module and a re-identification module;
the image acquisition module is configured to acquire a vehicle image to be identified as an input image;
the image dividing module is configured to detect key points of the input image, and acquire the key points of the vehicle to be identified and the corresponding confidence coefficients of the key points; dividing the vehicles to be identified in the input image into N parts as local images according to the detected key points and the corresponding confidence coefficients thereof; n is a natural number;
the characteristic extraction module is configured to enlarge and reduce the local image by a set time, and then combine the local image with the original local image to be used as a preprocessed local image; inputting the input image and each preprocessed local image into a pre-constructed vehicle weight recognition model, performing convolution processing on each preprocessed image, and extracting the characteristics of the input image and each preprocessed local image after the convolution processing through a characteristic extraction layer of the pre-constructed vehicle weight recognition model to be used as global characteristics and local characteristics; after extraction, splicing each local feature and the global feature to serve as a comprehensive feature corresponding to the vehicle to be identified;
the re-recognition module is configured to calculate and sort the distances between the comprehensive features and the corresponding features of the images in the vehicle image library by combining the local features and the global features of the vehicle to be recognized through a preset self-adaptive region weighted alignment method, and output a sorting result as a re-recognition result;
the method comprises the following steps of calculating the distance between the comprehensive characteristic and the corresponding characteristic of each image in the vehicle image library by a preset self-adaptive region weighted alignment method, wherein the method comprises the following steps:
wherein the content of the first and second substances,
representing integrated characteristics of a vehicle to be identified
Corresponding comprehensive characteristics to each image in vehicle image library
Distance of euc: (
) The expression of the euclidean distance,
、
respectively, of the vehicle to be identified
The local features and the first image corresponding to each image in the vehicle image library
The local characteristics of the image are measured,
、
representing an image of a vehicle to be recognized
Each local image and each image in the vehicle image library correspond to the first image
The degree of saliency of the individual partial images,
；
the method for calculating the saliency of the local image comprises the following steps:
for each local image, obtaining the confidence coefficient of the corresponding key point and the number of pixel points of the part which is not covered by the mask, and calculating the corresponding significance of each local image by combining the total number of pixel points of the input image; the method specifically comprises the following steps:
wherein the content of the first and second substances,
representing local image correspondences
The confidence level of each of the key points,
is not shown byThe number of pixel points of the mask covering portion,
representing a total number of pixel points of the input image;
the vehicle weight recognition model is constructed on the basis of a one-dimensional convolutional layer and a deep learning network.
6. An apparatus, comprising:
at least one processor; and
a memory communicatively coupled to at least one of the processors; wherein the content of the first and second substances,
the memory stores instructions executable by the processor to implement the method of any of claims 1-4 for vehicle weight identification based on keypoint detection and local feature alignment.
7. A computer-readable storage medium storing computer instructions for execution by the computer to implement the method for vehicle weight identification based on keypoint detection and local feature alignment of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110504848.8A CN112990152B (en) | 2021-05-10 | 2021-05-10 | Vehicle weight identification method based on key point detection and local feature alignment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110504848.8A CN112990152B (en) | 2021-05-10 | 2021-05-10 | Vehicle weight identification method based on key point detection and local feature alignment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112990152A CN112990152A (en) | 2021-06-18 |
| CN112990152B true CN112990152B (en) | 2021-07-30 |
Family
ID=76337419
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110504848.8A Active CN112990152B (en) | 2021-05-10 | 2021-05-10 | Vehicle weight identification method based on key point detection and local feature alignment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112990152B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113569912A (en) * | 2021-06-28 | 2021-10-29 | 北京百度网讯科技有限公司 | Vehicle identification method and device, electronic equipment and storage medium |
| CN114022869B (en) * | 2022-01-05 | 2022-03-22 | 中科视语(北京)科技有限公司 | Vehicle heavy identification method and device based on cascade network |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108345837A (en) * | 2018-01-17 | 2018-07-31 | 浙江大学 | A kind of pedestrian's recognition methods again based on the study of human region alignmentization feature representation |
| CN110598715A (en) * | 2019-09-04 | 2019-12-20 | 腾讯科技（深圳）有限公司 | Image recognition method and device, computer equipment and readable storage medium |
| CN110765954A (en) * | 2019-10-24 | 2020-02-07 | 浙江大华技术股份有限公司 | Vehicle weight recognition method, equipment and storage device |
| CN111738172A (en) * | 2020-06-24 | 2020-10-02 | 中国科学院自动化研究所 | Cross-domain target re-identification method based on feature counterstudy and self-similarity clustering |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113056743A (en) * | 2018-09-20 | 2021-06-29 | 辉达公司 | Training neural networks for vehicle re-recognition |
-
2021
- 2021-05-10 CN CN202110504848.8A patent/CN112990152B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108345837A (en) * | 2018-01-17 | 2018-07-31 | 浙江大学 | A kind of pedestrian's recognition methods again based on the study of human region alignmentization feature representation |
| CN110598715A (en) * | 2019-09-04 | 2019-12-20 | 腾讯科技（深圳）有限公司 | Image recognition method and device, computer equipment and readable storage medium |
| CN110765954A (en) * | 2019-10-24 | 2020-02-07 | 浙江大华技术股份有限公司 | Vehicle weight recognition method, equipment and storage device |
| CN111738172A (en) * | 2020-06-24 | 2020-10-02 | 中国科学院自动化研究所 | Cross-domain target re-identification method based on feature counterstudy and self-similarity clustering |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112990152A (en) | 2021-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109615611B (en) | Inspection image-based insulator self-explosion defect detection method | |
| EP3975123A1 (en) | Map constructing method, positioning method and system, wireless communication terminal, and computer-readable medium | |
| Xie et al. | Multilevel cloud detection in remote sensing images based on deep learning | |
| Lin et al. | Line segment extraction for large scale unorganized point clouds | |
| WO2019001481A1 (en) | Vehicle appearance feature identification and vehicle search method and apparatus, storage medium, and electronic device | |
| CN110675487B (en) | Three-dimensional face modeling and recognition method and device based on multi-angle two-dimensional face | |
| CN112990152B (en) | Vehicle weight identification method based on key point detection and local feature alignment | |
| US20180349716A1 (en) | Apparatus and method for recognizing traffic signs | |
| CN112016401A (en) | Cross-modal-based pedestrian re-identification method and device | |
| CN111932577B (en) | Text detection method, electronic device and computer readable medium | |
| CN110222572A (en) | Tracking, device, electronic equipment and storage medium | |
| CN108550165A (en) | A kind of image matching method based on local invariant feature | |
| CN111724443B (en) | Unified scene visual positioning method based on generative confrontation network | |
| CN110852327A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| Liu et al. | Registration of infrared and visible light image based on visual saliency and scale invariant feature transform | |
| CN110909582B (en) | Face recognition method and equipment | |
| CN112633222B (en) | Gait recognition method, device, equipment and medium based on countermeasure network | |
| CN112861776A (en) | Human body posture analysis method and system based on dense key points | |
| CN113793370A (en) | Three-dimensional point cloud registration method and device, electronic equipment and readable medium | |
| CN113658195B (en) | Image segmentation method and device and electronic equipment | |
| CN115577768A (en) | Semi-supervised model training method and device | |
| CN112232162B (en) | Pedestrian detection method and device based on multi-feature fusion cascade classifier | |
| Jiao et al. | Individual building rooftop and tree crown segmentation from high-resolution urban aerial optical images | |
| CN114067277A (en) | Pedestrian image recognition method and device, electronic equipment and storage medium | |
| CN114332809A (en) | Image identification method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |