WO2020258714A1 - 一种骑行者重识别方法、装置及设备 - Google Patents

一种骑行者重识别方法、装置及设备 Download PDF

Info

Publication number
WO2020258714A1
WO2020258714A1 PCT/CN2019/121517 CN2019121517W WO2020258714A1 WO 2020258714 A1 WO2020258714 A1 WO 2020258714A1 CN 2019121517 W CN2019121517 W CN 2019121517W WO 2020258714 A1 WO2020258714 A1 WO 2020258714A1
Authority
WO
WIPO (PCT)
Prior art keywords
people
person
vehicle
pictures
vehicles
Prior art date
Application number
PCT/CN2019/121517
Other languages
English (en)
French (fr)
Inventor
魏新明
胡文泽
王孝宇
Original Assignee
深圳云天励飞技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳云天励飞技术有限公司 filed Critical 深圳云天励飞技术有限公司
Publication of WO2020258714A1 publication Critical patent/WO2020258714A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition

Definitions

  • the invention relates to the field of image processing, in particular to the technology of re-identification of cyclists.
  • monitoring will be installed at points above the motor vehicle lanes.
  • people and vehicles combination of cyclists and non-motor vehicles
  • all violations of non-motor vehicles occupying motor lanes will be captured and recorded.
  • the management personnel need to obtain two snapshots of the front and back of the offending vehicle.
  • the front of the passenger car has the facial information of the rider, and the rear of the electric car has the license plate information. This information can better assist in the subsequent handling of violations.
  • relevant departments will install the monitoring in pairs. One camera shoots in the direction of traffic (the back can be captured), and the other is in the opposite direction (the front can be captured).
  • the invention provides a method, device and equipment for re-identification of riders to improve the matching degree of re-identification of riders.
  • a method for re-identification of cyclists including:
  • At least two pictures of people and vehicles corresponding to the highest similarity value are identified as the same rider.
  • the migration learning of the pedestrian re-identification model to obtain the cyclist re-identification model includes:
  • the person and vehicle picture includes a front person and vehicle picture and a back person and vehicle picture of each cyclist, and the cyclist re-recognition model is used to extract feature vectors of multiple person and vehicle pictures in the person and vehicle picture library.
  • the calculating multiple similarity values between the feature vectors includes:
  • Inner product operations are performed on the feature vectors of the front-side people-vehicle picture and the feature vectors of the back-side people-vehicle picture respectively to obtain the inner product results between the multiple feature vectors, where the inner product results are used as each Group the similarity value between the feature vector of the front view of people and cars and the feature vector of the back view of people and cars.
  • the method further includes:
  • Obtain image information of multiple pictures of people and vehicles through a monitoring device where the monitoring device includes at least one camera that shoots the front of the person and the vehicle, and a camera that shoots the back of the person and the vehicle, and the image information includes the person and vehicle.
  • the capture location and/or the capture time store the person and vehicle picture in a corresponding type of person and vehicle picture library
  • the extracting feature vectors of multiple pictures of people and cars in a picture library of people and cars using the cyclist re-identification model includes:
  • target person and vehicle picture library is a person and vehicle picture library of a type specified by the user
  • the cyclist re-recognition model is used to extract the feature vectors of the multiple pictures of people and vehicles in the target people and vehicles picture library.
  • the method further includes:
  • a device for re-identifying a cyclist including:
  • the learning module is used to perform migration learning on the pedestrian re-identification model to obtain the cyclist re-identification model;
  • An extraction module for extracting feature vectors of multiple pictures of people and cars in a picture library of people and cars using the cyclist re-identification model
  • a calculation module for calculating multiple similarity values between the feature vectors
  • the recognition module is configured to recognize at least two pictures of people and vehicles corresponding to the highest similarity value as the same rider.
  • the person and vehicle picture includes a front person and vehicle picture and a back person and vehicle picture of each cyclist;
  • the extraction module is specifically configured to extract the feature vector of the front view of people and vehicles and the feature vector of the back view of people and vehicles respectively;
  • the calculation module is specifically configured to perform inner product operations on the feature vectors of the front-side people-vehicle pictures and the feature vectors of the back-side people-vehicle pictures to obtain the inner product results between the multiple feature vectors.
  • the result of the inner product is used as the similarity value between the feature vector of each group of frontal pictures of people and cars and the feature vector of the back side of pictures of people and cars.
  • the device further includes:
  • the acquisition module is used to acquire image information of multiple pictures of people and vehicles through a monitoring device, where the monitoring device includes at least one camera that shoots the front of the person and the vehicle, and a camera that shoots the back of the person and the vehicle, the image information Including the capture location and capture time of the person and vehicle picture;
  • the storage module is configured to store the person and vehicle picture in a corresponding type of person and vehicle picture library according to the capture location and/or the capture time;
  • the extraction module includes:
  • the first acquiring unit is configured to acquire a target person and vehicle picture library, wherein the target person and vehicle picture library is a person and vehicle picture library of the type specified by the user;
  • the extracting unit is configured to use the cyclist re-identification model to extract feature vectors of the multiple pictures of the person and vehicle in the target person and vehicle picture library.
  • the learning module includes:
  • the second acquiring unit is used to acquire the backbone network parameters of the pedestrian re-identification model
  • the adding unit is used to add a classification layer after the backbone network of the pedestrian re-identification model according to the number of categories of the training samples of the cyclist;
  • the adjustment unit is configured to adjust the parameters of the classification layer and the backbone network parameters to obtain the rider re-identification model.
  • the device further includes:
  • a setting module configured to set the storage time of the pictures of people and cars in the people and cars picture library according to the position distance between the camera for front shooting and the camera for shooting behind;
  • the storage module is also used for when the storage time of the pictures of people and vehicles exceeds the storage time of the pictures of people and cars in the set of people and cars picture library, release the pictures of people and cars whose storage time exceeds the set time of storage.
  • a cyclist re-identification device including a processor, an input device, an output device, and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to By calling the program instructions, any one of the first aspect or the first aspect described above is executed to implement the method.
  • a computer-readable storage medium stores instructions that, when run on a computer, cause the computer to execute any one of the first aspect or the first aspect. Implement the described method.
  • a computer program product containing instructions which when run on a computer, causes the computer to execute any one of the foregoing first aspect or the first aspect to implement the method.
  • FIG. 1 is a schematic diagram of collecting pictures of people and vehicles according to an example of an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of a method for re-identifying a cyclist according to an embodiment of the present invention
  • FIG. 3 is a schematic flowchart of another method for re-identifying a cyclist provided by an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a process of performing transfer learning on a pedestrian re-identification model provided by an embodiment of the present invention
  • FIG. 5 is a schematic structural diagram of a device for re-identifying a cyclist according to an embodiment of the present invention.
  • Fig. 6 is a schematic structural diagram of a cyclist re-identification device provided by an embodiment of the present invention.
  • FIG. 1 it is a schematic diagram of collecting pictures of people and vehicles according to an example of the present invention.
  • a pair of monitoring equipment is installed within a certain distance above the motor vehicle lane, including a camera 1 that shoots people and vehicles in front, and a pair Camera 2 for rear shooting of people and vehicles.
  • Camera 1 shoots in the opposite direction of the traffic flow, and can capture the front of non-motorized vehicles and people driving illegally in the motor lane
  • camera 2 shoots in the direction of traffic flow, and can capture the back of non-motorized vehicles and people.
  • the embodiment of the present invention provides a method, device and equipment for re-identification of a cyclist.
  • a re-identification model of the cyclist is obtained, and the re-identification model of the cyclist is used to extract a plurality of pedestrians and vehicles in a picture library of people and vehicles.
  • the feature vector of the picture is calculated, the similarity value between the feature vectors is calculated, and at least two pictures of people and vehicles corresponding to the highest similarity value are recognized as the same rider, which improves the matching degree of re-identification of riders, Realize the monitoring of illegal activities of non-motor vehicles.
  • Fig. 2 is a schematic flowchart of a method for re-identifying a cyclist provided by an embodiment of the present invention.
  • the method may include the following steps:
  • Pedestrian re-identification also known as pedestrian re-identification, is a technology that uses computer vision technology to determine whether there are specific pedestrians in an image or video sequence. At present, the research of pedestrian re-identification is progressing rapidly. On several public data sets, the accuracy of pedestrian re-identification has improved significantly.
  • Pedestrian re-identification models such as residual network ResNet-50.
  • Transfer learning is a machine learning method. Due to the difference between the data set and the real data, the model trained on the data set A performs poorly on the real data B. At present, the method of transfer learning is mainly adopted, training on the labeled data set A and unlabeled data set B, and finally tested on the test set of data set B. That is, take the model developed for task A as the initial point and reuse it in the process of developing the model for task B. Therefore, it is a very good choice to change the pedestrian re-identification model to the cyclist re-identification model through transfer learning, which can solve the data bottleneck problem in the cyclist re-identification.
  • the traffic supervision department installs cameras in pairs, and stores the pictures of people and cars captured by these cameras in the people and cars picture library. Specifically, the person and car picture is detected and cropped from the original picture taken by the camera. Many people and vehicles pictures are stored in the people and vehicles picture library.
  • the cyclist re-recognition model can be used to extract the feature vectors of multiple pictures of people and vehicles in the people-vehicle image library.
  • Feature vector of car picture Feature vector of car picture.
  • multiple pictures of people and vehicles can be obtained in the people and vehicles picture library, and the above-mentioned cyclist re-recognition model can be used to extract the feature vectors of multiple pictures of people and vehicles one by one or at the same time.
  • the feature vector of the person and car picture includes the person's appearance, clothing, license plate, and the external characteristics of the car.
  • one person and vehicle picture corresponds to one feature vector or one type of feature vector.
  • the feature vector is the external feature of the rider and the vehicle.
  • one person and vehicle picture also corresponds to multiple feature vectors or multiple types of feature vectors.
  • multiple similarity values between the feature vectors are calculated. Specifically, the similarity values of any two or more feature vectors are respectively calculated, and multiple similarity values between all the extracted feature vectors are obtained.
  • the number of feature vectors for matching can be set as required.
  • the front-side people-vehicle pictures and the back-side people-vehicle pictures of the cyclist are separately stored, and the feature vectors of the above-mentioned front-side people-vehicle pictures and the back-side people-vehicle pictures can be extracted respectively, and either The similarity value between the feature vector of the front view of people and cars and all the feature vectors of the back view of people and cars in the extracted people and car image library.
  • one feature vector or one type of feature vector of multiple pictures of people and vehicles is extracted in the above steps, and the similarity between the feature vectors of the multiple pictures of people and vehicles is calculated to obtain multiple similarity values.
  • the similarity between each type of feature vectors of the multiple person and vehicle pictures is calculated according to the type of the feature vector. Then, according to the weight of each type of feature vector, a comprehensive similarity value between the extracted feature vectors of each type can be calculated as the final similarity value. For example, for a car picture A, feature vectors A1 and A2 are extracted; for a car picture B, feature vectors B1, B2 are extracted.
  • the feature vectors A1 and B1 are the same type of feature vectors, such as the facial feature vectors of a cyclist; the feature vectors A2 and B2 are another type of feature vectors, such as the clothing feature vector of the cyclist.
  • the highest similarity value is determined, and at least two pictures of people and vehicles corresponding to the highest similarity value are identified as the same rider. In this way, more characteristic information of the rider can be obtained, which is convenient for judging whether the rider is driving illegally.
  • the front side and back side pictures of the cyclist are stored in the people-vehicle picture library, and they are identified separately.
  • the back picture of people and cars with the highest similarity value to the front picture of people and cars can be determined, so that the group of front pictures of people and cars with the highest similarity value Recognize the same cyclist as the picture of the person and car on the back.
  • the rider drives illegally, but only the front of the rider and the vehicle can be obtained based on the captured frontal pictures of people and vehicles, but the license plate information of the vehicle cannot be obtained.
  • a cyclist re-identification model is obtained by performing migration learning on a pedestrian re-identification model, and the cyclist re-identification model is used to extract images of multiple pedestrians and vehicles in a vehicle image library.
  • Feature vector calculate the similarity value between the feature vectors, and identify at least two pictures of people and vehicles corresponding to the highest similarity value as the same rider, which improves the matching degree of re-identification of riders and realizes Monitoring of illegal activities of non-motor vehicles.
  • Fig. 3 is a schematic flowchart of a method for re-identifying a cyclist provided by an embodiment of the present invention.
  • the method may include the following steps:
  • the traffic supervision department sets up monitoring equipment above the road, and can take pictures of people and vehicles through the monitoring equipment.
  • the monitoring device includes at least one camera for front shooting of people and vehicles, and a camera for rear shooting of people and vehicles, so that the front side and back side images of the rider can be obtained.
  • the image information of the pictures of people and vehicles can also be obtained.
  • the image information includes the capture location and capture time of the pictures of people and vehicles.
  • S203 According to the capture location and/or the capture time, store the person and vehicle picture in a corresponding type of person and vehicle picture library.
  • the image source range of the human and vehicle image library is reduced.
  • the types of people and vehicles picture libraries are classified according to the location and time of the capture of the monitoring device corresponding to the source of the person and vehicle picture, or the location and time of the capture. That is, a person and vehicle picture library only stores the person and vehicle pictures of a number of monitoring devices in a certain capture location, or stores the people and vehicle pictures obtained within a certain capture time range, or stores the capture time range of a certain capture location. People and cars pictures.
  • camera A and camera B are a pair of monitoring equipment, that is, camera A is responsible for shooting the front of people and vehicles, and camera B is responsible for shooting the back of people and vehicles; camera C and camera D are A pair of monitoring equipment, and so on. Then, the photos taken by camera A and camera B can be stored in people and vehicle photo library 1, and camera C and camera D can be stored in people and vehicle photo library 2, and so on.
  • camera A and camera B can be Store the pictures of people and cars taken from 8:00 ⁇ 9:00 in the people and car picture library 1, and store the pictures of people and cars taken by camera A and camera B between 9:00 ⁇ 10:00 in the people and car picture library 2. , And so on.
  • S205 Add a classification layer after the backbone network of the pedestrian re-identification model according to the number of categories of the training samples of the cyclist.
  • the above steps S204 to S206 are to perform migration learning on the pedestrian re-identification model to obtain the rider re-identification model.
  • the principle is to regard the pedestrian re-identification model as the initial model of the cyclist re-identification model, and obtain a more accurate cyclist re-identification model through the training of several cyclist training samples.
  • FIG. 4 an embodiment of the present invention provides a schematic diagram of a process of performing migration learning on a pedestrian re-recognition model.
  • the specific training process is as follows: first, a trained pedestrian re-recognition model is obtained.
  • the backbone network parameters (Resnet-backbone) (a residual network) of the model are obtained, and the classification layer (FC) is added after the backbone network of the pedestrian re-identification model according to the number of categories of the cyclist training samples. For example, if 100,000 training samples of cyclists are manually classified, the training samples belonging to the same cyclist are classified into one category, and the above 100,000 training samples of cyclists are classified into 27,000 categories, then according to the number of categories of training samples, Add a classification layer, so that the output vector of the trained classification layer is a 27000-dimensional vector. Finally, adjust the parameters according to the loss function (softmaxloss), first adjust the parameters of the classification layer, and adjust the entire network parameters after the loss is stable, including the parameters of the classification layer and the backbone network parameters, to obtain the rider re-identification model.
  • the loss function softmaxloss
  • the user specifies the feature vector extraction and matching of a certain type of people and vehicle image libraries, that is, specifies the feature vector of the people and vehicle images at a certain capture location and/or a certain capture time Extraction and matching. Therefore, a target person and vehicle picture library is acquired, and the target person and vehicle picture library is a person and vehicle picture library of the type specified by the user.
  • the cyclist re-recognition model is used to extract the feature vectors of multiple human-vehicle pictures in the target human-vehicle picture library.
  • x i is the i-th human-vehicle picture
  • feat i is the corresponding human-vehicle feature vector.
  • net is the function of the above residual network.
  • the person-vehicle picture may include a front person-vehicle picture and a back person-vehicle picture of each cyclist, then S208 includes: extracting the feature vector of the front person-vehicle picture and the feature of the back person-vehicle picture respectively vector.
  • multiple similarity values between the feature vectors are calculated. Specifically, the similarity values of any two or more feature vectors are respectively calculated, and multiple similarity values between all the extracted feature vectors are obtained.
  • the number of feature vectors for matching can be set as required.
  • the person and vehicle picture may include the front person and vehicle picture and the back person and vehicle picture of each cyclist.
  • S209 specifically includes: combining the feature vector of the front person and vehicle picture and the feature vector of the back person and vehicle picture.
  • the following formula 2 can be used to obtain the aforementioned similarity value:
  • InnerProduct refers to the inner product result between two feature vectors.
  • S210 Identify at least two pictures of people and vehicles corresponding to the highest similarity value as the same rider.
  • the highest similarity value is determined, and at least two pictures of people and vehicles corresponding to the highest similarity value are identified as the same rider. In this way, more characteristic information of the rider can be obtained, which is convenient for judging whether the rider is driving illegally.
  • variable points feati and featj corresponding to the maximum value of InnerProduct can be obtained, thereby determining at least two feature vectors with the highest similarity.
  • At least two pictures of people and vehicles corresponding to the at least two feature vectors are identified as the same rider.
  • the storage time or storage duration set in the above step S201 when it is judged that the storage time of the people and vehicle pictures exceeds the storage time of the people and vehicle pictures in the people and vehicle picture library set above, the storage time will exceed the above The pictures of people and cars with the set storage time are released from the library.
  • a cyclist re-identification model is obtained by performing migration learning on a pedestrian re-identification model, and the cyclist re-identification model is used to extract images of multiple pedestrians and vehicles in a vehicle image library.
  • Feature vector calculate the similarity value between the feature vectors, and identify at least two pictures of people and vehicles corresponding to the highest similarity value as the same rider, which improves the matching degree of re-identification of riders and realizes Monitoring of illegal behaviors of non-motor vehicles; and according to the time and/or location of the capture, select the target human and vehicle image library, and extract the feature vectors of the human and vehicle images in the target human and vehicle image library for similarity matching to improve the image quality of people and vehicles The efficiency of feature vector matching; and timely release the pictures of people and vehicles that have been stored over time to save storage space.
  • an embodiment of the present invention also provides a cyclist re-identification device 100, which can be applied to the above-mentioned FIG. 2 and FIG. 3 Rider re-identification method.
  • the device 100 includes: a learning module 11, an extraction module 12, a calculation module 13, and an identification module 14, and may also include an acquisition module 15, a storage module 16, and a setting module 17.
  • a learning module 11 an extraction module 12, a calculation module 13, and an identification module 14, and may also include an acquisition module 15, a storage module 16, and a setting module 17.
  • the learning module 11 is used to perform migration learning on the pedestrian re-identification model to obtain the cyclist re-identification model;
  • the extraction module 12 is used for extracting feature vectors of multiple pictures of people and cars in a picture library of people and cars using the cyclist re-identification model;
  • the calculation module 13 is used to calculate multiple similarity values between the feature vectors
  • the recognition module 14 is configured to recognize at least two pictures of people and vehicles corresponding to the highest similarity value as the same rider.
  • the person and vehicle picture includes a front person and vehicle picture and a back person and vehicle picture of each cyclist;
  • the extraction module 12 is specifically configured to extract the feature vector of the front view of people and vehicles and the feature vector of the back view of people and vehicles respectively;
  • the calculation module 13 is specifically configured to perform inner product operations on the feature vectors of the front-side people-vehicle pictures and the feature vectors of the back-side people-vehicle pictures, respectively, to obtain an inner product result between the multiple feature vectors, wherein:
  • the inner product result is used as the similarity value between the feature vector of each group of frontal pictures of people and cars and the feature vector of the back side of pictures of people and cars.
  • the device further includes:
  • the obtaining module 15 is used to obtain image information of multiple pictures of people and vehicles through a monitoring device, wherein the monitoring device pair includes at least one camera for shooting the front of the person and vehicle, and a camera for shooting the back of the person and vehicle.
  • the image information includes the capture location and capture time of the person and vehicle picture;
  • the storage module 16 is configured to store the person and vehicle picture in a corresponding type of person and vehicle picture library according to the capture location and/or the capture time;
  • the extraction module 12 includes:
  • the first obtaining unit 121 is configured to obtain a target person and vehicle picture library, where the target person and vehicle picture library is a person and vehicle picture library of a type specified by the user;
  • the extracting unit 122 is configured to use the cyclist re-recognition model to extract feature vectors of multiple pictures of the person and vehicle in the target person and vehicle picture library.
  • the learning module 11 includes:
  • the second acquiring unit 111 is configured to acquire backbone network parameters of the pedestrian re-identification model
  • the adding unit 112 is configured to add a classification layer after the backbone network of the pedestrian re-identification model according to the number of categories of the cyclist training samples;
  • the adjustment unit 113 is configured to adjust the parameters of the classification layer and the backbone network parameters to obtain the rider re-identification model.
  • the device further includes:
  • the setting module 17 is configured to set the storage time of people and vehicles pictures in the people and vehicles picture library according to the positional distance between the camera for front shooting and the camera for back shooting;
  • the storage module 16 is also used for when the storage time of the pictures of people and vehicles exceeds the storage time of the pictures of people and cars in the set of people and cars picture library, release the pictures of people and cars whose storage time exceeds the set time of storage.
  • a cyclist re-identification model is obtained by performing migration learning on a pedestrian re-identification model, and the cyclist re-identification model is used to extract images of a plurality of people and vehicles in a picture library of people and vehicles.
  • Feature vector calculate the similarity value between the feature vectors, and identify at least two pictures of people and vehicles corresponding to the highest similarity value as the same rider, which improves the matching degree of re-identification of riders and realizes Monitoring of illegal behaviors of non-motor vehicles; and according to the time and/or location of the capture, select the target human and vehicle image library, and extract the feature vectors of the human and vehicle images in the target human and vehicle image library for similarity matching to improve the image quality of people and vehicles The efficiency of feature vector matching; and timely release the pictures of people and vehicles that have been stored over time to save storage space.
  • the device 200 includes a processor 21, and may also include an input device 22, an output device 23, and a memory 24.
  • the input device 22, the output device 23, the memory 24 and the processor 21 are connected to each other through a bus.
  • the memory 24 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or Portable read-only memory (compact disc read-only memory, CD-ROM), which is used for related instructions and data.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • CD-ROM Compact disc read-only memory
  • the input device 22 is used to input data and/or signals
  • the output device 23 is used to output data and/or signals.
  • the output device and the input device can be independent devices or a whole device.
  • the processor 21 may include one or more processors, for example, one or more central processing units (CPU).
  • processors for example, one or more central processing units (CPU).
  • CPU central processing units
  • the CPU may be a single-core CPU or Is a multi-core CPU.
  • the memory 24 is used to store program codes and data of the network device.
  • the processor 21 is used to call the program code and data in the memory, and execute the following steps:
  • At least two pictures of people and vehicles corresponding to the highest similarity value are identified as the same rider.
  • the processor 21 executes the migration learning of the pedestrian re-identification model to obtain the cyclist re-identification model, including:
  • the person and vehicle picture includes a front person and vehicle picture and a back person and vehicle picture of each rider, and the processor 21 executes the use of the rider re-identification model to extract more images from the person and vehicle picture library.
  • the steps of the feature vector of the personal car picture include:
  • the processor 21 executing the step of calculating multiple similarity values between the feature vectors includes:
  • Inner product operations are performed on the feature vectors of the front-side people-vehicle picture and the feature vectors of the back-side people-vehicle picture respectively to obtain the inner product results between the multiple feature vectors, where the inner product results are used as each Group the similarity value between the feature vector of the front view of people and cars and the feature vector of the back view of people and cars.
  • processor 21 further executes the following steps:
  • the capture location and/or the capture time store the person and vehicle picture in a corresponding type of person and vehicle picture library
  • the processor 21 executes the step of extracting the feature vectors of multiple pictures of people and cars in a picture library of people and cars by using the rider re-identification model, including:
  • target person and vehicle picture library is a person and vehicle picture library of a type specified by the user
  • the cyclist re-recognition model is used to extract the feature vectors of the multiple pictures of people and vehicles in the target people and vehicles picture library.
  • processor 21 further executes the following steps:
  • FIG. 5 only shows a simplified design of the rider re-identification device.
  • the electronic equipment may also contain other necessary components, including but not limited to any number of input/output devices, processors, controllers, memories, etc., and all electronic equipment that can implement the embodiments of the present application are in Within the scope of protection of this application.
  • a cyclist re-identification model is obtained by performing migration learning on a pedestrian re-identification model, and the cyclist re-identification model is used to extract images of multiple people and vehicles in a picture library of people and vehicles.
  • Feature vector calculate the similarity value between the feature vectors, and identify at least two pictures of people and vehicles corresponding to the highest similarity value as the same rider, which improves the matching degree of re-identification of riders and realizes Monitoring of illegal behaviors of non-motor vehicles; and according to the time and/or location of the capture, select the target human-vehicle picture library, extract the feature vectors of the human-vehicle pictures in the target human-vehicle picture library, and perform similarity matching to improve the quality of the human-vehicle pictures The efficiency of feature vector matching; and timely release the pictures of people and vehicles that have been stored over time to save storage space.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • General Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

一种骑行者重识别方法、装置及设备。通过对行人重识别模型进行迁移学习，得到骑行者重识别模型（S101），采用所述骑行者重识别模型提取人车图片库中多个人车图片的特征向量（S102），计算特征向量之间的相似度值（S103），将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者（S104），提高了骑行者重识别的匹配度，实现对非机动车辆违法行为的监控。

Description

一种骑行者重识别方法、装置及设备
本申请要求于2019年6月24日提交中国专利局，申请号为201910550548.6、发明名称为“一种骑行者重识别方法、装置及设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及图像处理领域，尤其涉及骑行者重识别技术。
背景技术
随着城镇化率的升高，全国各城镇的常驻人口在逐渐增加，这给原本拥挤的交通带来了很大压力。各类交通违规行为也给相关治理部门带来麻烦。其中非机动车占机动车道的违规行为有多发、常发的特点。目前机动车的违规识别抓拍取证已基本做到高准确度、自动高效，而非机动车的治理目前比较低效。主要原因在于人车相关的检测、识别算法的研究非常欠缺。
在实际治理中，机动车道上方会设点安装监控。通过对监控视频进行隔帧人车(骑行者与非机动车的组合体)检测，所有的非机动车占机动车道的违规行为将会被抓拍记录。但实际情况下只做到这步还不够，治理人员需要获取违规人车的前面与背面两张抓拍图片。人车的前面有骑行者的脸部信息，后面有电动车的牌照信息。这些信息能更好地辅助后续的违规处置。为达到这个要求，相关部门会将监控进行成对安装。一个摄像头朝车流方向拍摄(可抓拍背面)，另一个则朝相反方向拍摄(可抓拍正面)。无论是正面还是背面，人车检测都没有问题。难以处理的问题是，把一个监控下抓拍的正面人车与另一个监控下抓拍背面人车进行匹配，即骑行者重识别。实际场景下，载人、挡风物件等都会使得人车的前后两面相差巨大，这极大的增加了匹配困难。而且，可用于训练识别模型的数据极少。因此，需要提高骑行者重识别的匹配度。
发明内容
本发明提供一种骑行者重识别方法、装置及设备，以提高骑行者重识别的匹配度。
第一方面，提供了一种骑行者重识别方法，包括：
对行人重识别模型进行迁移学习，得到骑行者重识别模型；
采用所述骑行者重识别模型提取人车图片库中多个人车图片的特征向量；
计算所述特征向量之间的多个相似度值；
将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者。
在一个实现中，所述对行人重识别模型进行迁移学习，得到骑行者重识别模型，包括：
获取行人重识别模型的主干网络参数；
根据骑行者训练样本的类别数，在所述行人重识别模型的主干网络后添加分类层；
调整所述分类层的参数和所述主干网络参数，得到所述骑行者重识别模型。
在又一个实现中，所述人车图片包括每一骑行者的正面人车图和背面人车图，所述采用所述骑行者重识别模型提取人车图片库中多个人车图片的特征向量，包括：
分别提取所述正面人车图的特征向量和所述背面人车图的特征向量；
所述计算所述特征向量之间的多个相似度值，包括：
将所述正面人车图片的特征向量和所述背面人车图片的特征向量分别进行内积运算，得到所述多个特征向量之间的内积结果，其中，所述内积结果作为每一组正面人车图片的特征向量和背面人车图片的特征向量之间的相似度值。
在又一个实现中，所述方法还包括：
通过监控设备获取多个人车图片的图像信息，其中，所述监控设备至少包括一个对人车进行前方拍摄的摄像机，以及一个对人车进行后方拍摄的摄像机，所述图像信息包括所述人车图片的抓拍地点和抓拍时间；
根据所述抓拍地点和/或所述抓拍时间，将所述人车图片存储至相应类型的 人车图片库；
所述采用所述骑行者重识别模型提取人车图片库中中多个人车图片的特征向量，包括：
获取目标人车图片库，其中，所述目标人车图片库为所述用户指定类型的人车图片库；
采用所述骑行者重识别模型提取所述目标人车图片库中多个所述人车图片的特征向量。
在又一个实现中，所述方法还包括：
根据所述进行前方拍摄的摄像机以及所述进行后方拍摄的摄像机之间的位置距离，设置所述人车图片库中人车图片的存储时间；
当人车图片的存储时间超过所述设置的人车图片库中人车图片的存储时间时，将存储时间超过所述设置的存储时间的人车图片出库。
第二方面，提供了一种骑行者重识别装置，包括：
学习模块，用于对行人重识别模型进行迁移学习，得到骑行者重识别模型；
提取模块，用于采用所述骑行者重识别模型提取人车图片库中多个人车图片的特征向量；
计算模块，用于计算所述特征向量之间的多个相似度值；
识别模块，用于将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者。
在一个实现中，所述人车图片包括每一骑行者的正面人车图和背面人车图；
所述提取模块具体用于分别提取所述正面人车图的特征向量和所述背面人车图的特征向量；
所述计算模块具体用于将所述正面人车图片的特征向量和所述背面人车图片的特征向量分别进行内积运算，得到所述多个特征向量之间的内积结果，其中，所述内积结果作为每一组正面人车图片的特征向量和背面人车图片的特征向量之间的相似度值。
在又一个实现中，所述装置还包括：
获取模块，用于通过监控设备获取多个人车图片的图像信息，其中，所述监控设备至少包括一个对人车进行前方拍摄的摄像机，以及一个对人车进行后方拍摄的摄像机，所述图像信息包括所述人车图片的抓拍地点和抓拍时间；
存储模块，用于根据所述抓拍地点和/或所述抓拍时间，将所述人车图片存储至相应类型的人车图片库；
所述提取模块包括：
第一获取单元，用于获取目标人车图片库，其中，所述目标人车图片库为所述用户指定类型的人车图片库；
提取单元，用于采用所述骑行者重识别模型提取所述目标人车图片库中多个所述人车图片的特征向量。
在又一个实现中，所述学习模块包括：
第二获取单元，用于获取行人重识别模型的主干网络参数；
添加单元，用于根据骑行者训练样本的类别数，在所述行人重识别模型的主干网络后添加分类层；
调整单元，用于调整所述分类层的参数和所述主干网络参数，得到所述骑行者重识别模型。
在又一个实现中，所述装置还包括：
设置模块，用于根据所述进行前方拍摄的摄像机以及所述进行后方拍摄的摄像机之间的位置距离，设置所述人车图片库中人车图片的存储时间；
所述存储模块还用于当人车图片的存储时间超过所述设置的人车图片库中人车图片的存储时间时，将存储时间超过所述设置的存储时间的人车图片出库。
第三方面，提供了一种骑行者重识别设备，包括处理器、输入设备、输出设备和存储器，所述存储器用于存储计算机程序，所述计算机程序包括程序指令，所述处理器被配置用于调用所述程序指令，执行上述第一方面或第一方面中的任一种实现所述的方法。
第四方面，提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行上述第一方面或第一方面中的任一种实现所述的方法。
第五方面，提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述第一方面或第一方面中的任一种实现所述的方法。
本发明实施例具有以下有益效果：
由于行人重识别在算法以及数据方面都有较好的积累，而骑行者重识别则是一个全新的问题，用于训练的样本有限，迁移学习能对行人重识别所学习的规律进行借鉴利用，因此通过对行人重识别模型进行迁移学习，得到骑行者重识别模型，利用该骑行者重识别模型提取人车图片库中多个人车图片的特征向量，计算特征向量之间的相似度值，并将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者，提高了骑行者重识别的匹配度，实现对非机动车辆违法行为的监控。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例示例的一种人车图片的采集示意图；
图2是本发明实施例提供的一种骑行者重识别方法的流程示意图；
图3是本发明实施例提供的又一种骑行者重识别方法的流程示意图；
图4是本发明实施例提供的对行人重识别模型进行迁移学习的过程示意图；
图5是本发明实施例提供的一种骑行者重识别装置的结构示意图；
图6是本发明实施例提供的一种骑行者重识别设备的结构示意图。
具体实施方式
为了获得人车更多的信息，一般地，交通监管部门会成对安装摄像机。如图1所示，是本发明实施例示例的一种人车图片的采集示意图，在机动车道上方的一定距离内安装监控设备对，包括一个对人车进行前方拍摄的摄像机1，以及一个对人车进行后方拍摄的摄像机2。摄像机1朝向车流的相反方向拍摄，可抓拍违法在机动车道行驶的非机动车和人的正面，摄像机2朝向车流方向拍摄，可抓拍非机动车和人的背面。然而，把一个监控下抓拍的正面人车与另一个监控下抓拍背面人车进行匹配，即骑行者重识别，目前没有相应的技术。实际场景下，载人、挡风物件等都会使得人车的前后两面相差巨大，这极大的增加了匹配困难。而且，可用于训练识别模型的数据极少。
本发明实施例提供一种骑行者重识别方法、装置及设备，通过对行人重识别模型进行迁移学习，得到骑行者重识别模型，利用该骑行者重识别模型提取人车图片库中多个人车图片的特征向量，计算特征向量之间的相似度值，并将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者，提高了骑行者重识别的匹配度，实现对非机动车辆违法行为的监控。
图2为本发明实施例提供的一种骑行者重识别方法的流程示意图，示例性的，该方法可包括以下步骤：
S101、对行人重识别模型进行迁移学习，得到骑行者重识别模型。
行人重识别(Person re-identification)也称行人再识别，是利用计算机视觉技术判断图像或者视频序列中是否存在特定行人的技术。目前行人重识别的研究进展迅速，在几个公开数据集上，行人重识别的精度提高显著。行人重识别模型如残差网络ResNet-50。
由于行人重识别在算法以及数据方面都有较好的积累，而骑行者重识别则是一个全新的问题，用于训练的样本较少，迁移学习能对行人重识别所学习的规律进行借鉴利用。迁移学习是一种机器学习方法，由于数据集与现实数据之 间的差异，导致在数据集A上训练好的模型在现实数据B上性能表现不佳。目前主要采用迁移学习的方法，在有标签的数据集A和无标签数据集B上训练，最后在数据集B的测试集上测试。也就是把为任务A开发的模型作为初始点，重新使用在为任务B开发模型的过程中。因此将行人重识别模型通过迁移学习改变为骑行者重识别模型是非常不错的选择，这样能解决骑行者重识别中的数据瓶颈问题。
S102、采用所述骑行者重识别模型提取人车图片库中多个人车图片的特征向量。
如图1所示，为了对违法骑行者进行监管，交通监管部门成对安装了摄像机，将这些摄像机摄取的人车图片存储到人车图片库中。具体地，该人车图片是从摄像机摄取的原始图片中检测并剪裁出来的。人车图片库中存储了多个人车图片。
在获得上述骑行者重识别模型后，可以采用该骑行者重识别模型提取人车图片库中多个人车图片的特征向量，即将人车图片输入骑行者重识别模型，通过模型的复杂运算得到人车图片的特征向量。具体实现中，可以在人车图片库中获得多张人车图片，可以采用上述骑行者重识别模型逐一或者同时提取多张人车图片的特征向量。该人车图片的特征向量包括人的外貌、衣着、车牌、车的外在特征等。
可选地，一个人车图片对应一条特征向量或一个类型的特征向量。例如，该特征向量为骑行者和车辆的外在特征。
可选的，一个人车图片也对应多条特征向量或多个类型的特征向量。
S103、计算所述特征向量之间的多个相似度值。
在提取出多个人车图片的特征向量后，计算特征向量之间的多个相似度值。具体地，分别计算任意两个或者多个特征向量的相似度值，得到提取出的所有特征向量之间的多个相似度值。可以根据需要设置进行匹配的特征向量个数。
例如，人车图片库中分别存储了骑行者的正面人车图片和背面人车图片， 并且分别进行了标识，可以分别提取上述正面人车图片和背面人车图片的特征向量，并且计算任一正面人车图片的特征向量与提取的人车图片库中的所有的背面人车图片的特征向量之间的相似度值。
可选地，上述步骤中提取了多个人车图片的一条特征向量或一个类型的特征向量，则计算这多个人车图片的特征向量之间的相似度，得到多个相似度值。
可选地，上述步骤中提取了多个类型的特征向量，则按照特征向量的类型，分别计算这多个人车图片的每个类型的特征向量之间的相似度。然后，可以按照每个类型的特征向量的权重，计算提取出的各个类型的特征向量之间的综合的相似度值，作为最终的相似度值。例如，对于人车图片A，提取出特征向量A1、A2；对于人车图片B，提取出特征向量B1、B2。而特征向量A1、B1为同一类型的特征向量，例如为骑行者的脸部特征向量；特征向量A2、B2为另一类型的特征向量，例如为骑行者的服装特征向量。设置特征向量A1、B1这一类特征向量的权重为0.7，设置特征向量A2、B2这一类特征向量的权重为0.3。则当计算出特征向量A1、B1之间的相似度值为98％，计算出特征向量A2、B2之间的相似度值为80％时，得到人车图片A、B的特征向量的综合的相似度为98％*0.7+80％*0.3。
S104、将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者。
在计算出上述特征向量之间的多个相似度值之后，确定最高的相似度值，将最高的相似度值对应的至少两个人车图片识别为同一骑行者。这样，就可以获取该骑行者更多的特征信息，便于判断骑行者是否违法驾驶。
例如，人车图片库中分别存储了骑行者的正面人车图片和背面人车图片，并且分别进行了标识，在提取了任一正面人车图片的特征向量与提取的人车图片库中的所有的背面人车图片的特征向量之间的相似度值后，可确定与该正面人车图片之间相似度值最高的背面人车图片，从而将该组相似度值最高的正面人车图片和背面人车图片识别为同一骑行者。假设该骑行者违法驾驶，但根据 抓拍到的正面人车图片只能获取该骑行者和车辆的正面，但获取不到车辆的车牌信息，通过本申请的方法，将匹配到与正面人车图片相似度最高的背面人车图片，可以确定该组正面人车图片和背面人车图片为同一骑行者，则可以根据该背面人车图片获取到车牌信息，从而获取到了该骑行者更多的特征信息，便于判断骑行者是否违法驾驶。
根据本发明实施例提供的一种骑行者重识别方法，通过对行人重识别模型进行迁移学习，得到骑行者重识别模型，利用该骑行者重识别模型提取人车图片库中多个人车图片的特征向量，计算特征向量之间的相似度值，并将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者，提高了骑行者重识别的匹配度，实现对非机动车辆违法行为的监控。
图3为本发明实施例提供的一种骑行者重识别方法的流程示意图，示例性的，该方法可包括以下步骤：
S201、根据进行前方拍摄的摄像机以及进行后方拍摄的摄像机之间的位置距离，设置人车图片库中人车图片的存储时间。
由于机动车道上设置的摄像机每天/每小时/每分钟都要拍摄和存储大量的照片，因此，为了节省存储空间，需要设置人车图片库中人车图片的存储时间，超出存储时间的人车图片删除出库。
一般地，进行前方拍摄的摄像机和进行后方拍摄的摄像机之间有一定的位置距离，因此，获得正面拍摄照片和反面拍摄照片之间有一定的时间差，然而，正面拍摄照片和反面拍摄照片是相似度最高的人车图片组，因此，需要将正面拍摄照片和反面拍摄照片同时保留在人车图片库，以便于后续的匹配。因此，可以根据进行前方拍摄的摄像机以及进行后方拍摄的摄像机之间的位置距离，设置人车图片库的存储时间，或者称人车图片库的持续存储时长(最新入库人车图片与库中最早的人车图片的时间差)。
S202、通过监控设备获取多个人车图片的图像信息。
如图1所示，交通监管部门在道路上方设置监控设备，可以通过监控设备摄取人车图片。其中，该监控设备至少包括一个对人车进行前方拍摄的摄像机，以及一个对人车进行后方拍摄的摄像机，从而可以获取骑行者的正面人车图片和背面人车图片。
在存储人车图片的同时，还可以获取人车图片的图像信息。其中，该图像信息包括人车图片的抓拍地点和抓拍时间。
S203、根据所述抓拍地点和/或所述抓拍时间，将所述人车图片存储至相应类型的人车图片库。
本实施例中，为了提高匹配效率，缩小人车图片库的图片来源范围。具体地，根据人车图片源对应的监控设备抓拍地点、抓拍时间、或者抓拍地点和抓拍时间来划分人车图片库的类型。即一个人车图片库只存储某个抓拍地点的若干个监控设备的人车图片，或者存储某一个抓拍时间范围内获得的人车图片，或者存储某个抓拍地点的某一个抓拍时间范围内获得的人车图片。
例如，在某一车道上间隔设置了十个摄像机，其中，摄像机A和摄像机B是一个监控设备对，即摄像机A负责拍摄人车正面，摄像机B负责拍摄人车背面；摄像机C和摄像机D是一个监控设备对，以此类推。则可以将摄像机A和摄像机B拍摄的照片存储至人车图片库1，将摄像机C和摄像机D存储至人车图片库2，等等。又例如，摄像机A和摄像机B分别在8:00～9:00拍摄得到多个人车图片，又分别在9:00～10:00拍摄得到多个人车图片，则可以将摄像机A和摄像机B在8:00～9:00拍摄得到的多个人车图片存储至人车图片库1，将摄像机A和摄像机B在9:00～10:00拍摄得到的多个人车图片存储至人车图片库2，以此类推。
S204、获取行人重识别模型的主干网络参数。
S205、根据骑行者训练样本的类别数，在所述行人重识别模型的主干网络后添加分类层。
S206、调整所述分类层的参数和所述主干网络参数，得到所述骑行者重识 别模型。
上述步骤S204～S206为对行人重识别模型进行迁移学习，得到骑行者重识别模型。其原理为将行人重识别模型视为骑行者重识别模型的初始模型，通过若干骑行者训练样本的训练，得到较为精确的骑行者重识别模型。如图4所示的本发明实施例提供的对行人重识别模型进行迁移学习的过程示意图，具体的训练过程为：首先，获取一个训练过的行人重识别模型。然后，获取该模型的主干网络参数(Resnet-backbone)(一种残差网络)，并根据骑行者训练样本的类别数，在该行人重识别模型的主干网络后添加分类层(FC)。例如，人工将10万个骑行者训练样本进行归类，属于同一个骑行者的训练样本归为一类，将上述10万个骑行者训练样本归为27000类，则根据训练样本的类别数，添加分类层，这样训练后的分类层的输出向量为27000维的向量。最后，根据损失函数(softmaxloss)调整参数，先调整分类层的参数，待损失稳定后再调整整个网络参数，包括分类层的参数和主干网络参数，得到骑行者重识别模型。
S207、获取目标人车图片库。
根据上述人车图片库的存储分类，用户指定对某个类型的人车图片库进行特征向量的提取和匹配，即指定对某个抓拍地点和/或某个抓拍时间的人车图片进行特征向量的提取和匹配。因此，获取目标人车图片库，该目标人车图片库为所述用户指定类型的人车图片库。
S208、采用所述骑行者重识别模型提取所述目标人车图片库中多个所述人车图片的特征向量。
在获取目标人车图片库后，采用骑行者重识别模型提取该目标人车图片库中多个人车图片的特征向量。
具体地，如下面的公式1所示：
feat i＝net(x i)       ……公式1
其中，x i为第i张人车图片，feat i为对应的人车特征向量。net即为上述残差网络的函数。
在具体的示例中，人车图片可以包括每一骑行者的正面人车图和背面人车图，则S208包括：分别提取所述正面人车图的特征向量和所述背面人车图的特征向量。
S209、计算所述特征向量之间的多个相似度。
在提取出多个人车图片的特征向量后，计算特征向量之间的多个相似度值。具体地，分别计算任意两个或者多个特征向量的相似度值，得到提取出的所有特征向量之间的多个相似度值。可以根据需要设置进行匹配的特征向量个数。
在具体的示例中，人车图片可以包括每一骑行者的正面人车图和背面人车图，S209具体包括：将所述正面人车图片的特征向量和所述背面人车图片的特征向量分别进行内积运算，得到所述多个特征向量之间的内积结果，其中，所述内积结果作为每一组正面人车图片的特征向量和背面人车图片的特征向量之间的相似度值。具体地，可以采用下面的公式2得到上述相似度值：
similarity＝InnerProduct(feat i，feat j)     ……公式2
其中，InnerProduct是指两个特征向量间的内积结果。
S210、将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者。
在计算出上述特征向量之间的多个相似度值之后，确定最高的相似度值，将最高的相似度值对应的至少两个人车图片识别为同一骑行者。这样，就可以获取该骑行者更多的特征信息，便于判断骑行者是否违法驾驶。
具体地，可以采用下面的公式3确定相似度最高的特征向量：
Figure PCTCN2019121517-appb-000001
其中，根据最大自变量点集函数argmax可以得到使得InnerProduct(feati,featj)取得最大值所对应的变量点feati和featj，从而确定相似度最高的至少两个特征向量。
在确定了相似度最高的至少两个特征向量后，将至少两个特征向量对应的至少两个人车图片识别为同一骑行者。
S211、当人车图片的存储时间超过所述设置的人车图片库中人车图片的存储时间时，将存储时间超过所述设置的存储时间的人车图片出库。
为了节省存储空间，根据上述步骤S201中设置的存储时间或者称存储持续时长，在判断人车图片的存储时间超过上述设置的人车图片库中人车图片的存储时间时，将存储时间超过上述设置的存储时间的人车图片出库。
根据本发明实施例提供的一种骑行者重识别方法，通过对行人重识别模型进行迁移学习，得到骑行者重识别模型，利用该骑行者重识别模型提取人车图片库中多个人车图片的特征向量，计算特征向量之间的相似度值，并将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者，提高了骑行者重识别的匹配度，实现对非机动车辆违法行为的监控；且根据抓拍时间和/或抓拍地点，选取目标人车图片库，提取目标人车图片库中的人车图片的特征向量进行相似度匹配，以提高人车图片的特征向量匹配的效率；且对存储超时的人车图片及时出库，以节省存储空间。
基于上述实施例中的骑行者重识别方法的同一构思，如图5所示，本发明实施例还提供一种骑行者重识别装置100，该装置可应用于上述图2、图3所述的骑行者重识别方法中。该装置100包括：学习模块11、提取模块12、计算模块13和识别模块14，还可以包括获取模块15、存储模块16和设置模块17。示例性的：
学习模块11，用于对行人重识别模型进行迁移学习，得到骑行者重识别模型；
提取模块12，用于采用所述骑行者重识别模型提取人车图片库中多个人车图片的特征向量；
计算模块13，用于计算所述特征向量之间的多个相似度值；
识别模块14，用于将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者。
在一个实现中，所述人车图片包括每一骑行者的正面人车图和背面人车图；
所述提取模块12具体用于分别提取所述正面人车图的特征向量和所述背面人车图的特征向量；
所述计算模块13具体用于将所述正面人车图片的特征向量和所述背面人车图片的特征向量分别进行内积运算，得到所述多个特征向量之间的内积结果，其中，所述内积结果作为每一组正面人车图片的特征向量和背面人车图片的特征向量之间的相似度值。
在又一个实现中，所述装置还包括：
获取模块15，用于通过监控设备获取多个人车图片的图像信息，其中，所述监控设备对至少包括一个对人车进行前方拍摄的摄像机，以及一个对人车进行后方拍摄的摄像机，所述图像信息包括所述人车图片的抓拍地点和抓拍时间；
存储模块16，用于根据所述抓拍地点和/或所述抓拍时间，将所述人车图片存储至相应类型的人车图片库；
所述提取模块12包括：
第一获取单元121，用于获取目标人车图片库，其中，所述目标人车图片库为所述用户指定类型的人车图片库；
提取单元122，用于采用所述骑行者重识别模型提取所述目标人车图片库中多个所述人车图片的特征向量。
在又一个实现中，所述学习模块11包括：
第二获取单元111，用于获取行人重识别模型的主干网络参数；
添加单元112，用于根据骑行者训练样本的类别数，在所述行人重识别模型的主干网络后添加分类层；
调整单元113，用于调整所述分类层的参数和所述主干网络参数，得到所述骑行者重识别模型。
在又一个实现中，所述装置还包括：
设置模块17，用于根据所述进行前方拍摄的摄像机以及所述进行后方拍摄 的摄像机之间的位置距离，设置所述人车图片库中人车图片的存储时间；
所述存储模块16还用于当人车图片的存储时间超过所述设置的人车图片库中人车图片的存储时间时，将存储时间超过所述设置的存储时间的人车图片出库。
有关上述各个模块、单元更详细的描述可以参考上述图2、图3所述的骑行者重识别方法的相关描述得到，这里不加赘述。
根据本发明实施例提供的一种骑行者重识别装置，通过对行人重识别模型进行迁移学习，得到骑行者重识别模型，利用该骑行者重识别模型提取人车图片库中多个人车图片的特征向量，计算特征向量之间的相似度值，并将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者，提高了骑行者重识别的匹配度，实现对非机动车辆违法行为的监控；且根据抓拍时间和/或抓拍地点，选取目标人车图片库，提取目标人车图片库中的人车图片的特征向量进行相似度匹配，以提高人车图片的特征向量匹配的效率；且对存储超时的人车图片及时出库，以节省存储空间。
图6为本发明实施例提供的一种骑行者重识别设备的结构示意图，该设备200包括：包括处理器21，还可包括输入装置22、输出装置23和存储器24。该输入装置22、输出装置23、存储器24和处理器21之间通过总线相互连接。
存储器24包括但不限于是随机存储记忆体(random access memory，RAM)、只读存储器(read-only memory，ROM)、可擦除可编程只读存储器(erasable programmable read only memory，EPROM)、或便携式只读存储器(compact disc read-only memory，CD-ROM)，该存储器用于相关指令及数据。
输入装置22用于输入数据和/或信号，以及输出装置23用于输出数据和/或信号。输出装置和输入装置可以是独立的器件，也可以是一个整体的器件。
处理器21可以包括是一个或多个处理器，例如包括一个或多个中央处理器(central processing unit，CPU)，在处理器是一个CPU的情况下，该CPU可 以是单核CPU，也可以是多核CPU。
存储器24用于存储网络设备的程序代码和数据。
处理器21用于调用该存储器中的程序代码和数据，执行如下步骤：
对行人重识别模型进行迁移学习，得到骑行者重识别模型；
采用所述骑行者重识别模型提取人车图片库中多个人车图片的特征向量；
计算所述特征向量之间的多个相似度值；
将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者。
在一个实现中，所述处理器21执行所述对行人重识别模型进行迁移学习，得到骑行者重识别模型的步骤，包括：
获取行人重识别模型的主干网络参数；
根据骑行者训练样本的类别数，在所述行人重识别模型的主干网络后添加分类层；
调整所述分类层的参数和所述主干网络参数，得到所述骑行者重识别模型。
在又一个实现中，所述人车图片包括每一骑行者的正面人车图和背面人车图，所述处理器21执行所述采用所述骑行者重识别模型提取人车图片库中多个人车图片的特征向量的步骤，包括：
分别提取所述正面人车图的特征向量和所述背面人车图的特征向量；
所述处理器21执行所述计算所述特征向量之间的多个相似度值的步骤，包括：
将所述正面人车图片的特征向量和所述背面人车图片的特征向量分别进行内积运算，得到所述多个特征向量之间的内积结果，其中，所述内积结果作为每一组正面人车图片的特征向量和背面人车图片的特征向量之间的相似度值。
在又一个实现中，所述处理器21还执行如下步骤：
通过监控设备获取多个人车图片的图像信息，其中，所述监控设备对至少包括一个对人车进行前方拍摄的摄像机，以及一个对人车进行后方拍摄的摄像机，所述图像信息包括所述人车图片的抓拍地点和抓拍时间；
根据所述抓拍地点和/或所述抓拍时间，将所述人车图片存储至相应类型的人车图片库；
所述处理器21执行所述采用所述骑行者重识别模型提取人车图片库中中多个人车图片的特征向量的步骤，包括：
获取目标人车图片库，其中，所述目标人车图片库为所述用户指定类型的人车图片库；
采用所述骑行者重识别模型提取所述目标人车图片库中多个所述人车图片的特征向量。
在又一个实现中，所述处理器21还执行如下步骤：
根据所述进行前方拍摄的摄像机以及所述进行后方拍摄的摄像机之间的位置距离，设置所述人车图片库中人车图片的存储时间；
当人车图片的存储时间超过所述设置的人车图片库中人车图片的存储时间时，将存储时间超过所述设置的存储时间的人车图片出库。
可以理解的是，图5仅仅示出了骑行者重识别设备的简化设计。在实际应用中，电子设备还可以分别包含必要的其他元件，包含但不限于任意数量的输入/输出装置、处理器、控制器、存储器等，而所有可以实现本申请实施例的电子设备都在本申请的保护范围之内。
根据本发明实施例提供的一种骑行者重识别设备，通过对行人重识别模型进行迁移学习，得到骑行者重识别模型，利用该骑行者重识别模型提取人车图片库中多个人车图片的特征向量，计算特征向量之间的相似度值，并将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者，提高了骑行者重识别的匹配度，实现对非机动车辆违法行为的监控；且根据抓拍时间和/或抓拍地点，选取目标人车图片库，提取目标人车图片库中的人车图片的特征向量进行相似度匹配，以提高人车图片的特征向量匹配的效率；且对存储超时的人车图片及时出库，以节省存储空间。

Claims (10)

  1. 一种骑行者重识别方法，其特征在于，包括：
    对行人重识别模型进行迁移学习，得到骑行者重识别模型；
    采用所述骑行者重识别模型提取人车图片库中多个人车图片的特征向量；
    计算所述特征向量之间的多个相似度值；
    将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者。
  2. 根据权利要求1所述的方法，其特征在于，所述对行人重识别模型进行迁移学习，得到骑行者重识别模型，包括：
    获取行人重识别模型的主干网络参数；
    根据骑行者训练样本的类别数，在所述行人重识别模型的主干网络后添加分类层；
    调整所述分类层的参数和所述主干网络参数，得到所述骑行者重识别模型。
  3. 根据权利要求1所述的方法，其特征在于，所述人车图片包括每一骑行者的正面人车图和背面人车图，所述采用所述骑行者重识别模型提取人车图片库中多个人车图片的特征向量，包括：
    分别提取所述正面人车图的特征向量和所述背面人车图的特征向量；
    所述计算所述特征向量之间的多个相似度值，包括：
    将所述正面人车图片的特征向量和所述背面人车图片的特征向量分别进行内积运算，得到所述多个特征向量之间的内积结果，其中，所述内积结果作为每一组正面人车图片的特征向量和背面人车图片的特征向量之间的相似度值。
  4. 根据权利要求1～3中任一项所述的方法，其特征在于，所述方法还包括：
    通过监控设备获取多个人车图片的图像信息，其中，所述监控设备至少包括一个对人车进行前方拍摄的摄像机，以及一个对人车进行后方拍摄的摄像机，所述图像信息包括所述人车图片的抓拍地点和抓拍时间；
    根据所述抓拍地点和/或所述抓拍时间，将所述人车图片存储至相应类型的人车图片库；
    所述采用所述骑行者重识别模型提取人车图片库中中多个人车图片的特征 向量，包括：
    获取目标人车图片库，其中，所述目标人车图片库为所述用户指定类型的人车图片库；
    采用所述骑行者重识别模型提取所述目标人车图片库中多个所述人车图片的特征向量。
  5. 根据权利要求4所述的方法，其特征在于，所述方法还包括：
    根据所述进行前方拍摄的摄像机以及所述进行后方拍摄的摄像机之间的位置距离，设置所述人车图片库中人车图片的存储时间；
    当人车图片的存储时间超过所述设置的人车图片库中人车图片的存储时间时，将存储时间超过所述设置的存储时间的人车图片出库。
  6. 一种骑行者重识别装置，其特征在于，包括：
    学习模块，用于对行人重识别模型进行迁移学习，得到骑行者重识别模型；
    提取模块，用于采用所述骑行者重识别模型提取人车图片库中多个人车图片的特征向量；
    计算模块，用于计算所述特征向量之间的多个相似度值；
    识别模块，用于将最高的所述相似度值对应的至少两个所述人车图片识别为同一骑行者。
  7. 根据权利要求6所述的装置，其特征在于，所述人车图片包括每一骑行者的正面人车图和背面人车图；
    所述提取模块具体用于分别提取所述正面人车图的特征向量和所述背面人车图的特征向量；
    所述计算模块具体用于将所述正面人车图片的特征向量和所述背面人车图片的特征向量分别进行内积运算，得到所述多个特征向量之间的内积结果，其中，所述内积结果作为每一组正面人车图片的特征向量和背面人车图片的特征向量之间的相似度值。
  8. 根据权利要求6或7所述的装置，其特征在于，所述装置还包括：
    获取模块，用于通过监控设备获取多个人车图片的图像信息，其中，所述监控设备至少包括一个对人车进行前方拍摄的摄像机，以及一个对人车进行后方拍摄的摄像机，所述图像信息包括所述人车图片的抓拍地点和抓拍时间；
    存储模块，用于根据所述抓拍地点和/或所述抓拍时间，将所述人车图片存储至相应类型的人车图片库；
    所述提取模块包括：
    第一获取单元，用于获取目标人车图片库，其中，所述目标人车图片库为所述用户指定类型的人车图片库；
    提取单元，用于采用所述骑行者重识别模型提取所述目标人车图片库中多个所述人车图片的特征向量。
  9. 一种骑行者重识别设备，其特征在于，包括处理器、输入设备、输出设备和存储器，所述存储器用于存储计算机程序，所述计算机程序包括程序指令，所述处理器被配置用于调用所述程序指令，执行如权利要求1至5中任一权利要求所述的方法。
  10. 一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行如权利要求1至5中任一权利要求所述的方法。
PCT/CN2019/121517 2019-06-24 2019-11-28 一种骑行者重识别方法、装置及设备 WO2020258714A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910550548.6 2019-06-24
CN201910550548.6A CN110427814A (zh) 2019-06-24 2019-06-24 一种骑行者重识别方法、装置及设备

Publications (1)

Publication Number Publication Date
WO2020258714A1 true WO2020258714A1 (zh) 2020-12-30

Family

ID=68409506

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/121517 WO2020258714A1 (zh) 2019-06-24 2019-11-28 一种骑行者重识别方法、装置及设备

Country Status (2)

Country Link
CN (1) CN110427814A (zh)
WO (1) WO2020258714A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114821629A (zh) * 2021-01-27 2022-07-29 天津大学 一种基于神经网络并行训练架构进行交叉图像特征融合的行人重识别方法

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110427814A (zh) * 2019-06-24 2019-11-08 深圳云天励飞技术有限公司 一种骑行者重识别方法、装置及设备
CN113129597B (zh) * 2019-12-31 2022-06-21 深圳云天励飞技术有限公司 一种机动车道违法车辆识别方法及装置
CN111582277A (zh) * 2020-06-15 2020-08-25 深圳天海宸光科技有限公司 一种基于迁移学习的车牌识别系统及方法
CN112464922B (zh) * 2021-02-02 2021-05-28 长沙海信智能系统研究院有限公司 人车重识别及其模型训练方法、装置、设备及存储介质
CN113627352A (zh) * 2021-08-12 2021-11-09 塔里木大学 一种行人重识别方法和系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106778804A (zh) * 2016-11-18 2017-05-31 天津大学 基于类别属性迁移学习的零样本图像分类方法
CN109002761A (zh) * 2018-06-13 2018-12-14 中山大学新华学院 一种基于深度卷积神经网络的行人重识别监控系统
CN109117823A (zh) * 2018-08-31 2019-01-01 常州大学 一种基于多层神经网络的跨场景行人重识别的方法
CN109446898A (zh) * 2018-09-20 2019-03-08 暨南大学 一种基于迁移学习和特征融合的行人重识别方法
CN110427814A (zh) * 2019-06-24 2019-11-08 深圳云天励飞技术有限公司 一种骑行者重识别方法、装置及设备

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108021933B (zh) * 2017-11-23 2020-06-05 深圳市华尊科技股份有限公司 神经网络识别装置及识别方法
CN107944017B (zh) * 2017-12-11 2021-06-25 浙江捷尚视觉科技股份有限公司 一种视频中非机动车的检索方法
CN109344842A (zh) * 2018-08-15 2019-02-15 天津大学 一种基于语义区域表达的行人重识别方法
CN109657552B (zh) * 2018-11-16 2022-11-29 北京邮电大学 基于迁移学习实现跨场景冷启动的车型识别装置与方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106778804A (zh) * 2016-11-18 2017-05-31 天津大学 基于类别属性迁移学习的零样本图像分类方法
CN109002761A (zh) * 2018-06-13 2018-12-14 中山大学新华学院 一种基于深度卷积神经网络的行人重识别监控系统
CN109117823A (zh) * 2018-08-31 2019-01-01 常州大学 一种基于多层神经网络的跨场景行人重识别的方法
CN109446898A (zh) * 2018-09-20 2019-03-08 暨南大学 一种基于迁移学习和特征融合的行人重识别方法
CN110427814A (zh) * 2019-06-24 2019-11-08 深圳云天励飞技术有限公司 一种骑行者重识别方法、装置及设备

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114821629A (zh) * 2021-01-27 2022-07-29 天津大学 一种基于神经网络并行训练架构进行交叉图像特征融合的行人重识别方法

Also Published As

Publication number Publication date
CN110427814A (zh) 2019-11-08

Similar Documents

Publication Publication Date Title
WO2020258714A1 (zh) 一种骑行者重识别方法、装置及设备
US9652694B2 (en) Object detection method, object detection device, and image pickup device
US10212397B2 (en) Abandoned object detection apparatus and method and system
Abdullah et al. YOLO-based three-stage network for Bangla license plate recognition in Dhaka metropolitan city
TW202034270A (zh) 車輛事故的鑑定方法及裝置、電子設備
TWI712980B (zh) 理賠資訊提取方法和裝置、電子設備
WO2022021029A1 (zh) 检测模型训练方法、装置、检测模型使用方法及存储介质
CN111723773B (zh) 遗留物检测方法、装置、电子设备及可读存储介质
Razalli et al. Emergency vehicle recognition and classification method using HSV color segmentation
KR101493009B1 (ko) 전후면 차량 번호 인식 방법과 그 시스템
WO2019144416A1 (zh) 信息处理方法、系统、云处理设备以及计算机程序产品
CN104463232A (zh) 一种基于hog特征和颜色直方图特征的密度人群计数的方法
CN112614102A (zh) 一种车辆检测方法、终端及其计算机可读存储介质
CN110889388A (zh) 违章识别方法、装置、设备及存储介质
CN113111838A (zh) 行为识别方法及装置、设备和存储介质
TW201140470A (en) System and method for monitoring objects and key persons of the objects
CN111753601B (zh) 一种图像处理的方法、装置以及存储介质
CN111382606A (zh) 摔倒检测方法、摔倒检测装置和电子设备
WO2022228325A1 (zh) 行为检测方法、电子设备以及计算机可读存储介质
CN116311154A (zh) 一种基于YOLOv5模型优化的车辆检测与识别方法
KR20200084946A (ko) 차량의 주차면 분석을 위한 지능형 cctv 장치
Alkan et al. Driver Cell Phone Usage Violation Detection using License Plate Recognition Camera Images.
US20220405527A1 (en) Target Detection Methods, Apparatuses, Electronic Devices and Computer-Readable Storage Media
CN111914704B (zh) 一种三轮车载人的识别方法、装置、电子设备及存储介质
WO2022263904A1 (en) Target detection methods, apparatuses, electronic devices and computer-readable storage media

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19935577

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19935577

Country of ref document: EP

Kind code of ref document: A1