CN108764167B - Space-time correlated target re-identification method and system - Google Patents
Space-time correlated target re-identification method and system Download PDFInfo
- Publication number
- CN108764167B CN108764167B CN201810543066.3A CN201810543066A CN108764167B CN 108764167 B CN108764167 B CN 108764167B CN 201810543066 A CN201810543066 A CN 201810543066A CN 108764167 B CN108764167 B CN 108764167B
- Authority
- CN
- China
- Prior art keywords
- time
- camera
- target
- candidate
- probability
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention relates to a space-time associated target re-identification method, which combines the pixel motion rate of a target in video data to estimate the probability distribution of the time length of the target crossing two adjacent cameras with a certain distance in each section of video data; based on the time length probability, the candidate targets appearing in the video can be screened and preprocessed, the candidate targets exceeding a reasonable crossing time interval are filtered, and the probability that similar targets are mistakenly matched as tracking targets is reduced. The invention also relates to a space-time associated target re-identification system. The matching result generated by the method is constrained by the space-time position and the target motion information, and compared with the original unconstrained matching structure which only depends on visual features, the accuracy of re-identification can be effectively improved.
Description
Technical Field
The invention relates to a target re-identification technology, in particular to a space-time associated target re-identification method and a corresponding target re-identification system.
Background
The object re-identification problem is a problem of judging whether a specific object exists in an image or a video sequence by utilizing a computer vision technology. Specifically, when a specific target is tracked by using a video, since a video source is from a fixed position, cross-video relay tracking is required when the target leaves a visual field, and at this time, a problem that the specific target is detected in other video sources belongs to a target re-identification problem.
The object re-identification problem uses the visual features of the object obtained from the image to perform feature matching and give possible candidate objects. Because the features between different targets have similarity, matching using the features may also result in a situation where the candidate target is not a true tracking target.
Through the search of the prior art, although the current target re-identification technology is widely applied to relay tracking, the target re-identification module therein mostly only utilizes the visual characteristic information of the target in the image. Such as patent No. CN201210201004.7, patent name: although the GPS and GIS information, namely the spatial information, is combined for screening, the utilization of the time and spatial information only stays at the stage of drawing a GIS map and a target motion track, the time information and the spatial information are not directly used for the technology of improving the accuracy of target re-identification, and a target re-identification module is still only carried out based on the visual characteristics and still can generate a large number of candidate targets with low possibility in an unreasonable time interval due to the similarity of the visual characteristics.
In addition, although there is a patent combining target time and position information, such as application (patent) No. CN201610221993.4, which is a target re-identification method based on space-time constraint, which gives a shortest motion time to each pair of adjacent cameras and gives a probability that a target appears at that time according to weber distribution and measured candidate target occurrence times, and gives a joint probability distribution in combination with visual matching features, this method does not consider the real-time motion state of the target, and the probability of the target appearing at a specific time depends on the measurement time completely, and there are two main problems: firstly, the time description according to the patent is the local time of each camera with its own clock, but is not described as uniform global positioning time service information, so that the time is not synchronized due to the clock synchronization of different cameras, and the whole prediction result is directly influenced. In addition, more importantly, the patent only considers the common problems of crossing time, space distance and the like, and does not consider the individual problems of target displacement direction, speed and the like: directly appointing the shortest motion time for the adjacent cameras, wherein the information amount represented by the value is not essentially different from the path distance of the adjacent cameras on the GIS information, and is only represented by space distance constraint; in practical situations, different targets have individual differences such as speed, the moving speed of some targets is high, the moving speed of some targets is low, and if the time estimation of the target appearing in the visual field of the camera is to be given more accurately, the possible arrival time of different targets must be calculated and predicted by combining the real-time moving information of the targets. For example, two objects with similar visual characteristics are in the field of view of the camera a, and are moved to the camera B, one is a tracking object, and the other is not, according to the algorithm of the patent, the obtained probability distribution of the entering time of the two objects is completely consistent weber distribution, but if the two objects move faster and slower, the time of the two objects reaching the camera B is greatly different, and the conclusion is completely different from that obtained by the method provided by the application (patent) No. CN201610221993.4, so that the candidate object screening is inaccurate.
In further retrieval, a target re-identification method which combines visual features and space-time constraints and motion information of target individuals and adopts global unified time service to perform time synchronization is not found at present.
Disclosure of Invention
Aiming at the current situation that the existing target re-identification method mainly utilizes target visual characteristics and the space-time correlation information is not sufficiently utilized, a space-time correlation target re-identification method is provided.
In order to achieve the purpose, the invention adopts the following technical scheme: the method combines the pixel motion rate of the target in the video data to estimate the probability distribution of the time length of the target crossing two adjacent cameras with a certain distance in each section of video data; based on the time length probability, the candidate targets appearing in the video can be screened and preprocessed, the candidate targets exceeding a reasonable crossing time interval are filtered, the probability that similar targets are mistakenly matched into tracking targets is reduced, the generated matching result is constrained by the space-time position and the target motion information, and compared with the original unconstrained matching structure which only depends on visual features, the accuracy of re-recognition can be effectively improved.
According to a first object of the present invention, there is provided a method for re-identifying spatio-temporal correlated targets, comprising:
for camera CiRecording the initial time t of a selected object a to be checkedsAnd starting to track, and acquiring the pixel motion rate V of the image by using the tracking resultaAnd the movement direction information, and extracting the visual characteristics of the object a to be checked for re-identification;
obtaining null by using GIS informationOn the room and the camera CiM camera sets which are adjacent and matched with the advancing direction of the object a to be checked, and M adjacent cameras in the setjAnd a camera C is obtained by utilizing GIS or manual measurementiTo camera MjActual path length L ofi,j；
Object a to be checked secondary camera CiTo the adjacent camera MjCrossing time ti,jAt path length Li,jIn certain cases, a linear rate-time model is used
Prediction is made, using the predicted crossing time
Make a camera MjIn the time interval
As a candidate for re-recognition, wherein
Statistical standard deviation of (1), i.e. assumption
Obeying normal distribution, and obtaining a standard deviation of the normal distribution by using training data;
to camera MjExtracts its visual characteristics for re-recognition, and first appears in the camera M with the candidate object bjGlobally uniform time service information acquired by time synchronization and used as the target in the camera MjTime of occurrence te(ii) a Obtaining each candidate target b in the camera M by utilizing motion tracking calculationjVelocity V of pixel motionbPredicting the time of crossing by using a linear rate-time model
To camera MjEach pair of (V)b,Li,j) Crossing time of candidate object b
Assuming obedience to a mean value of tmeanVariance is σ2Based on the normal distribution, the candidate object b is calculated at a given value (V)b,Li,j) At time t under the conditioneAppear in camera MiProbability P oftimespace((te-ts))～N(tmean，σ2)；
Based on the visual characteristics of the object a to be checked and the candidate object b, calculating the recognition probability P of each candidate object b by using an object re-recognition methodvision(ii) a P of each candidate target bvisionAnd PtimespaceMultiplying, and taking the obtained product as the target re-identification probability, and sequencing according to the probability to obtain the final result of re-identification.
Preferably, the obtaining of the pixel motion rate Va and the motion direction information by using the tracking result means: the pixel motion rate is the motion speed taking image pixels as unit distance and image acquisition time intervals as time units, and does not relate to the actual target motion rate, so that calibration is not needed, and additional acquisition equipment is not needed; the moving direction is the moving direction of the target, which is the section where the target moving direction falls, that is, the section is taken as the target, and the image is divided into one direction section every N degrees on the plane space by combining the camera calibration information.
Preferably, the pixel motion velocity V is obtained by using the tracking resultaAnd motion direction information, wherein: the pixel motion rate is the motion speed taking image pixels as unit distance and image acquisition time intervals as time units, and does not relate to the actual target motion rate; the moving direction is the moving direction of the target, which is the moving direction of the target with the interval as the target, by combining the camera calibration information and dividing the image into one direction interval every N degrees on the plane space。
Preferably, the space and the camera C are obtained by utilizing GIS informationiThe M camera sets which are adjacent and matched with the advancing direction of the object a to be checked refer to: taking the moving direction interval of the object a to be checked as the center, adding two intervals adjacent in space as the search range of the adjacent camera, and taking the search range as the search range of the adjacent camera, wherein the search range is positioned in the direction range and is positioned at the camera CiThe cameras which are adjacent in space form an adjacent camera set matched with the advancing direction of the object a to be checked.
Preferably, the object a to be checked is selected from a camera CiTo the adjacent camera MjCrossing time ti,jAt path length Li,jIn certain cases, a linear rate-time model is used
Predicted, means:
for a pixel velocity of VaTarget of (2), crossing path Li,jSatisfies the linear relationship:
in the formula: alpha and beta are model parameters, and the linear relation model is fitted by using training data acquired under a line to obtain each parameter of the model; the model parameters can be dynamically learned and updated according to the data acquired on line.
Preferably, when the image is collected, the global unified time service information obtained by reading the GPS or Beidou global time service module of the collection equipment or other global time service equipment and modules is used as the generation time of the equipment when the equipment collects the image of the frame, and the candidate target b is used as the generation time of the MjThe generation time of the first appearance image is taken as the appearance time te。
Preferably, the time of crossing is predicted by using a linear rate-time model
The method comprises the following steps:
for a pixel velocity of VbTarget of (2), crossing path Li,jSatisfies the linear relationship:
in the formula: eta and theta are model parameters, and the linear relation model is fitted by using training data acquired offline to obtain each parameter of the model; the model parameters can be dynamically learned and updated according to the data acquired on line.
Preferably, the pair of cameras MjEach pair of (V)b,Li,j) Crossing time of candidate object b
Assuming obedience to a mean value of tmeanVariance is σ2Normal distribution of (a) means:
quantizing the pixel motion rate of the candidate object b into M rate levels and a pixel rate V falling into one rate levelbUsing the average velocity V of the class intervalmeanInstead of the original rate; for each given combination of conditions (V)mean，Li,j) Crossing time of candidate object b
Obeying a parameter of (t)mean，σ2) Normal distribution of (a), a parameter (t) of the normal distributionmean，σ2) Training and fitting are carried out on training data acquired offline; the model parameters can be dynamically learned and updated according to the data acquired online.
Preferably, said calculating candidate object b is given (V) based on the distributionb,Li,j) At time t under the conditioneAppear in camera MiProbability of (2)Ptimespace((te-ts))～N(tmean，σ2) The method comprises the following steps:
assuming that the object b is an object to be checked, the slave camera CiTo camera MjReal crossing time tb＝te-tsDue to the crossing time
Obeying normal distribution, calculating the crossing time C by using a normal distribution model and the real crossing timeiTo MjProbability of (2)
And using the probability as a candidate target b at a given value (V)b,Li,j) At time t under the conditioneAppear in camera MiProbability of (2)
According to a second object of the present invention, there is provided a spatiotemporal correlated target re-identification system, comprising:
a target detection and tracking module: for camera CiRecording the initial time t of a selected object a to be checkedsAnd starting to track, and acquiring the pixel motion rate V of the image by using the tracking resultaAnd motion direction information;
visual feature extraction and re-identification module: extracting visual features of each object to be checked and the candidate object for re-identification based on the result of the object detection and tracking module; spatial and camera C obtained by utilizing GIS informationiM camera sets which are adjacent and matched with the advancing direction of the object a to be checked, and M adjacent cameras in the setjAnd a camera C is obtained by utilizing GIS or manual measurementiTo camera MjActual path length L ofi,j(ii) a Object a to be checked secondary camera CiTo the adjacent camera MjCrossing time ti,jAt path length Li,jUnder certain circumstances, linear velocity is used-time model
Predicting to obtain; using the predicted crossing time
Make a camera MjIn the time interval
As a candidate for re-recognition, wherein
Statistical standard deviation of (1), i.e. assumption
Obeying normal distribution, and obtaining a standard deviation of the normal distribution by using training data;
the space-time association and target screening module: to camera MjWith which it first appears at the camera MjGlobally uniform time service information acquired by time synchronization and used as the target in the camera MjTime of occurrence te(ii) a Obtaining each candidate target at the camera M by utilizing motion tracking calculationjVelocity V of pixel motionb(ii) a Similarly, a linear rate-time model is used to predict the time it spans
To camera MjEach pair of (V)b,Li,j) Crossing time of candidate object b
Assuming obedience to a mean value of tmeanVariance is σ2Based on the normal distribution of (a), b is calculated at a given value (V)b,Li,j) At time t under the conditioneAppear in camera MiProbability P oftimespace((te-ts))～N(tmean，σ2)；
The recognition probability calculation and reordering module: based on the visual information characteristics of the object a to be checked and the candidate object b, calculating the recognition probability P of each candidate object b by using an object re-recognition methodvision(ii) a P of each candidate target bvisionAnd PtimespaceMultiplying, and taking the obtained product as the target re-identification probability, and sequencing according to the probability to obtain the final result of re-identification.
The visual features of the object a to be checked and the candidate object b include, but are not limited to, traditional artificial design features such as color texture and the like, and depth features learned by a deep neural network model. The invention uses the video data combined with the actual distance change information to analyze, calculates the movement speed of the target in real time, predicts the time of the target, filters the candidate target beyond the reasonable interval range and improves the accuracy of the target re-identification problem.
Compared with the prior art, the embodiment of the invention has the following effects:
the method and the system provided by the invention are a method for improving the accuracy of the re-identification process on the problem of re-identification of the target which is generally carried out only by using image data or time and position data by combining target motion information.
According to the method and the system, the relevance relation of the cross-camera video data in time and space and the motion information of the target individual are utilized to correlate the re-recognition candidate target in time and space, and the non-correlated candidate target in time and space is filtered or reduced, so that a more accurate candidate target range is obtained, and the target re-recognition precision is effectively improved.
Drawings
FIG. 1 is a block diagram of an embodiment of spatial-temporal correlation object re-identification according to the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that variations and modifications can be made by persons skilled in the art without departing from the spirit of the invention. All falling within the scope of the present invention.
The method combines the pixel motion rate of the target in the video data to estimate the probability distribution of the time length of the target crossing two adjacent cameras with a certain distance in each section of video data; based on the time length probability, the candidate targets appearing in the video can be screened and preprocessed, the candidate targets exceeding a reasonable crossing time interval are filtered, and the probability that similar targets are mistakenly matched as tracking targets is reduced.
Specifically, in the embodiment of the method for re-identifying the space-time associated target of the present invention, the following steps may be referred to:
s1: for camera CiRecording the initial time t of a selected object a to be checkedsAnd starting to track, and acquiring the pixel motion rate V of the image by using the tracking resultaAnd the movement direction information, and extracting the visual characteristics used for re-identification; the visual features include, but are not limited to, traditional artificial design features such as color textures and the like, and depth features learned by a deep neural network model.
S2: obtaining spatial sum C by using GIS informationiM camera sets which are adjacent and matched with the advancing direction of the object to be checked; for each adjacent camera M in the setjUsing GIS or manual measurements to obtain Ci—〉MjActual path length L ofi,j；
S3: object to be checked a from CiTo the adjacent camera MjCrossing time ti,jAt path length Li,jIn certain cases, a linear rate-time model may be utilized
And (6) obtaining a prediction. Using the predicted crossing time
Can be combined with MjIn the time interval
The target of (2) is used as a candidate target for re-identification;
s4: to MjOn the one hand, extracting visual features of the candidate target b for re-recognition, wherein the visual features include, but are not limited to, traditional artificial design features such as color textures and the like, and deep features learned by a deep neural network model; on the other hand, the first occurrence in M with candidate object bjGlobally unified time service information acquired by time synchronization and taken as the target at MjTime of occurrence te(ii) a Obtaining the position M of each candidate target by utilizing motion tracking calculationjVelocity V of pixel motionb(ii) a Similarly, a linear rate-time model is used to predict the time it spans
To MjEach pair of (V)b,Li,j) Crossing time of candidate object b
One can assume obey to a mean value of tmeanVariance is σ2Is normally distributed. Based on this distribution, b is calculated at a given (V)b,Li,j) At time t under the conditionePresent at MiProbability P oftimespace((te-ts))～N(tmean，σ2)；
S5: based on the visual characteristics of the object a to be checked and the candidate object b, calculating the recognition probability P of each candidate object b by using an object re-recognition methodvision(ii) a P of each candidate target bvisionAnd PtimespaceMultiplying, and taking the obtained product as the target re-identification probability, and sequencing according to the probability to obtain the final result of re-identification.
Of course, it will be understood by those skilled in the art that the execution sequence of the steps in the above embodiments may also be adjusted according to actual situations, and the steps are not required to be strictly performed.
In this embodiment, in S1: acquiring the pixel motion rate Va and motion direction information thereof by using the tracking result, which means that: the pixel motion rate is the motion speed taking image pixels as unit distance and image acquisition time intervals as time units, and does not relate to the actual target motion rate, so that calibration is not needed, and additional acquisition equipment is not needed; the moving direction is the moving direction of the target, which is the section where the target moving direction falls, that is, the section is taken as the target, and the image is divided into one direction section every N degrees on the plane space by combining the camera calibration information.
In this embodiment, in S2, obtaining M camera sets spatially adjacent to C and matched with the advancing direction of the target to be checked by using GIS information means: and taking the target motion direction interval as a center, adding two spatially adjacent intervals to serve as the search range of adjacent cameras, and forming an adjacent camera set matched with the advancing direction of the target to be checked, wherein the adjacent cameras are positioned in the direction range and are spatially adjacent to the C.
In this embodiment, in S3, the object a to be checked is selected from CiTo the adjacent camera MjCrossing time ti,jAt path length Li,jIn certain cases, a linear rate-time model may be utilized
The prediction is that: for a pixel velocity of VaTarget of (2), crossing path Li,jSatisfies the linear relationship:
alpha and β are model parameters, the linear relation model can be fitted by using the training data acquired under the line to obtain each parameter of the model, and the model parameters can be dynamically learned and updated according to the data acquired on the line.
In this embodiment, in S4, M is the pairjWith which it first appears at MjGlobal unified time service letter acquired by time synchronizationTo the target at MjTime of occurrence teSpecifically, the method comprises the following steps: when the image is collected, the global unified time service information is obtained by reading a GPS or Beidou global time service module of the collection equipment or other global time service equipment and modules, is used as the generation time of the equipment when the image of the current frame is collected, and the target b is positioned at MjThe generation time of the first appearance image is used as the appearance time t of be。
In this embodiment, in the step S4, the time spanned by the time is predicted by using a linear rate-time model
The method specifically comprises the following steps: for a pixel velocity of VbTarget of (2), crossing path Li,jSatisfies the linear relationship:
η and theta are model parameters, the linear relation model can be fitted by using the training data collected off-line to obtain each parameter of the model, and the model parameters can be dynamically learned and updated according to the data collected on-line.
In this embodiment, in S4, M is the pairjEach pair of (V)b,Li,j) Crossing time of candidate object b
One can assume obey to a mean value of tmeanVariance is σ2The normal distribution of (a) specifically means: quantizing the pixel motion rate of the candidate target b into M rate levels; pixel rate V falling on one rate levelbUsing the average velocity V of the class intervalmeanInstead of the original rate; for each given combination of conditions (V)mean，Li,j) Crossing time of object b
Obeying a parameter of (t)mean，σ2) Is normally distributed, thisParameter of normal distribution (t)mean，σ2) Training and fitting are carried out on training data acquired offline; the model parameters can be dynamically learned and updated according to the data acquired online.
In this embodiment, in S4, b is calculated based on the distribution at a given value (V)b,Li,j) At time t under the conditionePresent at MiProbability P oftimespace((te-ts))～N(tmean，σ2) The method comprises the following steps: assuming object b as the object to be looked up, it is driven from CiTo MjReal crossing time tb＝te-ts(ii) a Because of crossing time
Obeying normal distribution, calculating the crossing time C by using a normal distribution model and the real crossing timei--〉MjProbability of (2)
And using the probability as b at a given value (V)b,Li,j) At time t under the conditionePresent at MiProbability of (2)
On the basis of the method, the accuracy rate of target re-identification is improved by modeling by utilizing motion information. In another system embodiment, the target re-identification system mainly comprises a target detection and tracking module, a visual feature extraction and re-identification module, a spatio-temporal association and target screening module, and an identification probability calculation and reordering module, wherein:
a target detection and tracking module: for camera CiRecording the initial time t of a selected object a to be checkedsAnd starting to track, and acquiring the pixel motion rate V of the image by using the tracking resultaAnd motion direction information;
visual feature extraction and re-identification module: object-based detectionExtracting visual features for re-identification of each object to be checked and the candidate object from the result of the tracking module, wherein the visual features comprise traditional artificial design features such as but not limited to color textures and the like and depth features learned by a deep neural network model; obtaining spatial and camera C by using GIS informationiM camera sets which are adjacent and matched with the advancing direction of the object a to be checked, and M adjacent cameras in the setjAnd a camera C is obtained by utilizing GIS or manual measurementiTo camera MjActual path length L ofi,j(ii) a Object a to be checked secondary camera CiTo the adjacent camera MjCrossing time ti,jAt path length Li,jIn certain cases, a linear rate-time model is used
Predicting to obtain; using the predicted crossing time
Make a camera MjIn the time interval
As a candidate for re-recognition, wherein
Statistical standard deviation of (1), i.e. assumption
Obeying normal distribution, and obtaining a standard deviation of the normal distribution by using training data;
the space-time association and target screening module: to camera MjWith which it first appears at the camera MjGlobally uniform time service information acquired by time synchronization and used as the target in the camera MjTime of occurrence te(ii) a Benefit toObtaining each candidate target at the camera M by using motion tracking calculationjVelocity V of pixel motionb(ii) a Similarly, a linear rate-time model is used to predict the time it spans
To camera MjEach pair of (V)b,Li,j) Crossing time of candidate object b
Assuming obedience to a mean value of tmeanVariance is σ2Based on the normal distribution of (a), b is calculated at a given value (V)b,Li,j) At time t under the conditioneAppear in camera MiProbability P oftimespace((te-ts))～N(tmean，σ2)；
The recognition probability calculation and reordering module: based on the visual information characteristics of the object a to be checked and the candidate object b, calculating the recognition probability P of each candidate object b by using an object re-recognition methodvision(ii) a P of each candidate target bvisionAnd PtimespaceMultiplying, and taking the obtained product as the target re-identification probability, and sequencing according to the probability to obtain the final result of re-identification.
Referring to FIG. 1, in one embodiment:
the invention takes the cross-camera pedestrian re-identification in the video monitoring system as an embodiment and carries out application description. Cross-camera pedestrian re-identification in a video monitoring system refers to that when a specific pedestrian target a appearing in an initial camera C leaves C and enters the visual field of other cameras under a video monitoring network, based on the space-time correlation target re-identification method, the cross-camera space-time correlation is utilized to carry out candidate target constraint and is fused with the visual information of the target, and the visual characteristics are assisted in a joint probability mode to determine the probability that each candidate target and the target a to be detected are the same target.
Setting an initial camera C, and sending the collected video into a target for detectionThe tracking module is used for tracking the selected pedestrian target a as a target to be searched; furthermore, advanced tracking methods such as related filtering combined with depth features can be used, the change situation of the center of the tracking frame along with time is calculated by utilizing the tracking result, namely the target tracking frame on continuous time, so that the motion rate V of the target a pixel can be obtainedaAnd motion direction information. Similarly, the target camera B adjacent to the target camera C performs detection and tracking analysis by using the same target detection and tracking module, and obtains information such as a pixel motion rate of each detected candidate target.
The target camera C and the target camera B to be checked, the target detection frame output by the target detection and tracking modules of the target camera C and the target camera B, and the like are sent to the visual feature extraction and re-identification module, the visual features are extracted, and the re-identification probability P is carried out based on the featuresvisionAnd (4) calculating.
Target camera C and target camera B to be checked, and target a pixel motion rate V output by target detection and tracking modules of the two camerasaAnd the motion direction information is transmitted to a space-time correlation and target screening module, candidate targets are screened based on the space-time correlation by combining information such as GIS (geographic information System), and the recognition probability P under the space-time constraint of each candidate target is calculatedtimespace。
The two probabilities output by the visual feature extraction and re-identification module and the space-time association and target screening module are sent to the identification probability calculation and reordering module together, probability multiplication is carried out to obtain a new identification probability, and reordering is carried out based on the probability to obtain a final re-identification result.
The working process and the realized function of the system in the embodiment are as follows:
(1) and generating time information containing target motion information, position information and unified time service for each detection target and each candidate target, and providing accurate, specific and synchronous space-time information for video analysis.
(2) The target re-identification method is combined with visual features, space-time constraints and motion information of target individuals, and adopts global unified time service to perform time synchronization.
The specific implementation technology of each module in the system according to the above embodiment of the present invention may adopt the corresponding technology in the step corresponding to the target re-identification method, and is not described herein again.
In summary, the target re-identification method and the target re-identification system provided by the invention combine visual characteristics, space-time constraints and motion information of target individuals, adopt global unified time service to carry out time synchronization, and are fused to improve the target re-identification accuracy. The matching result generated by the method is constrained by the space-time position and the target motion information, and compared with the original unconstrained matching structure which only depends on visual features, the accuracy of re-identification can be effectively improved.
It should be noted that, the steps in the target re-identification method provided by the present invention may be implemented by using corresponding modules, devices, units, and the like in the target re-identification system, and those skilled in the art may refer to the technical solution of the system to implement the step flow of the method, that is, the embodiment in the system may be understood as a preferred example for implementing the method, and details are not described herein.
Those skilled in the art will appreciate that, in addition to implementing the system and its various devices provided by the present invention in purely computer readable program code means, the method steps can be fully programmed to implement the same functions by implementing the system and its various devices in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system and various devices thereof provided by the present invention can be regarded as a hardware component, and the devices included in the system and various devices thereof for realizing various functions can also be regarded as structures in the hardware component; means for performing the functions may also be regarded as structures within both software modules and hardware components for performing the methods.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes and modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention.
Claims (9)
1. A space-time associated target re-identification method is characterized by comprising the following steps:
for camera CiRecording the initial time t of a selected object a to be checkedsAnd starting to track, and acquiring the pixel motion rate V of the image by using the tracking resultaAnd the movement direction information, and extracting the visual characteristics of the object a to be checked for re-identification;
spatial and camera C obtained by utilizing GIS informationiM camera sets which are adjacent and matched with the advancing direction of the object a to be checked, and M adjacent cameras in the setjAnd a camera C is obtained by utilizing GIS or manual measurementiTo camera MjActual path length L ofi,j；
Object a to be checked secondary camera CiTo the adjacent camera MjCrossing time ti,jAt path length Li,jIn certain cases, a linear rate-time model is used
Prediction is made, using the predicted crossing time
Make a camera MjIn the time interval
As a candidate for re-recognition, wherein
Statistical standard deviation of (1), i.e. assumption
Obeying normal distribution, and obtaining a standard deviation of the normal distribution by using training data;
to camera MjIs extracted for re-recognitionOther visual characteristics, first appearing in camera M with candidate object bjGlobally uniform time service information acquired by time synchronization and used as the target in the camera MjTime of occurrence te(ii) a Obtaining each candidate target b in the camera M by utilizing motion tracking calculationjVelocity V of pixel motionbPredicting the time of crossing by using a linear rate-time model
To camera MjEach pair of (V)b,Li,j) Crossing time of candidate object b
Assuming obedience to a mean value of tmeanVariance is σ2Based on the normal distribution, the candidate object b is calculated at a given value (V)b,Li,j) At time t under the conditioneAppear in camera MjProbability P oftimespace((te-ts))～N(tmean，σ2)；
Based on the visual characteristics of the object a to be checked and the candidate object b, calculating the recognition probability P of each candidate object b by using an object re-recognition methodvision(ii) a P of each candidate target bvisionAnd PtimespaceMultiplying, and taking the obtained product as the target re-identification probability, and sequencing according to the probability to obtain the final result of re-identification.
2. The method as claimed in claim 1, wherein the pixel motion velocity V is obtained by using the tracking resultaAnd motion direction information, wherein: the pixel motion rate is the motion speed taking image pixels as unit distance and image acquisition time intervals as time units, and does not relate to the actual target motion rate; the direction of movement is then combinedThe camera marks information, and divides the image into a direction section every N degrees on a plane space, wherein the target motion direction falls into which section, namely the section is taken as the motion direction of the target.
3. The method for re-identifying spatio-temporal correlated targets as claimed in claim 1, wherein said obtaining of spatial and camera C by using GIS informationiThe M camera sets which are adjacent and matched with the advancing direction of the object a to be checked refer to: taking the moving direction interval of the object a to be checked as the center, adding two intervals adjacent in space as the search range of the adjacent camera, and taking the search range as the search range of the adjacent camera, wherein the search range is positioned in the direction range and is positioned at the camera CiThe cameras which are adjacent in space form an adjacent camera set matched with the advancing direction of the object a to be checked.
4. The method for re-identifying spatio-temporal correlated targets as claimed in claim 1, characterized in that said target a to be checked is selected from a camera CiTo the adjacent camera MjCrossing time ti,jAt path length Li,jIn certain cases, a linear rate-time model is used
Predicted, means:
for a pixel velocity of VaTarget of (2), crossing path Li,jSatisfies the linear relationship:
in the formula: alpha and beta are model parameters, and the linear relation model is fitted by using training data acquired under a line to obtain each parameter of the model; the model parameters can be dynamically learned and updated according to the data acquired on line.
5. The spatio-temporal correlation target re-identification method according to claim 1, characterized in thatWhen the image is collected, the global unified time service information is obtained by reading a GPS or Beidou global time service module of the collection equipment or other global time service equipment and modules, is used as the generation time of the equipment when the current frame of image is collected, and the candidate target b is used as the generation time of the camera MjThe generation time of the first appearance image is taken as the appearance time te。
6. The method as claimed in claim 1, wherein the time of crossing is predicted by linear rate-time model
The method comprises the following steps:
for a pixel velocity of VbTarget of (2), crossing path Li,jSatisfies the linear relationship:
in the formula: eta and theta are model parameters, and the linear relation model is fitted by using training data acquired offline to obtain each parameter of the model; the model parameters can be dynamically learned and updated according to the data acquired on line.
7. The spatiotemporal correlated target re-identification method according to claim 1, characterized in that the pair of cameras MjEach pair of (V)b,Li,j) Crossing time of candidate object b
Assuming obedience to a mean value of tmeanVariance is σ2Normal distribution of (a) means:
quantizing the pixel motion rate of the candidate object b into S rate levels, and the pixel rate V falling into one rate levelbUsing the average velocity V of the class intervalmeanTo replace the originalA rate; for each given combination of conditions (V)mean，Li,j) Crossing time of candidate object b
Obeying a parameter of (t)mean，σ2) Normal distribution of (a), a parameter (t) of the normal distributionmean，σ2) Training and fitting are carried out on training data acquired offline; the model parameters can be dynamically learned and updated according to the data acquired online.
8. The method of claim 1, wherein the candidate object b is calculated based on the distributionb,Li,j) At time t under the conditioneAppear in camera MjProbability P oftimespace((te-ts))～N(tmean，σ2) The method comprises the following steps:
assuming that the object b is an object to be checked, the slave camera CiTo camera MjReal crossing time tb＝te-tsDue to the crossing time
Obeying normal distribution, calculating the crossing time C by using a normal distribution model and the real crossing timeiTo MjProbability of (2)
And using the probability as a candidate target b at a given value (V)b,Li,j) At time t under the conditioneAppear in camera MjProbability of (2)
9. A spatiotemporal correlated target re-identification system, comprising:
a target detection and tracking module: for camera CiRecording the initial time t of a selected object a to be checkedsAnd starting to track, and acquiring the pixel motion rate V of the image by using the tracking resultaAnd motion direction information;
visual feature extraction and re-identification module: extracting visual features of each object to be checked and the candidate object for re-identification based on the result of the object detection and tracking module; spatial and camera C obtained by utilizing GIS informationiM camera sets which are adjacent and matched with the advancing direction of the object a to be checked, and M adjacent cameras in the setjAnd a camera C is obtained by utilizing GIS or manual measurementiTo camera MjActual path length L ofi,j(ii) a Object a to be checked secondary camera CiTo the adjacent camera MjCrossing time ti,jAt path length Li,jIn certain cases, a linear rate-time model is used
Predicting to obtain; using the predicted crossing time
Make a camera MjIn the time interval
As a candidate for re-recognition, wherein
Statistical standard deviation of (1), i.e. assumption
Obeying normal distribution, and obtaining a standard deviation of the normal distribution by using training data;
the space-time association and target screening module: to camera MjEach candidate object b in (1), using its headSecond occurrence in the camera MjGlobally uniform time service information acquired by time synchronization and used as the target in the camera MjTime of occurrence te(ii) a Obtaining each candidate target at the camera M by utilizing motion tracking calculationjVelocity V of pixel motionb(ii) a Similarly, a linear rate-time model is used to predict the time it spans
To camera MjEach pair of (V)b,Li,j) Crossing time of candidate object b
Assuming obedience to a mean value of tmeanVariance is σ2Based on the normal distribution of (a), b is calculated at a given value (V)b,Li,j) At time t under the conditioneAppear in camera MjProbability P oftimespace((te-ts))～N(tmean，σ2)；
The recognition probability calculation and reordering module: based on the visual information characteristics of the object a to be checked and the candidate object b, calculating the recognition probability P of each candidate object b by using an object re-recognition methodvision(ii) a P of each candidate target bvisionAnd PtimespaceMultiplying, and taking the obtained product as the target re-identification probability, and sequencing according to the probability to obtain the final result of re-identification.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810543066.3A CN108764167B (en) | 2018-05-30 | 2018-05-30 | Space-time correlated target re-identification method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810543066.3A CN108764167B (en) | 2018-05-30 | 2018-05-30 | Space-time correlated target re-identification method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108764167A CN108764167A (en) | 2018-11-06 |
| CN108764167B true CN108764167B (en) | 2020-09-29 |
Family
ID=64004566
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810543066.3A Active CN108764167B (en) | 2018-05-30 | 2018-05-30 | Space-time correlated target re-identification method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108764167B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109902551A (en) * | 2018-11-09 | 2019-06-18 | 阿里巴巴集团控股有限公司 | The real-time stream of people's statistical method and device of open scene |
| CN109558831B (en) * | 2018-11-27 | 2023-04-07 | 成都索贝数码科技股份有限公司 | Cross-camera pedestrian positioning method fused with space-time model |
| CN109598240B (en) * | 2018-12-05 | 2019-11-05 | 深圳市安软慧视科技有限公司 | Video object quickly recognition methods and system again |
| CN110110598A (en) * | 2019-04-01 | 2019-08-09 | 桂林电子科技大学 | The pedestrian of a kind of view-based access control model feature and space-time restriction recognition methods and system again |
| CN110087039B (en) * | 2019-04-30 | 2021-09-14 | 苏州科达科技股份有限公司 | Monitoring method, device, equipment, system and storage medium |
| CN110264497B (en) * | 2019-06-11 | 2021-09-17 | 浙江大华技术股份有限公司 | Method and device for determining tracking duration, storage medium and electronic device |
| CN110728702B (en) * | 2019-08-30 | 2022-05-20 | 深圳大学 | High-speed cross-camera single-target tracking method and system based on deep learning |
| CN110796074B (en) * | 2019-10-28 | 2022-08-12 | 桂林电子科技大学 | Pedestrian re-identification method based on space-time data fusion |
| CN111061825B (en) * | 2019-12-10 | 2020-12-18 | 武汉大学 | Method for identifying matching and correlation of space-time relationship between mask and reloading camouflage identity |
| CN111178284A (en) * | 2019-12-31 | 2020-05-19 | 珠海大横琴科技发展有限公司 | Pedestrian re-identification method and system based on spatio-temporal union model of map data |
| CN111666823B (en) * | 2020-05-14 | 2022-06-14 | 武汉大学 | Pedestrian re-identification method based on individual walking motion space-time law collaborative identification |
| CN113688776B (en) * | 2021-09-06 | 2023-10-20 | 北京航空航天大学 | Space-time constraint model construction method for cross-field target re-identification |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8098888B1 (en) * | 2008-01-28 | 2012-01-17 | Videomining Corporation | Method and system for automatic analysis of the trip of people in a retail space using multiple cameras |
| CN103810476A (en) * | 2014-02-20 | 2014-05-21 | 中国计量学院 | Method for re-identifying pedestrians in video monitoring network based on small-group information correlation |
| CN107133575A (en) * | 2017-04-13 | 2017-09-05 | 中原智慧城市设计研究院有限公司 | A kind of monitor video pedestrian recognition methods again based on space-time characteristic |
| CN107255468A (en) * | 2017-05-24 | 2017-10-17 | 纳恩博（北京）科技有限公司 | Method for tracking target, target following equipment and computer-readable storage medium |
| CN107545256A (en) * | 2017-09-29 | 2018-01-05 | 上海交通大学 | A kind of camera network pedestrian recognition methods again of combination space-time and network consistency |
-
2018
- 2018-05-30 CN CN201810543066.3A patent/CN108764167B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8098888B1 (en) * | 2008-01-28 | 2012-01-17 | Videomining Corporation | Method and system for automatic analysis of the trip of people in a retail space using multiple cameras |
| CN103810476A (en) * | 2014-02-20 | 2014-05-21 | 中国计量学院 | Method for re-identifying pedestrians in video monitoring network based on small-group information correlation |
| CN107133575A (en) * | 2017-04-13 | 2017-09-05 | 中原智慧城市设计研究院有限公司 | A kind of monitor video pedestrian recognition methods again based on space-time characteristic |
| CN107255468A (en) * | 2017-05-24 | 2017-10-17 | 纳恩博（北京）科技有限公司 | Method for tracking target, target following equipment and computer-readable storage medium |
| CN107545256A (en) * | 2017-09-29 | 2018-01-05 | 上海交通大学 | A kind of camera network pedestrian recognition methods again of combination space-time and network consistency |
Non-Patent Citations (2)
| Title |
|---|
| LEARNING DISCRIMINATIVE AND SHAREABLE PATCHES FOR SCENE CLASSIFICATION;Shoucheng Ni;《IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP2016)》;20160321;第1317-1321页 * |
| 距离度量学习的摄像网络中行人重识别;章东平;《中国计量大学学报》;20170213;第27卷(第4期);第424-428,434页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108764167A (en) | 2018-11-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108764167B (en) | Space-time correlated target re-identification method and system | |
| CN111563442B (en) | Slam method and system for fusing point cloud and camera image data based on laser radar | |
| Wang et al. | Tracklet association by online target-specific metric learning and coherent dynamics estimation | |
| CN102542289B (en) | Pedestrian volume statistical method based on plurality of Gaussian counting models | |
| CN108256431B (en) | Hand position identification method and device | |
| Azevedo et al. | Automatic vehicle trajectory extraction by aerial remote sensing | |
| CN110660082A (en) | Target tracking method based on graph convolution and trajectory convolution network learning | |
| CN109598794B (en) | Construction method of three-dimensional GIS dynamic model | |
| CN112990310A (en) | Artificial intelligence system and method for serving electric power robot | |
| CN110874583A (en) | Passenger flow statistics method and device, storage medium and electronic equipment | |
| CN106251362B (en) | A kind of sliding window method for tracking target and system based on fast correlation neighborhood characteristics point | |
| CN109583373B (en) | Pedestrian re-identification implementation method | |
| CN106815563B (en) | Human body apparent structure-based crowd quantity prediction method | |
| CN113592905B (en) | Vehicle driving track prediction method based on monocular camera | |
| CN105825520A (en) | Monocular SLAM (Simultaneous Localization and Mapping) method capable of creating large-scale map | |
| CN115376034A (en) | Motion video acquisition and editing method and device based on human body three-dimensional posture space-time correlation action recognition | |
| CN111402632B (en) | Risk prediction method for pedestrian movement track at intersection | |
| CN110119768A (en) | Visual information emerging system and method for vehicle location | |
| CN109934096B (en) | Automatic driving visual perception optimization method based on characteristic time sequence correlation | |
| CN104182747A (en) | Object detection and tracking method and device based on multiple stereo cameras | |
| CN113076808A (en) | Method for accurately acquiring bidirectional pedestrian flow through image algorithm | |
| CN104915967B (en) | The Forecasting Methodology in vehicle movement path in a kind of tunnel | |
| JP6894395B2 (en) | Information acquisition device, information aggregation system, and information aggregation device | |
| Mancusi et al. | TrackFlow: Multi-Object Tracking with Normalizing Flows | |
| Guo et al. | Research and Implementation of Robot Vision Scanning Tracking Algorithm Based on Deep Learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |