CN114663796A - Target person continuous tracking method, device and system - Google Patents
Target person continuous tracking method, device and system Download PDFInfo
- Publication number
- CN114663796A CN114663796A CN202210000443.5A CN202210000443A CN114663796A CN 114663796 A CN114663796 A CN 114663796A CN 202210000443 A CN202210000443 A CN 202210000443A CN 114663796 A CN114663796 A CN 114663796A
- Authority
- CN
- China
- Prior art keywords
- target person
- person
- target
- information
- next frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000001514 detection method Methods 0.000 claims description 16
- 238000010586 diagram Methods 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 10
- 230000000694 effects Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000011161 development Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 210000003423 ankle Anatomy 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000009977 dual effect Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 210000003127 knee Anatomy 0.000 description 2
- 210000000707 wrist Anatomy 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011840 criminal investigation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method, a device and a system for continuously tracking a target person, which comprise the following steps: reading video data in a pan-tilt camera, identifying all figure images in a video area, obtaining coordinate information of each figure, and cutting out an image of each figure according to the coordinate information of each figure; carrying out face recognition on the image of each figure to obtain face information of each figure, and comparing the face information of each figure with the face information of the target figure to recognize the target figure; acquiring next frame coordinate information of the target person based on the image and the coordinate information of the target person; and controlling the pan-tilt camera to move based on the next frame coordinate information of the target person, so that the target person is in the shooting area of the pan-tilt camera, and finishing continuous tracking of the target person. By the scheme provided by the invention, continuous tracking can be performed no matter whether the pedestrian is shielded or not, and the identification accuracy is improved.
Description
Technical Field
One or more embodiments of the invention relate to the technical field of intelligent monitoring, and in particular to a method, a device and a system for continuously tracking a target person.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein may include concepts that could be pursued, but are not necessarily ones that have been previously conceived or pursued. Thus, unless otherwise indicated herein, what is described in this section is not prior art to the description and claims in this application and is not admitted to be prior art by inclusion in this section.
The pedestrian tracking algorithm is an important research field and an application direction, with the development of a deep learning technology, at present, researchers use the deep learning technology to track a target person more, and the technology can be applied to the fields of security, criminal investigation and the like.
In the current research, methods for tracking a target pedestrian mainly include methods of target tracking and pedestrian re-identification. Target tracking can be further divided into a single target tracking method and a multi-target tracking method. The single-target tracking algorithm mainly continuously tracks one target in a video area, the representative algorithms comprise siamFC, siamRPN and the like, and the application of a twin network brings great improvement on the tracking algorithm effect; the multi-target tracking algorithm is to track each detected target in a video area, so each target has a single label serial number, also called ID, and if the ID of one target is kept unchanged all the time, the tracking effect of the target is good, and typical multi-target tracking algorithms include deppsort, JDE, FairMOT algorithms and the like. The pedestrian re-recognition algorithm is different from the target tracking algorithm, the pedestrian re-recognition algorithm focuses more on different camera areas, the target person is recognized according to the characteristics of the target person such as the clothing posture and the like, the pedestrian re-recognition algorithm is a pedestrian matching algorithm spanning multiple cameras, the pedestrian re-recognition algorithm mainly comprises two steps of pedestrian detection and re-recognition, and the target person is searched in different cameras, so that the purpose of tracking the target person is achieved.
The deep learning field has many subdivided research directions, such as image recognition, target detection, target tracking, instance segmentation, and the like. Based on the current research, the invention provides a set of pedestrian tracking technology, which mainly relates to the aspects of target detection, face recognition, target tracking and the like, and then realizes a set of complete pedestrian tracking system by combining the image processing technology and the camera holder linkage technology.
Disclosure of Invention
One or more embodiments of the present specification describe a method, an apparatus, and a system for continuously tracking a target person, which can not only complete continuous tracking of the target person when face recognition can be performed; and the tracking algorithm is optimized, the part information characteristics of the human body are increased, and the tracking can be performed when the pedestrian is shielded and the face recognition cannot be performed, so that the recognition accuracy is improved.
The technical scheme provided by one or more embodiments of the specification is as follows:
in a first aspect, the present invention provides a method for continuously tracking a target person, including:
reading video data in a pan-tilt camera, identifying all figure images in a video area, obtaining coordinate information of each figure, and cutting out an image of each figure according to the coordinate information of each figure;
performing face recognition on the image of each person to obtain face information of each person, and comparing the face information of each person with the face information of a target person to recognize the target person;
acquiring next frame coordinate information of the target person based on the image and the coordinate information of the target person;
and controlling the pan-tilt camera to move based on the next frame coordinate information of the target person, so that the target person is in the shooting area of the pan-tilt camera, and finishing continuous tracking of the target person.
Preferably, the comparing the face information of each person with the face information of the target person to identify the target person specifically includes:
comparing the face information of each person with the face information of the target person to obtain the similarity between each person and the face of the target person;
and determining the person with the similarity reaching the threshold as the target person.
Preferably, the obtaining the next frame of coordinate information of the target person based on the image and the coordinate information of the target person specifically includes:
predicting next frame coordinate information of the target person based on the image and the coordinate information of the target person;
and identifying the face information of the person appearing at the predicted next frame coordinate, comparing the face information with the face information of the target person, and outputting the next frame coordinate information if the person appearing at the next frame coordinate is determined to be the target person.
Preferably, the obtaining the next frame coordinate information of the target person based on the image and the coordinate information of the target person includes:
predicting next frame coordinate information of the target person based on the image and the coordinate information of the target person;
if the face information of the person appearing at the next frame coordinate is not recognized, the part information characteristics of the person are recognized and compared with the part information characteristics of the target person, and if the person appearing at the next frame coordinate is determined to be the target person, the predicted next frame coordinate information is output.
Preferably, when all the person images in the video area are identified, a label is set for each person;
when a target person is identified, acquiring a label of the target person;
acquiring next frame coordinate information of a label of the target person;
and controlling the holder camera to move based on the next frame of coordinate information of the label of the target person, so that the target person is in the shooting area of the holder camera, and the target person is continuously tracked.
In a second aspect, the present invention provides a target person continuous tracking apparatus, including:
the target detection module is used for reading video data in the camera, identifying all the figure images in the video area, obtaining the coordinate information of each figure, and cutting out the image of each figure according to the coordinate information of each figure;
the face recognition module is used for carrying out face recognition on the image of each figure to obtain face information of each figure, comparing the face information of each figure with the face information of a target figure and recognizing the target figure;
the target tracking module is used for acquiring the next frame of coordinate information of the target person according to the image and the coordinate information of the target person;
and the holder control module is used for controlling the holder camera to move based on the next frame of coordinate information of the target person, so that the target person can be continuously tracked in the shooting area of the holder camera.
Preferably, the first and second electrodes are formed of a metal,
the target tracking module is also used for predicting the next frame of coordinate information of the target person based on the image and the coordinate information of the target person;
the face recognition module is also used for recognizing the face information of the person appearing in the next frame coordinate, comparing the face information with the face information of the target person and determining whether the person appearing in the next frame coordinate is the target person or not;
the target tracking module is further configured to output the next frame coordinate information when it is determined that the person appearing at the next frame coordinate is the target person.
Preferably, the apparatus further comprises a component information feature module;
the target tracking module is also used for predicting the next frame of coordinate information of the target person based on the image and the coordinate information of the target person;
the part information characteristic module is used for identifying the part information characteristics of the person when the face identification module cannot identify the face information of the person appearing in the next frame coordinate, comparing the part information characteristics of the person with the part information characteristics of the target person and determining whether the person appearing in the next frame coordinate is the target person or not;
and the target tracking module is used for outputting the next frame coordinate information when the next frame coordinate is determined to be the target person.
In a third aspect, the present invention provides a target person continuous tracking system, the system comprising at least one processor and a memory;
the memory to store one or more program instructions;
the processor is configured to execute one or more program instructions to perform the method according to one or more of the first aspects.
In a fourth aspect, the invention provides a computer readable storage medium comprising one or more program instructions executable by a system according to the third aspect to implement a method according to one or more of the first aspects.
In the technical solution of the target person continuous tracking provided in one or more embodiments of the present specification, in the tracking stage after the target person is determined, the face recognition algorithm may always recognize the person in the video area, and the target person may be continuously tracked through the dual recognition of the face recognition algorithm and the target tracking algorithm. In order to improve the accuracy of tracking identification, the target tracking algorithm is optimized, the part information characteristics of a human body are added, and when the face information of a target person cannot be acquired, the target can be well tracked by the scheme provided by the invention, so that the identification accuracy of personnel under the shielding condition is improved.
Drawings
FIG. 1 is a diagram of the hardware device of the present invention;
FIG. 2 is a schematic overall flow diagram of the system of the present invention;
FIG. 3 is a schematic flow chart of the algorithm of the present invention;
FIG. 4 is a schematic flow chart of a target person continuous tracking method provided by the present invention;
FIG. 5 is a first flowchart illustrating a process of obtaining next frame coordinate information of a target person;
FIG. 6 is a second flowchart illustrating a process of obtaining coordinate information of a next frame of the target person;
FIG. 7 is a diagram illustrating the recognition result of the human body gesture by the pedestrian key point algorithm;
FIG. 8 is a first schematic structural diagram of a device for continuously tracking a target person according to the present invention;
FIG. 9 is a schematic structural diagram of a second device for continuously tracking a target person according to the present invention;
fig. 10 is a schematic structural diagram of a target person continuous tracking system provided by the present invention.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be further noted that, for the convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to realize the continuous tracking of the target person, the invention constructs a set of complete pedestrian tracking system by combining the technologies of using a target detection algorithm, a face recognition algorithm, a target tracking algorithm and the like and then assisting with an image processing technology and a camera holder linkage technology.
The system mainly comprises three modules: the hardware equipment module, the intelligent algorithm module and the front-end display and control module are explained as follows:
referring to fig. 1, the hardware devices related to the system and the connection mode of the devices are mainly composed of an inference server, a pan-tilt camera, a front-end desktop computer, a network switch, a hard disk video recorder and the like, and a hardware environment required by the system operation is formed through linkage of the hardware devices, as shown in fig. 1. The inference server is a basis for running an artificial intelligence algorithm and is used for running a pedestrian tracking algorithm, the input of the tracking algorithm is real-time video stream of a camera, the output of the tracking algorithm is position information of a target person, and in some outdoor scenes, the inference server can be replaced by some ARM architecture development kits for more convenient carrying and installation; the main function of the pan-tilt camera is to collect video information, and then track a target person through the movement of the pan-tilt, wherein the pan-tilt camera can be installed on an unmanned aerial vehicle or an intelligent vehicle, and continuous tracking of a target pedestrian can be realized through linkage; the function of the front-end desktop computer is to see the currently detected real-time picture, and the real-time picture is a video stream pushed by a tracking algorithm on the reasoning server in real time; the network switch is used for constructing a set of local area network, and the hard disk video recorder is used for storing video data of the pan-tilt camera. Thus, these hardware devices form a part of a system, fig. 2 is a schematic overall flow diagram of the system of the present invention, and as shown in fig. 2, the main workflow of the tracking system of the present invention is as follows: in the same local area network, firstly, a tripod head camera can continuously record video data, an algorithm on an inference server can acquire the video data of the camera, then a pedestrian tracking algorithm is started to recognize the video data in real time, after a target person is recognized, position information of the target person is obtained, then the target person is tracked by operating the tripod head camera, and the target person is always kept in a shooting area through the up-down and left-right movement of the tripod head, so that the tracking of the target person is realized.
After the hardware device is introduced, a software algorithm is described below, and the software algorithm mainly includes two aspects, one is a pedestrian tracking algorithm, and the other is a display interface at the front end. The main function of the display interface at the front end is to display the video information identified by the algorithm, the position information of the target person and the like, so as to help users to better observe the target person. The pedestrian tracking algorithm mainly comprises the target detection algorithm, the face recognition algorithm, the target tracking algorithm and the like. FIG. 3 is a schematic flow chart of an algorithm involved in the present invention; fig. 4 is a schematic flow chart of the target person continuous tracking method, and with reference to fig. 3 and 4, the following detailed description is made on the overall flow of target person continuous tracking, including the following steps:
s10, reading the video data in the pan-tilt camera, identifying all the figure images in the video area, obtaining the coordinate information of each figure, and cutting out the image of each figure according to the coordinate information of each figure.
The step mainly utilizes an object detection algorithm, and aims to obtain the person image in the video area.
And S20, performing face recognition on the image of each person to obtain face information of each person, and comparing the face information of each person with the face information of the target person to recognize the target person.
The face recognition algorithm mainly used in this step is intended to identify a target person to be tracked, and in one example, the target person is identified by the following method:
comparing the face information of each person with the face information of the target person to obtain the similarity between each person and the face of the target person; and setting a threshold value, and determining the person with the similarity reaching the threshold value as a target person.
S30, based on the image of the target person and the coordinate information, the next frame coordinate information of the target person is obtained.
The step mainly utilizes a target tracking algorithm, aims to obtain the coordinate information of a target person, and is beneficial to the follow-up tracking of the pan-tilt camera. In this step, the single-target tracking algorithm is adopted, and in the process of tracking the target person by the single-target tracking algorithm, in order to avoid the situations of loss of tracking and wrong tracking, the face recognition algorithm can always recognize the person in the current area, so that the target person can be continuously tracked by double recognition of the face recognition algorithm and the single-target tracking algorithm, and a better tracking effect is achieved, specifically, as shown in fig. 5, the step of obtaining the next frame of coordinate information of the target person is S3011-S3012:
s3011, predicting the next frame coordinate information of the target person based on the image and the coordinate information of the target person;
s3012, recognizing the face information of the person appearing at the predicted next frame coordinate, comparing the face information with the face information of the target person, and if it is determined that the person appearing at the next frame coordinate is the target person, outputting the next frame coordinate information.
The algorithms used in the steps S10 to S30 are described in a supplementary manner, and in the tracking system constructed by the present invention, the algorithms mainly used include a target detection algorithm, a single-target tracking algorithm, a face recognition algorithm, and the like. The target detection algorithm can use a single-stage algorithm such as Yolo and SSD, and compared with a two-stage detection algorithm of an RCNN series, the single-stage target detection algorithm can directly output the class probability and the position coordinate of the target, so that the reasoning speed is higher, and the real-time reasoning effect can be realized on some development kits. For a single-target tracking algorithm, a SimFC algorithm can be selected and used, the SimFC is a tracking algorithm based on a twin network, the algorithm is good in recognition effect and high in reasoning speed, the twin network is very colorful in the field of tracking algorithms, and many tracking algorithms with good effects are derived, such as SimRPN, SimMask and the like.
In the tracking process, sometimes pedestrians are shielded, and when the face information of a target person cannot be identified by using a face identification algorithm, in order to improve the accuracy of tracking identification, the single-target tracking algorithm is optimized, and the part information characteristics of a human body are added, so that when the face information of the target person cannot be obtained, the single-target tracking algorithm provided by the invention can achieve higher accuracy. Specifically, as shown in fig. 6, steps S3021 to S3022 of obtaining the next frame coordinate information of the target person:
s3021, predicting the next frame coordinate information of the target person based on the image and the coordinate information of the target person;
s3022, if the face information of the person appearing at the predicted next frame coordinate cannot be recognized, recognizing the part information feature of the person, comparing the recognized part information feature with the part information feature of the target person, and if it is determined that the person appearing at the next frame coordinate is the target person, outputting the predicted next frame coordinate information.
In other words, the tracking scheme provided by the invention optimizes the single-target tracking algorithm, and the idea is to add human component characteristics into the tracking algorithm for auxiliary identification, and the specific idea is as follows: the human body posture is recognized by using a pedestrian key point recognition algorithm, the key point position of a pedestrian is found, then the pedestrian part is divided according to the key point position, and the part characteristics of the head, the upper body, the lower body, the left arm, the right arm and the like can be obtained. The pedestrian parts can be divided differently as required. For example, fig. 7 is a schematic diagram of a recognition result of a human body posture by a pedestrian key point algorithm, as shown in fig. 7, each black dot in the diagram represents a key point position, 18 pieces of key point information of a pedestrian body component are obtained by recognizing a human body posture by the pedestrian key point algorithm, and the human body is divided into 18 component features including a nose part 0, a neck part 1, a right shoulder part 2, a right elbow part 3, a right wrist part 4, a left shoulder part 5, a left elbow part 6, a left wrist part 7, a right crotch part 8, a right knee part 9, a right ankle part 10, a left crotch part 11, a left knee part 12, a left ankle part 13, a right eye part 14, a left eye part 15, a right ear part 16, a left ear part 17, and the like. It should be noted that the features may be color features of the component or features extracted by a neural network, and then pedestrians in previous and subsequent frames in the video are compared at the level of the feature of the component, for example: if one person in the front frame and the back frame wears the blue coat and the black coat, the probability that the two frames are the same person is high, the person can be used as auxiliary judgment, the person with the maximum similarity is found in the front frame and the back frame, and the identification accuracy can be improved under the condition that the person is shielded.
In addition, instead of tracking the target person by using a single-target tracking algorithm, a multi-target tracking algorithm can be used instead, the multi-target tracking algorithm can track each person in the shooting area, each person has an individual tag and also becomes an ID, and only the position information of the ID needs to be tracked all the time after the ID of the target person is locked.
In one example, the specific implementation manner of the multi-target tracking algorithm is as follows:
setting a label for each person when all the person images in the video area are identified;
when a target person is identified, acquiring a label of the target person;
and obtaining the next frame coordinate information of the label of the target person.
In addition to the tracking algorithm, the face recognition algorithm in the system can help people to track the target better, the existing face recognition algorithm is relatively mature, and the recognition accuracy of a plurality of face recognition algorithms is high, so that all faces in the area are recognized through the face recognition algorithm, and the target person can be tracked more accurately.
And S40, controlling the movement of the pan-tilt camera based on the next frame coordinate information of the target person, so that the target person is in the shooting area of the pan-tilt camera, and completing the continuous tracking of the target person.
The step mainly utilizes a holder control algorithm, and achieves continuous tracking of the target by controlling the movement of a holder camera according to the coordinate information of the target person.
The above description of the software algorithm, that is, the description of the intelligent algorithm module, mainly includes the artificial intelligence algorithm and the pan-tilt control technique used in the tracking system. The method mainly comprises a target detection algorithm, a face recognition algorithm, a target tracking algorithm, an image processing technology, a holder control algorithm and the like. The method comprises the steps of firstly finding all people in a current area through a target detection algorithm, then sending a person image to a face recognition algorithm, finding out a target person from the face recognition algorithm, then sending information of the target person to a target tracking algorithm, continuously tracking the target person through the target tracking algorithm, simultaneously assisting the pedestrian part characteristics to be optimized, in the process, fully operating the face recognition algorithm to improve the recognition accuracy of the target person, then sending the position information of the recognized target person to a holder, controlling the movement of the camera holder through the holder control algorithm, enabling the target person to be always in a shooting area, and completing the continuous tracking of the pedestrian.
The display and control model of the front end is mainly used for facilitating observation and use of a user, the video image display of the cloud deck camera can be displayed on the front end interface, the user can observe the information of the target person better, and meanwhile, the control over hardware equipment and an algorithm can be achieved through the front end, for example, the moving direction of the cloud deck camera is controlled, the target person needing to be tracked is set, and the like.
Through the combined operation of the three modules, a set of complete pedestrian tracking system is realized. Simultaneously this system can operate on the development external member, and the installation is carried more conveniently, and this tracker also can link with unmanned aerial vehicle, unmanned car etc. has realized the purpose of removal tracking.
By the scheme provided by the invention, continuous tracking can be performed no matter whether the pedestrian is shielded or not, and the identification accuracy is improved.
Fig. 8 is a schematic structural diagram of a device for continuously tracking a target person according to an embodiment of the present invention, and as shown in fig. 8, the device includes: target detection module 18, face recognition module 19, target tracking module 20 and cloud platform control module 21, specifically:
the target detection module 18 is configured to read video data in the camera, identify images of all persons in the video area, obtain coordinate information of each person, and cut out an image of each person according to the coordinate information of each person.
The face recognition module 19 is configured to perform face recognition on the image of each person to obtain face information of each person, and compare the face information of each person with face information of a target person to identify the target person;
and the target tracking module 20 is used for obtaining the next frame coordinate information of the target person according to the image and the coordinate information of the target person.
The pan-tilt control module 21 is configured to control the pan-tilt camera to move based on the next frame coordinate information of the target person, so that the target person completes continuous tracking of the target person in a shooting area of the pan-tilt camera.
When the single-target tracking is performed,
the target tracking module 18 is further configured to predict the next frame coordinate information of the target person based on the image and the coordinate information of the target person.
The face recognition module 19 is further configured to recognize face information of a person appearing in the next frame coordinate, compare the face information with face information of a target person, and determine whether the person appearing in the next frame coordinate is the target person;
the target tracking module 20 is further configured to output the next frame coordinate information when it is determined that the person appearing at the next frame coordinate is the target person.
In order to improve the tracking accuracy if the face recognition module 19 cannot obtain face information during the single-target tracking process, as shown in fig. 9, the target person continuous tracking apparatus further includes a component information feature module 22:
the target tracking module 18 is further configured to predict the next frame of coordinate information of the target person based on the image and the coordinate information of the target person;
the part information feature module 22 is configured to, when the face recognition module cannot recognize the face information of the person appearing at the next frame coordinate, recognize the part information features of the person, compare the part information features with the part information features of the target person, and determine whether the person appearing at the next frame coordinate is the target person;
the target tracking module 20 is configured to output the next frame coordinate information when it is determined that the target person appears at the next frame coordinate.
The functions executed by each component in the apparatus provided in the embodiment of the present invention have been described in detail in the above-mentioned method, and therefore, redundant description is not repeated here.
Corresponding to the above embodiments, the embodiment of the present invention further provides a multi-target tracking system, specifically as shown in fig. 10, the system includes at least one processor 101 and a memory 102;
a memory 101 for storing one or more program instructions;
a processor 102 for executing one or more program instructions to perform any of the method steps of a method for continuously tracking a target person as described in the above embodiments.
In correspondence with the above embodiments, the present invention further provides a computer storage medium including one or more programs, where the one or more program instructions are used for executing the target person continuous tracking method introduced above by the target person continuous tracking system.
According to the scheme provided by the application, in the tracking stage after the target person is determined, the face recognition algorithm can always recognize the person in the video area, and the target person can be continuously tracked through dual recognition of the face recognition algorithm and the target tracking algorithm. In order to improve the accuracy of tracking and recognition, the target tracking algorithm is optimized, the part information characteristics of the human body are added, and when the face information of the target person cannot be obtained, the target can be well tracked by the scheme provided by the invention, so that the recognition accuracy of personnel under the shielding condition is improved.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied in hardware, a software module executed by a processor, or a combination of the two. A software module may reside in Random Access Memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The above embodiments are provided to further explain the objects, technical solutions and advantages of the present invention in detail, it should be understood that the above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the scope of the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. A target person continuous tracking method is characterized by comprising the following steps:
reading video data in a pan-tilt camera, identifying all figure images in a video area, obtaining coordinate information of each figure, and cutting out an image of each figure according to the coordinate information of each figure;
performing face recognition on the image of each person to obtain face information of each person, and comparing the face information of each person with the face information of a target person to recognize the target person;
acquiring next frame coordinate information of the target person based on the image and the coordinate information of the target person;
and controlling the pan-tilt camera to move based on the next frame coordinate information of the target person, so that the target person is in the shooting area of the pan-tilt camera, and finishing continuous tracking of the target person.
2. The method of claim 1, wherein the comparing the face information of each person with the face information of the target person identifies the target person by:
comparing the face information of each person with the face information of the target person to obtain the similarity between each person and the face of the target person;
and determining the person with the similarity reaching the threshold as the target person.
3. The method according to claim 1, wherein the obtaining of the next frame coordinate information of the target person based on the image and the coordinate information of the target person is specifically:
predicting next frame coordinate information of the target person based on the image and the coordinate information of the target person;
and identifying the face information of the person appearing at the predicted next frame coordinate, comparing the face information with the face information of the target person, and outputting the next frame coordinate information if the person appearing at the next frame coordinate is determined to be the target person.
4. The method according to claim 1, wherein the obtaining of the next frame coordinate information of the target person based on the image and the coordinate information of the target person is specifically:
predicting next frame coordinate information of the target person based on the image and the coordinate information of the target person;
if the face information of the person appearing at the next frame coordinate is not recognized, the part information characteristics of the person are recognized and compared with the part information characteristics of the target person, and if the person appearing at the next frame coordinate is determined to be the target person, the predicted next frame coordinate information is output.
5. The method according to claim 1, wherein a label is set for each person when all the person images in the video area are recognized;
when a target person is identified, acquiring a label of the target person;
acquiring next frame coordinate information of a label of the target person;
and controlling the holder camera to move based on the next frame of coordinate information of the label of the target person, so that the target person is in the shooting area of the holder camera, and the target person is continuously tracked.
6. A target person continuous tracking device, comprising:
the target detection module is used for reading video data in the camera, identifying images of all persons in the video area, obtaining coordinate information of each person, and cutting out the image of each person according to the coordinate information of each person;
the face recognition module is used for carrying out face recognition on the image of each figure to obtain face information of each figure, comparing the face information of each figure with the face information of a target figure and recognizing the target figure;
the target tracking module is used for acquiring the next frame of coordinate information of the target person according to the image and the coordinate information of the target person;
and the holder control module is used for controlling the holder camera to move based on the next frame of coordinate information of the target person, so that the target person can be continuously tracked in the shooting area of the holder camera.
7. The apparatus of claim 6,
the target tracking module is also used for predicting the next frame of coordinate information of the target person based on the image and the coordinate information of the target person;
the face recognition module is also used for recognizing the face information of the person appearing in the next frame coordinate, comparing the face information with the face information of the target person and determining whether the person appearing in the next frame coordinate is the target person or not;
the target tracking module is further configured to output the next frame coordinate information when it is determined that the person appearing at the next frame coordinate is the target person.
8. The apparatus of claim 6, further comprising a component information feature module;
the target tracking module is also used for predicting the next frame of coordinate information of the target person based on the image and the coordinate information of the target person;
the part information characteristic module is used for identifying the part information characteristics of the person when the face identification module cannot identify the face information of the person appearing in the next frame coordinate, comparing the part information characteristics of the person with the part information characteristics of the target person and determining whether the person appearing in the next frame coordinate is the target person or not;
and the target tracking module is used for outputting the next frame coordinate information when the next frame coordinate is determined to be the target person.
9. A target person continuous tracking system, the system comprising at least one processor and a memory;
the memory to store one or more program instructions;
the processor, configured to execute one or more program instructions to perform the method according to one or more of claims 1 to 5.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises one or more program instructions that are executable by the system of claim 9 to implement the method of one or more of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210000443.5A CN114663796A (en) | 2022-01-04 | 2022-01-04 | Target person continuous tracking method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210000443.5A CN114663796A (en) | 2022-01-04 | 2022-01-04 | Target person continuous tracking method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114663796A true CN114663796A (en) | 2022-06-24 |
Family
ID=82026413
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210000443.5A Pending CN114663796A (en) | 2022-01-04 | 2022-01-04 | Target person continuous tracking method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114663796A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115183763A (en) * | 2022-09-13 | 2022-10-14 | 南京北新智能科技有限公司 | Personnel map positioning method based on face recognition and grid method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110516578A (en) * | 2019-08-20 | 2019-11-29 | 开放智能机器（上海）有限公司 | A kind of monitoring system based on recognition of face and target following |
| WO2020155873A1 (en) * | 2019-02-02 | 2020-08-06 | 福州大学 | Deep apparent features and adaptive aggregation network-based multi-face tracking method |
| CN112215155A (en) * | 2020-10-13 | 2021-01-12 | 北京中电兴发科技有限公司 | Face tracking method and system based on multi-feature fusion |
| CN112257502A (en) * | 2020-09-16 | 2021-01-22 | 深圳微步信息股份有限公司 | Pedestrian identification and tracking method and device for surveillance video and storage medium |
| CN113838092A (en) * | 2021-09-24 | 2021-12-24 | 精英数智科技股份有限公司 | Pedestrian tracking method and system |
-
2022
- 2022-01-04 CN CN202210000443.5A patent/CN114663796A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020155873A1 (en) * | 2019-02-02 | 2020-08-06 | 福州大学 | Deep apparent features and adaptive aggregation network-based multi-face tracking method |
| CN110516578A (en) * | 2019-08-20 | 2019-11-29 | 开放智能机器（上海）有限公司 | A kind of monitoring system based on recognition of face and target following |
| CN112257502A (en) * | 2020-09-16 | 2021-01-22 | 深圳微步信息股份有限公司 | Pedestrian identification and tracking method and device for surveillance video and storage medium |
| CN112215155A (en) * | 2020-10-13 | 2021-01-12 | 北京中电兴发科技有限公司 | Face tracking method and system based on multi-feature fusion |
| CN113838092A (en) * | 2021-09-24 | 2021-12-24 | 精英数智科技股份有限公司 | Pedestrian tracking method and system |
Non-Patent Citations (1)
| Title |
|---|
| 张能波: "基于检测跟踪模型的视觉目标跟踪算法研究与应用", 中国期刊网优秀硕士论文 信息科技专辑, no. 2020, 15 September 2020 (2020-09-15) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115183763A (en) * | 2022-09-13 | 2022-10-14 | 南京北新智能科技有限公司 | Personnel map positioning method based on face recognition and grid method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110543867B (en) | Crowd density estimation system and method under condition of multiple cameras | |
| US10242266B2 (en) | Method and system for detecting actions in videos | |
| Fernandez-Sanjurjo et al. | Real-time visual detection and tracking system for traffic monitoring | |
| US20180342067A1 (en) | Moving object tracking system and moving object tracking method | |
| CN109919977B (en) | Video motion person tracking and identity recognition method based on time characteristics | |
| CN110751022A (en) | Urban pet activity track monitoring method based on image recognition and related equipment | |
| CN109657533A (en) | Pedestrian recognition methods and Related product again | |
| CN109325408A (en) | A kind of gesture judging method and storage medium | |
| CN111931654A (en) | Intelligent monitoring method, system and device for personnel tracking | |
| CN113608663B (en) | Fingertip tracking method based on deep learning and K-curvature method | |
| Shirsat et al. | Proposed system for criminal detection and recognition on CCTV data using cloud and machine learning | |
| Beh et al. | Micro-expression spotting using facial landmarks | |
| Belkada et al. | Do pedestrians pay attention? eye contact detection in the wild | |
| CN112185515A (en) | Patient auxiliary system based on action recognition | |
| Badave et al. | Head pose estimation based robust multicamera face recognition | |
| Ponce-López et al. | Non-verbal communication analysis in victim–offender mediations | |
| CN114663796A (en) | Target person continuous tracking method, device and system | |
| CN116824641B (en) | Gesture classification method, device, equipment and computer storage medium | |
| CN111460858A (en) | Method and device for determining pointed point in image, storage medium and electronic equipment | |
| Foytik et al. | Tracking and recognizing multiple faces using Kalman filter and ModularPCA | |
| Tur et al. | Isolated sign recognition with a siamese neural network of RGB and depth streams | |
| Sun et al. | Automated work efficiency analysis for smart manufacturing using human pose tracking and temporal action localization | |
| CN115909497A (en) | Human body posture recognition method and device | |
| CN114550291A (en) | Gait feature extraction method, device and equipment | |
| KR102356165B1 (en) | Method and device for indexing faces included in video |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |