CN113920164B - Actor identity re-identification method based on near infrared anti-counterfeiting ink in theatre environment - Google Patents
Actor identity re-identification method based on near infrared anti-counterfeiting ink in theatre environment Download PDFInfo
- Publication number
- CN113920164B CN113920164B CN202111252678.5A CN202111252678A CN113920164B CN 113920164 B CN113920164 B CN 113920164B CN 202111252678 A CN202111252678 A CN 202111252678A CN 113920164 B CN113920164 B CN 113920164B
- Authority
- CN
- China
- Prior art keywords
- actor
- ink
- near infrared
- target
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 230000008569 process Effects 0.000 claims abstract description 8
- 238000001514 detection method Methods 0.000 claims description 80
- 238000012549 training Methods 0.000 claims description 13
- 239000011248 coating agent Substances 0.000 claims description 5
- 238000000576 coating method Methods 0.000 claims description 5
- 238000013135 deep learning Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 238000005286 illumination Methods 0.000 abstract description 10
- 230000008859 change Effects 0.000 description 5
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 3
- 238000011176 pooling Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- RZVAJINKPMORJF-UHFFFAOYSA-N Acetaminophen Chemical compound CC(=O)NC1=CC=C(O)C=C1 RZVAJINKPMORJF-UHFFFAOYSA-N 0.000 description 1
- 239000011358 absorbing material Substances 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Credit Cards Or The Like (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an actor identity re-identification method based on near infrared anti-counterfeiting ink in a theatre environment. According to the method, near infrared anti-counterfeiting ink marks invisible in a color camera are obtained through the near infrared camera, then images are sent to a trained detector and classifier, and finally identification re-identification of apparent similar actors on a stage is achieved when tracking is carried out. The method uses the near infrared anti-counterfeiting ink mark which is only visible by the near infrared camera to replace the existing actor apparent information or sensor as key features for distinguishing the identity of the actor, and has the following advantages: the interference of complex stage illumination on the stability of identification in the actor tracking process is greatly reduced, and the problem of tracking difficulty caused by apparent similarity of actors is solved by introducing specific near infrared anti-counterfeiting ink mark information. Compared with the identification method based on the sensor, the method has low cost, strong operability and general applicability in stage performance.
Description
Technical Field
The invention relates to the technical field of multi-target tracking, in particular to a near infrared anti-counterfeiting ink-based actor identity re-identification method in a theatre environment.
Background
The target tracking task is widely applied in various fields, such as the fields of automatic driving, security protection or medical treatment. Identity re-identification (ReID) is a key issue that must be addressed in tracking tasks. ReID aims to solve the problem of ID change or restarting when the target is re-matched due to factors such as target interaction, shielding, complex illumination and the like
The existing ReID method faces the following several main challenges in practical application: the difference of target appearance caused by different camera visual angles is huge; partial effective characteristics are lost due to mutual shielding and self shielding of targets; different illumination causes a drastic change in the characteristics of the target; the apparent similarity of different targets results in insufficient differentiation of features.
The existing ReID algorithm based on deep learning can be divided into three methods, namely a global feature-based method, an auxiliary feature-based method and a local feature-based method according to the mode of extracting image features. Global feature-based methods extract global feature information for each image, such as by global max pooling or global average pooling. The method based on the global features cannot solve the interference problem caused by the background area, and has a limit on practical application. Therefore, researchers have begun to study methods based on assist features. At present, the method mainly adopts a generated countermeasure network (GAN) to amplify a target data set so as to achieve the aim of improving the generalization of the classifier. However, the pictures generated by the GAN network often lack reality, and have more redundant noise information, and have larger deviation from the actual application scene. In recent years, more and more research effort has focused on local features. If the image is subjected to horizontal dicing treatment, the segmented image is sent into a long-short-term memory artificial neural network (LSTM) for training according to the sequence from the head to the body; there is also a grid partitioning method, in which similar and different features of an image pair are extracted from the same grid region, and feature information of all region grids is fused to perform two classifications on an input image, and whether the input image is the same ID is determined. By adopting the method based on the local characteristics, the recognition accuracy of ReID is obviously improved.
In recent years, the search and research of ReID tasks have been developed, and possible future directions include robustness, fast learning, online learning and the like, and basically are still based on floor guidance. ReID in a multi-target tracking task in a stage scene also faces the following challenges: the target loss caused by rapid action change and mutual shielding among target actors; the apparent similarity and the clothes of the target actors are the same, compared with the common scene, the apparent difference is smaller, the apparent characteristic distinction degree is lower, and the ID identification is more difficult; stage illumination changes drastically, resulting in a decrease in target detection accuracy. In practical stage, the existing scheme can not well solve ReID problems under the application of actor tracking.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a method for identifying the identity of multiple target actors in a theatre environment based on near infrared anti-counterfeiting ink marks for assisting multi-target actor tracking (ReID), wherein the near infrared ink marks which are only visible by an infrared camera are used for replacing the existing actor apparent information or sensors to be used as key features for distinguishing the identity of the actors, so that the interference of stage complex illumination on the stability of the identity identification in the actor tracking process is greatly reduced, and the problem of difficult identification caused by apparent similarity of the actors is solved by introducing special infrared ink mark information.
The anti-counterfeiting ink is ink with an anti-counterfeiting function, is prepared by adding one or more near infrared absorbing materials into the ink, reflects visible light and is in an absorption state for light in a near infrared band. Due to the characteristic of near infrared anti-counterfeiting ink that the ink absorbs infrared rays, if the ink is used in a certain part of a printed matter, no trace exists in sunlight, but corresponding signals or images and texts can be observed in a detection instrument.
Based on the inspired of the method, the infrared anti-counterfeiting ink is applied to multi-target tracking, the near infrared ink is coated on the actor clothes in a pattern of a specific style, and the coated pattern is invisible in the human eye and the visible light camera because the near infrared light belongs to the invisible light wave band of the human eye, but the obvious pattern can be shown in the near infrared camera, so that the theatre environment is supplemented with the near infrared light on a stage, and different ink coating marks on each actor can be shot by the near infrared camera. The different ink symbols are used as apparent features for distinguishing different actors, so that ReID for tracking the multi-target actors is realized.
The invention provides a near infrared anti-counterfeiting ink-based actor identity re-identification method in a theatre environment, which comprises the following specific implementation procedures:
(1) Collecting stage performance data to construct a training set, comprising:
(1.1) coating a specific pattern of marks on the target actor clothes by using near infrared anti-counterfeiting ink, wherein different patterns of marks represent different actor identities;
(1.2) using a near infrared light filling lamp to fill light in the theatre environment, and collecting an image sequence of the actor performing on a stage through a near infrared camera;
(1.3) labeling the actor body area, the ink mark area and the corresponding categories for the acquired near infrared image to form a data set of the detector;
Cutting the ink marking area of the near infrared image, and dividing the cut ink marking image into N classes according to different ink marking patterns to form a data set of a classifier;
(2) Training a detector based on a deep learning method and a classifier model, wherein the detector is used for detecting coordinate positions and corresponding categories of target actors and ink marks in a near infrared image, and the classifier is used for identifying and classifying ink marks in different patterns;
(3) Sending the current frame into a detector to detect all target actors and ink marks in the image, and sending the detected ink mark areas into a classifier after cutting the detected ink mark areas from the original image;
(4) The classifier classifies the input ink marked images according to the ink marked patterns, different class numbers represent different actor IDs, if the confidence coefficient of classification is smaller than a preset threshold value, the result is considered unreliable, and the class number of the ink marked detection frame with the largest low confidence coefficient ink marked detection frame IOU of the current frame in the previous frame is taken as the class number of the low confidence coefficient detection frame of the current frame;
(5) One-to-one matching is carried out on the detected target actors and the ink marks in each frame, the intersection ratio (IOU) of each actor detection frame and all the ink mark detection frames in the current frame is calculated, and the type number of the ink mark detection frame with the largest IOU is taken as the actor ID of the target actor detection frame;
(6) And linking the video frame actor tracks according to the actor IDs, and sequentially linking actor detection frames with the same ID to form the actor tracks with multiple target tracking.
Further, in the step (1.1), the near-infrared anti-counterfeiting ink is used for coating the marks of specific patterns on the front and back of the clothes of the target actor, so that the near-infrared camera can shoot the ink marks on the actor as much as possible when the actor moves; the N pieces of apparel for the target actors are coated with N different patterns, the patterns are selected to have numbers with enough view points, so that the ID of the target actors can be determined by identifying and classifying the ink mark patterns on the apparel.
Further, in the step (1.2), a plurality of high-power near-infrared light compensating lamps are installed above the theatre obliquely to compensate light, so that the near-infrared camera can be used for shooting ink marks coated on the actor clothes.
Further, in the step (1.2), two near infrared cameras which form an angle of 90 degrees with the center of the stage are arranged in front of the stage obliquely, and under the condition that near infrared anti-counterfeiting ink marks are coated on the front and the back of the actor's clothing, at least one of the two cameras can acquire ink mark information on the actor's clothing, and images acquired by the two cameras are subjected to multi-angle image registration, so that each frame can acquire actor ID information.
In the step (1.4), the ink mark area of the near-infrared image is expanded outwards by a plurality of pixels and then cut, so that a more complete near-infrared anti-counterfeiting ink mark is obtained.
Further, in the step (3), since the near infrared image acquired in real time only contains 1 channel, the near infrared channel is duplicated to become a three-channel image, and then the three-channel image is sent to the detector.
Further, in the step (4), whether the classification result is accurate is determined by the confidence coefficient, if the confidence coefficient corresponding to the classification result of the ink mark detection frame is smaller than the preset threshold value, the result is considered to be unreliable, false detection may occur, and the target classification should be readjusted, where the adjustment method is as follows: and searching the ink mark detection frame with the largest ink mark detection frame IOU in the previous frame image and the current frame, and assigning the type number of the ink mark detection frame to the current ink mark detection frame, so that the type number of the ink mark detection frame which is frequently switched in the tracking process is stabilized, and the tracking accuracy is improved.
Further, in the step (5), calculating the IOU for each actor detection frame and all the ink mark detection frames in the current frame, if a plurality of matched IOUs are all greater than or equal to a preset threshold, taking the largest ink mark detection frame of the IOU and the actor detection frame as the best match, wherein the actor ID is the class number corresponding to the best matched ink mark; and for the actor detection frames of which the matched ink marks are not found, namely, the IOU of the actor detection frame and all the ink mark detection frames in the current frame is smaller than a preset threshold, the actor is not considered to be a target actor to be tracked, and an ID is not assigned to the actor detection frame.
Further, in the step (6), for the case where a new target appears in the tracking process: firstly, obtaining a new incoming actor ID through the steps (1) - (5), then associating each target actor detection frame of a current frame with each target actor detection frame of a previous frame according to ID matching, and directly linking the actor detection frame into the prior tracking track if the same ID can be found in the tracked actors; if there is no tracked actor with the same ID as the new actor, a tracking track head is created for the new actor.
Further, in the step (6), for the case of the actor leaving during tracking: and (3) keeping the tracking tracks of all the actors from beginning to end, and if the actors leave the field and are on the field again, adding the information of the actor detection frames on which the field is on the field again into the previous tracking tracks through the steps (1) and (5).
Inspired by the near infrared anti-counterfeiting ink, the invention provides an identity re-identification method based on near infrared anti-counterfeiting ink mark auxiliary multi-target actor tracking in theatre environment. Compared with the prior art, the invention has the following beneficial effects:
1. according to the invention, different actor IDs are distinguished by replacing apparent features of actors with near infrared ink marks of different styles, so that the problem of tracking difficulty caused by apparent similarity of actors is solved.
2. The invention replaces the visible light camera with the near infrared camera. The photosensitive part of the near infrared camera receives near infrared light and weak visible light, so that interference of complex stage illumination on the identification stability in the actor tracking process can be greatly reduced, the illumination is not required to be corrected on line according to stage illumination change, and guarantee is provided for the identification of actors in a scene with severe stage light change.
Drawings
FIG. 1 is a flow chart of a method for identifying the identity of an actor based on near infrared anti-counterfeit ink in a theatre environment provided by the invention;
FIG. 2 is a diagram of an overall construction of an actor identification re-identification method engineering implementation provided by an embodiment of the invention;
Fig. 3 is a graph showing the effect of ink marks on clothing under different illuminations according to an embodiment of the present invention.
Detailed Description
In order that the above objects, features and advantages of the invention will be readily understood, a more particular description of the invention will be rendered by reference to the appended drawings.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways other than those described herein, and persons skilled in the art will readily appreciate that the present invention is not limited to the specific embodiments disclosed below.
The embodiment of the invention provides a method for identifying identity re-identification (ReID) based on near infrared anti-counterfeiting ink mark auxiliary multi-target actor tracking in theatre environment, as shown in fig. 1 and 2, comprising the following specific steps:
(1) Design and coating of infrared ink mark
(1-1) For stage performance video data, the present invention applies a specific pattern to both the front and back of the target actor's apparel with near infrared anti-counterfeit ink, so that the camera is as much as possible shooting ink marks on the actor as the actor moves.
(1-2) Applying N different patterns to the apparel of N target actors, the pattern patterns selecting numbers with sufficient view points, as shown in fig. 3, so that the target actor ID can be determined by identifying and classifying the ink mark patterns on the apparel.
(2) Infrared video acquisition and framing processing
(2-1) Since the smeared near infrared anti-forgery ink is visible only in near infrared light, video data is collected by a near infrared camera. Considering that the near infrared light of the indoor stage is less, a plurality of high-power near infrared lamps are added above the stage obliquely, and the near infrared lamps with 300W are placed above the periphery of the stage obliquely in practical use because the illumination distance and intensity of the low-power light supplementing lamps can not enable the near infrared camera to capture near infrared information clearly in the test.
(2-2) Placing two near infrared cameras in front of the stage obliquely, collecting stage performance video data under near infrared light supplement, and decomposing the collected video into video frame images. Specifically, two near infrared cameras are placed at the inclined front position which forms an angle of 90 degrees with the center of a stage, and under the condition that near infrared ink marks are coated on the front and the back of the actor clothes, at least one of the two cameras can acquire ink mark information on the actor clothes, and images acquired by the two cameras are subjected to multi-angle image registration, so that actor ID information can be obtained in each frame, and the problem of lost caused by mutual shielding of actors is greatly reduced.
(3) Image annotation
And (3-1) labeling the acquired video frame image, wherein the video frame image specifically comprises an actor detection frame, an ink mark detection frame and corresponding class numbers thereof, the class numbers of the actor detection frame are person, and the class numbers of the ink mark detection frame are ink, and are used as a data set for training the detector.
And (3-2) expanding the ink mark detection frame in the step (3-1) outwards by a certain pixel, then cutting, dividing the cut image into N classes according to different ink mark patterns, giving corresponding class labels, wherein each class label represents an actor ID, and taking the data as a data set of a classifier.
(4) Detector and classifier training
(4-1) The detector uses yolov as the basic network skeleton, and the labeled dataset of (3-2) is processed according to 8: the scale of 2 is divided into training and test sets. And carrying out data enhancement on the training set data by MixUp, cutout, cutMix, mosaic and other methods to prevent the phenomenon of fitting the training, and sending the enhanced data batch into a target detection network for training.
(4-2) Classifier using MobileNetV2 as the basic network skeleton, data set made in (3-3) was prepared according to 8: the scale of 2 is divided into training and test sets. And uniformly scaling all the data to 100 x 200, performing data enhancement by adopting methods such as horizontal overturning, random erasing and the like, and sending K pictures of each class into a target classification network for training according to N classes.
(5) Object detection and classification
(5-1) When detecting video frame images, because the image acquired in real time by the near infrared camera only contains 1 channel, the near infrared channel is duplicated to become a three-channel image.
And (5-2) sending the three-channel image frame to a detector, and detecting all actors and ink marks in the picture to obtain the detection frame position and the confidence of each target.
(5-3) If the confidence of the actor's body is detected to be greater than a preset threshold, the detection is considered to be reliable. If the ink mark target is detected, the image is cut out by expanding a certain pixel outwards according to the range of the detection frame, so that a more complete ink mark is obtained, and the cut image is sent to the classifier. And classifying by the classifier according to the patterns of the ink marks to obtain the classification result of each ink mark and the corresponding confidence coefficient.
(6) ID stabilization
If the confidence coefficient corresponding to the ink mark classification result is smaller than the preset threshold value, the classification result is considered unreliable, and the type number of the low-confidence detection frame of the current frame is stabilized according to the type number of the ink mark detection frame with the largest IOU with the previous frame.
Specifically, whether the classification result is accurate is judged through the confidence coefficient, if the confidence coefficient corresponding to the classification result of the ink mark detection frame is smaller than a preset threshold value, the result is considered to be unreliable, false detection is possibly generated, the target classification is readjusted, and the adjustment method is as follows: the ink mark detection frame with the largest low confidence ink mark detection frame IOU in the previous frame image and the current frame is searched, and the class number is assigned to the current low confidence ink mark detection frame, so that the class number of the ink mark detection frame frequently switched in the tracking process is stabilized, only a target with a higher confidence value is guaranteed, the tracking accuracy is improved, and the frequent switching of the ID is avoided.
(7) Detection frame association
And carrying out one-to-one matching on the detected target actors in each frame and the classified ink marks, wherein the actor ID is the class number corresponding to the matched ink marks.
Specifically, calculating IOU for each actor detection frame and all ink mark detection frames in the current frame, if a plurality of matched IOUs are larger than or equal to a preset threshold, taking the largest ink mark detection frame of the IOU and the actor detection frame as optimal matching, wherein the actor ID is the class number corresponding to the best matched ink mark; and for the actor detection frames of which the matched ink marks are not found, namely, the IOU of the actor detection frame and all the ink mark detection frames in the current frame is smaller than a preset threshold, the actor is not considered to be a target actor to be tracked, and an ID is not assigned to the actor detection frame. The method solves the problem that the target classifier cannot accurately identify due to the fact that apparent features of actors in a stage environment are too similar to each other.
(8) Creating a tracking track
(8-1) Linking the video frame actor tracks according to actor IDs, linking actor positions of the same IDs to the previous frame, predicting each actor ID in the first frame, and detecting that the actor IDs of the same ID are linked to the ID actor tracks in the subsequent frame to form multi-target tracked actor tracks.
(8-2) For the case where a new target appears in the tracking process: firstly, obtaining a new incoming actor ID through the steps (5) - (7), then associating each target actor detection frame of the current frame with each target actor detection frame of the previous frame according to ID matching, and directly linking the actor detection frame into the prior tracking track if the same ID can be found in the tracked actors; if there is no tracked actor with the same ID as the new actor, a tracking track head is created for the new actor.
(8-3) For the case of actor departure during tracking: and (3) keeping the tracking tracks of all the actors from beginning to end, and if the actors leave the field and are on the field again, adding the information of the actor detection frames on which the field is on the field again into the previous tracking tracks through the steps (5) and (8).
The foregoing is merely a preferred embodiment of the present invention, and the present invention has been disclosed in the above description of the preferred embodiment, but is not limited thereto. Any person skilled in the art can make many possible variations and modifications to the technical solution of the present invention or modifications to equivalent embodiments using the methods and technical contents disclosed above, without departing from the scope of the technical solution of the present invention. Therefore, any simple modification, equivalent variation and modification of the above embodiments according to the technical substance of the present invention still fall within the scope of the technical solution of the present invention.
Claims (9)
1. The actor identity re-identification method based on near infrared anti-counterfeiting ink in theatre environment is characterized by comprising the following steps of:
(1) Collecting stage performance data to construct a training set, comprising:
(1.1) coating a specific pattern of marks on the target actor clothes by using near infrared anti-counterfeiting ink, wherein different patterns of marks represent different actor identities;
(1.2) using a near infrared light filling lamp to fill light in the theatre environment, and collecting an image sequence of the actor performing on a stage through a near infrared camera;
(1.3) labeling the actor body area, the ink mark area and the corresponding categories for the acquired near infrared image to form a data set of the detector;
Cutting the ink marking area of the near infrared image, and dividing the cut ink marking image into N classes according to different ink marking patterns to form a data set of a classifier;
(2) Training a detector based on a deep learning method and a classifier model, wherein the detector is used for detecting coordinate positions and corresponding categories of target actors and ink marks in a near infrared image, and the classifier is used for identifying and classifying ink marks in different patterns;
(3) Sending the current frame into a detector to detect all target actors and ink marks in the image, and sending the detected ink mark areas into a classifier after cutting the detected ink mark areas from the original image;
(4) The classifier classifies the input ink marked images according to the ink marked patterns, different class numbers represent different actor IDs, if the confidence coefficient of classification is smaller than a preset threshold value, the result is considered unreliable, and the class number of the ink marked detection frame with the largest low confidence coefficient ink marked detection frame IOU of the current frame in the previous frame is taken as the class number of the low confidence coefficient detection frame of the current frame;
(5) Carrying out one-to-one matching on the detected target actors and the ink marks in each frame, calculating the IOU of each actor detection frame and all the ink mark detection frames in the current frame, and taking the ink mark detection frame type number with the largest IOU as the actor ID of the target actor detection frame;
(6) And linking the video frame actor tracks according to the actor IDs, and sequentially linking actor detection frames with the same ID to form the actor tracks with multiple target tracking.
2. The method according to claim 1, wherein in the step (1.1), the front and rear of the clothes of the target actor are coated with marks of a specific pattern by using near infrared anti-counterfeit ink, so that the near infrared camera can shoot the ink marks on the actor as much as possible when the actor moves; the N pieces of clothes of the target actors are coated with N patterns in different modes, the patterns are selected to have numbers with enough view points, and accordingly the ID of the target actors is determined by identifying and classifying the ink mark patterns on the clothes.
3. The method of claim 1, wherein in the step (1.2), a plurality of high power near infrared light compensating lamps are installed obliquely above the theater so that the ink marks applied to the actor's apparel are photographed with a near infrared camera.
4. The method according to claim 1, wherein in the step (1.2), two near infrared cameras forming an angle of 90 degrees with the center of the stage are arranged obliquely in front of the stage, and in the case that near infrared anti-counterfeit ink marks are coated on the front and rear of the actor's clothes, at least one of the two cameras can acquire ink mark information on the actor's clothes, and images acquired by the two cameras are subjected to multi-angle image registration, so that actor ID information can be obtained for each frame.
5. The method according to claim 1, wherein in the step (1.4), the ink mark area of the near infrared image is expanded outward by a plurality of pixels and then cut, so as to obtain a more complete near infrared anti-counterfeit ink mark.
6. The method according to claim 1, wherein in the step (3), since the near infrared image acquired in real time contains only 1 channel, the near infrared channel is duplicated to be a three-channel image, and then the three-channel image is sent to the detector.
7. The method according to claim 1, wherein in the step (5), for each actor detection frame and all the ink mark detection frames in the current frame, an IOU is calculated, if a plurality of matched IOUs are greater than or equal to a preset threshold, the largest ink mark detection frame of the IOU and the actor detection frame are taken as the best match, and the actor ID is the class number corresponding to the best matched ink mark; and for the actor detection frames of which the matched ink marks are not found, namely, the IOU of the actor detection frame and all the ink mark detection frames in the current frame is smaller than a preset threshold, the actor is not considered to be a target actor to be tracked, and an ID is not assigned to the actor detection frame.
8. The method according to claim 1, wherein in the step (6), for the case where a new target appears in the tracking process: firstly, obtaining a new incoming actor ID through the steps (1) - (5), then associating each target actor detection frame of a current frame with each target actor detection frame of a previous frame according to ID matching, and directly linking the actor detection frame into the prior tracking track if the same ID can be found in the tracked actors; if there is no tracked actor with the same ID as the new actor, a tracking track head is created for the new actor.
9. The method of claim 1, wherein in the step (6), for the case of the actor being off-site during tracking: and (3) keeping the tracking tracks of all the actors from beginning to end, and if the actors leave the field and are on the field again, adding the information of the actor detection frames on which the field is on the field again into the previous tracking tracks through the steps (1) and (5).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111252678.5A CN113920164B (en) | 2021-10-27 | 2021-10-27 | Actor identity re-identification method based on near infrared anti-counterfeiting ink in theatre environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111252678.5A CN113920164B (en) | 2021-10-27 | 2021-10-27 | Actor identity re-identification method based on near infrared anti-counterfeiting ink in theatre environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113920164A CN113920164A (en) | 2022-01-11 |
| CN113920164B true CN113920164B (en) | 2024-05-24 |
Family
ID=79243169
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111252678.5A Active CN113920164B (en) | 2021-10-27 | 2021-10-27 | Actor identity re-identification method based on near infrared anti-counterfeiting ink in theatre environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113920164B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE19806646C1 (en) * | 1998-02-18 | 1999-08-12 | Gmd Gmbh | Camera tracking system for virtual television or video studio |
| US9176987B1 (en) * | 2014-08-26 | 2015-11-03 | TCL Research America Inc. | Automatic face annotation method and system |
| CN106909955A (en) * | 2017-02-22 | 2017-06-30 | 上海交通大学 | A kind of method for anti-counterfeit based on infrared external reflection |
| CN109886372A (en) * | 2019-01-29 | 2019-06-14 | 深圳市象形字科技股份有限公司 | It is a kind of that this recognition methods is drawn based on image recognition and stealthy infrared printing ink technology |
| CN110084831A (en) * | 2019-04-23 | 2019-08-02 | 江南大学 | Based on the more Bernoulli Jacob's video multi-target detecting and tracking methods of YOLOv3 |
| CN110298214A (en) * | 2018-03-23 | 2019-10-01 | 苏州启铭臻楠电子科技有限公司 | A kind of stage multi-target tracking and classification method based on combined depth neural network |
| CN111882586A (en) * | 2020-06-23 | 2020-11-03 | 浙江工商大学 | Multi-actor target tracking method oriented to theater environment |
| CN111914664A (en) * | 2020-07-06 | 2020-11-10 | 同济大学 | Vehicle multi-target detection and track tracking method based on re-identification |
| CN112926410A (en) * | 2021-02-03 | 2021-06-08 | 深圳市维海德技术股份有限公司 | Target tracking method and device, storage medium and intelligent video system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8330823B2 (en) * | 2006-11-01 | 2012-12-11 | Sony Corporation | Capturing surface in motion picture |
-
2021
- 2021-10-27 CN CN202111252678.5A patent/CN113920164B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE19806646C1 (en) * | 1998-02-18 | 1999-08-12 | Gmd Gmbh | Camera tracking system for virtual television or video studio |
| US9176987B1 (en) * | 2014-08-26 | 2015-11-03 | TCL Research America Inc. | Automatic face annotation method and system |
| CN106909955A (en) * | 2017-02-22 | 2017-06-30 | 上海交通大学 | A kind of method for anti-counterfeit based on infrared external reflection |
| CN110298214A (en) * | 2018-03-23 | 2019-10-01 | 苏州启铭臻楠电子科技有限公司 | A kind of stage multi-target tracking and classification method based on combined depth neural network |
| CN109886372A (en) * | 2019-01-29 | 2019-06-14 | 深圳市象形字科技股份有限公司 | It is a kind of that this recognition methods is drawn based on image recognition and stealthy infrared printing ink technology |
| CN110084831A (en) * | 2019-04-23 | 2019-08-02 | 江南大学 | Based on the more Bernoulli Jacob's video multi-target detecting and tracking methods of YOLOv3 |
| CN111882586A (en) * | 2020-06-23 | 2020-11-03 | 浙江工商大学 | Multi-actor target tracking method oriented to theater environment |
| CN111914664A (en) * | 2020-07-06 | 2020-11-10 | 同济大学 | Vehicle multi-target detection and track tracking method based on re-identification |
| CN112926410A (en) * | 2021-02-03 | 2021-06-08 | 深圳市维海德技术股份有限公司 | Target tracking method and device, storage medium and intelligent video system |
Non-Patent Citations (2)
| Title |
|---|
| 多任务网络融合多层信息的目标定位;田彦;王慧燕;王勋;黄刚;章国峰;;计算机辅助设计与图形学学报;20170715(第07期);全文 * |
| 自适应在线判别外观学习的分层关联多目标跟踪;方岚;于凤芹;;中国图象图形学报;20200415(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113920164A (en) | 2022-01-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ge et al. | Real-time pedestrian detection and tracking at nighttime for driver-assistance systems | |
| Chen et al. | A deep learning approach to drone monitoring | |
| Ogale | A survey of techniques for human detection from video | |
| CN103824070B (en) | A kind of rapid pedestrian detection method based on computer vision | |
| CN109919974A (en) | Online multi-object tracking method based on the more candidate associations of R-FCN frame | |
| CN102043953A (en) | Real-time-robust pedestrian detection method aiming at specific scene | |
| CN111460884A (en) | Multi-face recognition method based on human body tracking | |
| Galiyawala et al. | Person retrieval in surveillance video using height, color and gender | |
| Liang et al. | Methods of moving target detection and behavior recognition in intelligent vision monitoring. | |
| CN107103301B (en) | Method and system for matching discriminant color regions with maximum video target space-time stability | |
| CN110555867B (en) | Multi-target object tracking method integrating object capturing and identifying technology | |
| Lu et al. | A cross-scale and illumination invariance-based model for robust object detection in traffic surveillance scenarios | |
| Miller et al. | Person tracking in UAV video | |
| CN112509009B (en) | Target tracking method based on natural language information assistance | |
| Alagarsamy et al. | Identifying the Missing People using Deep Learning Method | |
| Hu et al. | Fast face detection based on skin color segmentation using single chrominance Cr | |
| Deshmukh et al. | Real-time traffic sign recognition system based on colour image segmentation | |
| CN117475353A (en) | Video-based abnormal smoke identification method and system | |
| CN113920164B (en) | Actor identity re-identification method based on near infrared anti-counterfeiting ink in theatre environment | |
| CN112347967A (en) | Pedestrian detection method fusing motion information in complex scene | |
| Li et al. | Detecting and tracking human faces in videos | |
| Yi et al. | Face detection method based on skin color segmentation and facial component localization | |
| KR100729265B1 (en) | A face detection method using difference image and color information | |
| CN110781769A (en) | Method for rapidly detecting and tracking pedestrians | |
| Sharma et al. | Multi-object tracking using TLD framework |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |