CN113920164A - Actor identity re-identification method based on near-infrared anti-counterfeiting ink in theater environment - Google Patents
Actor identity re-identification method based on near-infrared anti-counterfeiting ink in theater environment Download PDFInfo
- Publication number
- CN113920164A CN113920164A CN202111252678.5A CN202111252678A CN113920164A CN 113920164 A CN113920164 A CN 113920164A CN 202111252678 A CN202111252678 A CN 202111252678A CN 113920164 A CN113920164 A CN 113920164A
- Authority
- CN
- China
- Prior art keywords
- actor
- ink
- infrared
- target
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 230000008569 process Effects 0.000 claims abstract description 9
- 238000001514 detection method Methods 0.000 claims description 85
- 238000012549 training Methods 0.000 claims description 13
- 238000013459 approach Methods 0.000 claims description 10
- 230000001502 supplementing effect Effects 0.000 claims description 4
- 239000011248 coating agent Substances 0.000 claims description 3
- 238000000576 coating method Methods 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 claims description 3
- 238000005286 illumination Methods 0.000 abstract description 9
- 239000013589 supplement Substances 0.000 description 5
- 230000008859 change Effects 0.000 description 3
- 210000003128 head Anatomy 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- RZVAJINKPMORJF-UHFFFAOYSA-N Acetaminophen Chemical compound CC(=O)NC1=CC=C(O)C=C1 RZVAJINKPMORJF-UHFFFAOYSA-N 0.000 description 1
- 239000011358 absorbing material Substances 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 210000000887 face Anatomy 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Credit Cards Or The Like (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an actor identity re-identification method based on near-infrared anti-counterfeiting ink in a theater environment. According to the method, an invisible near-infrared anti-counterfeiting ink mark in a color camera is obtained through a near-infrared camera, then an image is sent to a trained detector and a classifier, and finally identity re-identification is achieved when similar actors in the appearance on a stage are tracked. The method uses the near-infrared anti-counterfeiting ink mark which is only visible by the near-infrared camera to replace the existing actor apparent information or the sensor as the key characteristic for distinguishing the actor identity, and has the following advantages: the interference of the complex illumination of the stage on the identification stability of the actors in the tracking process is greatly reduced, and the problem of difficult tracking caused by similar appearance of the actors is solved by introducing the special near-infrared anti-counterfeiting ink mark information. Compared with the identity recognition method based on the sensor, the method is low in cost, strong in operability and universal in stage performance.
Description
Technical Field
The invention relates to the technical field of multi-target tracking, in particular to an actor identity re-identification method based on near-infrared anti-counterfeiting ink in a theater environment.
Background
The target tracking task has wide application in various fields, such as the fields of automatic driving, security protection or medical treatment. While re-identification of identity (ReID) is a critical issue that must be addressed in the tracking task. The ReID aims to solve the problem that when the target is matched again, the ID is changed or restarted due to the factors of target interaction, shielding, illumination complexity and the like
The existing ReID method faces several major challenges in practical applications: the difference of the target appearances caused by the different visual angles of the cameras is huge; the mutual shielding and self shielding of targets cause the loss of partial effective features; the different illumination causes the characteristics of the target to change drastically; the apparent similarity of different objects results in insufficient feature discrimination.
The existing ReID algorithm based on deep learning can be roughly divided into three methods based on global features, auxiliary features and local features according to different image feature extraction modes. The global feature-based approach is to extract global feature information for each image, such as by global maximum pooling or global average pooling. The method based on the global features cannot solve the problem of interference caused by the background area, and has limitation in practical application. Therefore, researchers have begun to study methods based on assist features. Currently, this kind of method mainly adopts a generation countermeasure network (GAN) to augment a target data set, so as to achieve the purpose of improving the generalization of a classifier. However, pictures generated by the GAN network are often lack of authenticity, and have more redundant noise information and larger deviation from an actual application scene. In recent years, more and more research work has focused on the study of local features. If the image is horizontally cut into blocks, the segmented image is sent into a long-short term memory artificial neural network (LSTM) for training according to the sequence from the head to the body; the method of grid blocking is also adopted, the similar and different characteristics of the image pair are extracted in the same grid area, and then the characteristic information of all area grids is fused to carry out secondary classification on the input image, so as to judge whether the input image is the same ID or not. By adopting the method based on the local features, the identification accuracy of the ReID is obviously improved.
In recent years, the exploration and research of the ReID task have been greatly developed, and the possible directions in the future include robustness, fast learning, online learning and the like, and are basically ground-oriented. ReID in multi-target tracking tasks in stage scenarios also faces the following challenges: target loss caused by rapid action change and mutual shielding between target actors; the target actors are similar in appearance and same in dress, and compared with a common scene, the appearance difference is smaller, so that the apparent feature discrimination is lower, and the ID identification is more difficult; the stage illumination changes violently, which causes the target detection accuracy to be reduced. On the actual stage, no existing scheme can well solve the ReID problem under the application of actor-oriented tracking.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides an identity re-identification (ReiD) method for assisting multi-target actor tracking based on near-infrared anti-counterfeiting ink marks in a theater environment, wherein the near-infrared ink marks visible only by an infrared camera are used for replacing the existing actor apparent information or sensors as key features for distinguishing the actor identities, so that the interference of stage complex illumination on the identity identification stability in the actor tracking process is greatly reduced, and the problem of identification difficulty caused by actor apparent similarity is solved by introducing specific infrared ink mark information.
The anti-counterfeiting ink is prepared by adding one or more near-infrared absorbing materials into the ink, reflects visible light and is in an absorbing state for the light in a near-infrared band. Due to the infrared absorption characteristic of the near-infrared anti-counterfeiting ink, if the ink is used in a certain part of a printed product, no trace is generated in the sunlight, and a corresponding signal or image and text can be observed under a detection instrument.
Based on the inspiration of the method, the application applies the infrared anti-counterfeiting ink to multi-target tracking, firstly, the near-infrared ink is coated on the actor clothes in a pattern of a specific pattern, because the near-infrared light belongs to a light wave band invisible to human eyes, the coated pattern is invisible in the human eyes and a visible camera, but an obvious pattern can be shown in the near-infrared camera, so that the theater environment is supplemented with a near-infrared supplement lamp on a stage, and different ink coating marks on each actor can be shot by the near-infrared camera. The different ink symbols are used as apparent characteristics for distinguishing different actors, and the ReID of multi-target actor tracking is realized.
The invention provides an actor identity re-identification method based on near-infrared anti-counterfeiting ink in a theater environment, which comprises the following specific implementation flows of:
(1) collecting stage performance data to construct a training set, comprising:
(1.1) coating marks of specific patterns on target actor clothes by using near-infrared anti-counterfeiting ink, wherein the marks of different patterns represent identities of different actors;
(1.2) supplementing light to a theater environment by using a near-infrared light supplementing lamp, and collecting an image sequence of actors performing on the stage by using a near-infrared camera;
(1.3) marking the body area of an actor, the ink marking area and the corresponding category of the collected near-infrared image to form a data set of a detector;
(1.4) cutting an ink marking area of the near-infrared image, and dividing the cut ink marking image into N types according to different ink marking styles to form a data set of a classifier;
(2) training a detector and a classifier model based on a deep learning method, wherein the detector is used for detecting coordinate positions and corresponding categories of target actors and ink marks in a near-infrared image, and the classifier is used for identifying and classifying different styles of ink marks;
(3) sending the current frame to a detector to detect all target actors and ink marks in the image, cutting the detected ink mark area from the original image, and sending the cut ink mark area to a classifier;
(4) the classifier classifies the input ink marking images according to the ink marking patterns, different class numbers represent different actor IDs, if the confidence coefficient of the classification is smaller than a preset threshold value, the result is considered to be unreliable, and the class number of the ink marking detection frame with the highest IOU (input output unit) in the previous frame and the current frame is taken as the class number of the current frame low confidence coefficient detection frame;
(5) matching the target actor and the ink marks detected in each frame one to one, calculating the intersection and comparison (IOU) of each actor detection frame and all the ink mark detection frames in the current frame, and taking the ink mark detection frame type number with the largest IOU as the actor ID of the target actor detection frame;
(6) and linking the actor tracks of the video frames according to the actor IDs, and sequentially linking the actor detection frames with the same IDs to form the multi-target tracked actor tracks.
Further, in the step (1.1), marks of specific patterns are applied to the front and the back of the target actor clothes by using near-infrared anti-counterfeiting ink, so that the near-infrared camera can shoot the ink marks on the actor as much as possible when the actor moves; the clothing of the N target actors is painted with N patterns of different patterns, and the pattern patterns select numbers with enough view points, so that the ID of the target actors can be determined by identifying and classifying the ink mark patterns on the clothing.
Further, in the step (1.2), a plurality of high-power near-infrared light supplement lamps are installed above the theater in an inclined mode to supplement light, so that the ink marks painted on the actor clothes can be shot by a near-infrared camera.
Further, in the step (1.2), two near-infrared cameras forming an angle of 90 degrees with the center of the stage are arranged in the oblique front of the stage, and under the condition that near-infrared anti-counterfeiting ink marks are coated on the front and the back of the actor clothes, at least one of the two cameras can acquire ink mark information on the actor clothes, and images acquired by the two cameras are subjected to multi-angle image registration, so that actor ID information can be acquired in each frame.
Further, in the step (1.4), the ink mark area of the near-infrared image is extended outwards by a plurality of pixels and then cut, so as to obtain a more complete near-infrared anti-counterfeiting ink mark.
Further, in the step (3), since the near-infrared image obtained in real time only contains 1 channel, the near-infrared channel is copied to form a three-channel image, and then the three-channel image is sent to the detector.
Further, in the step (4), whether the classification result is accurate is determined according to the confidence level, if the confidence level corresponding to the classification result of the ink mark detection frame is smaller than the preset threshold, the result is considered to be unreliable, false detection may occur, and the target classification should be readjusted, where the adjustment method is: and searching the ink mark detection frame with the largest ink mark detection frame IOU in the previous frame image and the current frame image, and assigning the type number of the ink mark detection frame to the current ink mark detection frame, so as to stabilize the type number of the ink mark detection frame which is frequently switched in the tracking process and improve the tracking accuracy.
Further, in the step (5), an IOU is calculated for each actor detection frame and all ink mark detection frames in the current frame, if a plurality of matched IOUs are greater than or equal to a preset threshold, the ink mark detection frame with the largest IOU and the actor detection frame are taken as the best match, and the actor ID is the category number corresponding to the ink mark with the best match; and for the actor detection frames in which the matched ink marks are not found, namely the IOUs of a certain actor detection frame and all the ink mark detection frames in the current frame are smaller than a preset threshold value, the actor is considered not to be the target actor to be tracked, and the actor detection frames are not endowed with IDs.
Further, in the step (6), for a case that a new target appears in the tracking process: firstly, obtaining ID of a new approach actor through the steps (1) to (5), then associating each target actor detection frame of a current frame with each target actor detection frame of a previous frame according to ID matching, and directly linking the actor detection frame to a previous existing tracking track if the same ID can be found in the tracked actors; and if no tracked actor with the same ID as the new approach actor exists, establishing a tracking track head for the new approach actor.
Further, in the step (6), for the case that the actor leaves the scene during the tracking process: and (3) keeping the tracking tracks of all actors all the time, and if the actors are on the scene again after leaving the scene, adding the actor detection frame information on the scene again into the previous tracking tracks through the steps (1) to (5).
The invention provides an identity re-identification method based on near-infrared anti-counterfeiting ink marks to assist multi-target actor tracking in a theater environment, which is inspired by near-infrared anti-counterfeiting ink. Compared with the prior art, the invention has the following beneficial effects:
1. according to the invention, near-infrared ink marks of different styles are used for distinguishing different actor IDs instead of actor apparent characteristics, so that the problem of difficulty in tracking due to similar actor appearances is solved.
2. The invention uses near infrared camera to replace visible light camera. The light-sensitive part of the near-infrared camera receives near-infrared light and weak visible light, so that the interference of complex stage illumination on the identification stability of actors in the tracking process can be greatly reduced, the illumination does not need to be corrected on line according to the variation of the stage illumination, and the identification of the actors in scenes with severe stage light variation is guaranteed.
Drawings
FIG. 1 is a flow chart of a near-infrared anti-counterfeiting ink-based actor identity re-identification method in a theater environment according to the present invention;
fig. 2 is an overall structure diagram of an engineering implementation of the actor identity re-identification method according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating the effect of ink marks on apparel under different illuminations according to an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those specifically described and will be readily apparent to those of ordinary skill in the art without departing from the spirit of the present invention, and therefore the present invention is not limited to the specific embodiments disclosed below.
The embodiment of the invention provides an identity re-identification (ReiD) method based on near-infrared anti-counterfeiting ink marks to assist multi-target actor tracking in a theater environment, which comprises the following specific steps as shown in figures 1 and 2:
(1) infrared ink marking design and application
(1-1) for the video data of the stage performance, the invention uses near-infrared anti-counterfeiting ink to coat patterns with specific patterns on the front and the back of the target actor clothes, so that a camera can shoot ink marks on the actor as much as possible when the actor moves.
(1-2) painting the apparel of the N target actors with N different patterns, the pattern patterns selecting numbers with sufficient view points, as shown in fig. 3, whereby the target actor ID can be determined by identifying and classifying the ink mark patterns on the apparel.
(2) Infrared video acquisition and framing processing
And (2-1) acquiring video data by a near-infrared camera because the coated near-infrared anti-counterfeiting ink is only visible in near-infrared light. Considering that the indoor stage has less near infrared light, a plurality of high-power near infrared lamps are added above the stage in an inclined manner for light supplement, and in a test, the light distance and the intensity of the low-power light supplement lamps cannot enable a near infrared camera to clearly capture near infrared information, and 300W near infrared lamps are placed above the stage in an inclined manner in practical use.
And (2-2) placing two near-infrared cameras in front of the stage slope, acquiring stage performance video data under the near-infrared supplementary lighting, and decomposing the acquired video into video frame images. Specifically, two near-infrared cameras are placed in the oblique front position forming an angle of 90 degrees with the center of the stage, under the condition that near-infrared ink marks are coated on the front and the back of an actor garment, at least one of the two cameras can acquire ink mark information on the actor garment, and images collected by the two cameras are subjected to multi-angle image registration, so that actor ID information can be acquired in each frame, and the problem of loss caused by mutual blocking of actors is greatly reduced.
(3) Image annotation
(3-1) marking the collected video frame image, specifically comprising an actor detection frame, an ink mark detection frame and corresponding class numbers thereof, wherein the class number of the actor detection frame is 'person', the class number of the ink mark detection frame is 'ink', and the class numbers are used as a data set of a training detector.
And (3-2) expanding the ink mark detection frame in the step (3-1) outwards for a certain pixel, then cutting, dividing the cut image into N types according to different ink mark styles, giving corresponding category labels, wherein each category label represents an actor ID, and taking the data as a data set of a classifier.
(4) Detector and classifier training
(4-1) the detector adopts yolov5 as a basic network framework, and the labeled data set of (3-2) is obtained according to the ratio of 8: the scale of 2 is divided into a training set and a test set. And performing data enhancement on the training set data by using methods such as MixUp, Cutout, CutMix, Mosaic and the like, preventing the training from appearing according to an overfitting phenomenon, and sending the enhanced data into a target detection network in batches for training.
(4-2) the classifier uses MobileNetV2 as a basic network skeleton, and the data set produced in (3-3) is as follows: the scale of 2 is divided into a training set and a test set. All data are uniformly scaled to the size of 100 x 200, data enhancement is carried out by adopting methods of horizontal turning, random erasing and the like, and K pictures of each class are sent into a target classification network for training according to N classes.
(5) Object detection and classification
And (5-1) when the video frame image is detected, because the image acquired by the near-infrared camera in real time only contains 1 channel, copying the near-infrared channel to form a three-channel image.
And (5-2) sending the three-channel image frame to a detector, detecting all actors and ink marks in the image, and obtaining the position and confidence of the detection frame of each target.
And (5-3) if the confidence of the detected actor body is greater than a preset threshold, the detection is considered to be reliable. If the ink mark target is detected, the image is cut by extending a certain pixel outwards according to the range of the detection frame to obtain a more complete ink mark, and the cut image is sent to a classifier. And classifying by the classifier according to the pattern of the ink marks to obtain a classification result and a corresponding confidence coefficient of each ink mark.
(6) ID stabilization
And if the confidence corresponding to the classification result of the ink marks is smaller than a preset threshold, the classification result is considered to be unreliable, and the class number of the current frame low-confidence detection frame is stabilized according to the class number of the ink mark detection frame with the largest IOU of the previous frame.
Specifically, whether the classification result is accurate is judged through the confidence level, if the confidence level corresponding to the classification result of the ink mark detection frame is smaller than a preset threshold value, the result is considered to be unreliable, false detection may occur, the target classification should be readjusted, and the adjustment method is as follows: and searching the ink mark detection frame with the highest IOU (input/output) in the previous frame of image and the current frame of low confidence ink mark detection frame, and assigning the type number of the ink mark detection frame with the highest IOU to the current low confidence ink mark detection frame so as to stabilize the type number of the ink mark detection frame which is frequently switched in the tracking process, thus ensuring that only the target with higher confidence value exists, improving the tracking accuracy and avoiding the frequent switching of the ID.
(7) Detecting frame associations
And performing one-to-one matching on the target actor detected in each frame and the classified ink marks, wherein the actor ID is a class number corresponding to the matched ink mark.
Specifically, IOU is calculated for each actor detection frame and all ink mark detection frames in the current frame, if a plurality of matched IOU are more than or equal to a preset threshold value, the ink mark detection frame with the largest IOU and the actor detection frame are taken as the best match, and the actor ID is the class number corresponding to the ink mark with the best match; and for the actor detection frames in which the matched ink marks are not found, namely the IOUs of a certain actor detection frame and all the ink mark detection frames in the current frame are smaller than a preset threshold value, the actor is considered not to be the target actor to be tracked, and the actor detection frames are not endowed with IDs. The method solves the problem that the target classifier cannot accurately identify due to the fact that the apparent characteristics of actors in the stage environment are too similar to each other to a great extent.
(8) Creating a tracking trajectory
And (8-1) linking the actor tracks of the video frames according to the actor IDs, linking the actor positions with the same IDs into the previous frame, predicting each actor ID in the first frame, and linking the actor IDs with the same IDs detected in the subsequent frames into the actor tracks with the IDs to form the multi-target tracked actor tracks.
(8-2) for the case where a new target appears during tracking: firstly, obtaining ID of a new approach actor through the steps (5) to (7), then associating each target actor detection frame of a current frame with each target actor detection frame of a previous frame according to ID matching, and directly linking the actor detection frame to a previous existing tracking track if the same ID can be found in the tracked actors; and if no tracked actor with the same ID as the new approach actor exists, establishing a tracking track head for the new approach actor.
(8-3) for the case where the actor leaves the scene during tracking: and (3) keeping the tracking tracks of all actors all the time, and if the actors are on the scene again after leaving the scene, adding the actor detection frame information on the scene again into the previous tracking tracks through the steps (5) to (8).
The foregoing is only a preferred embodiment of the present invention, and although the present invention has been disclosed in the preferred embodiments, it is not intended to limit the present invention. Those skilled in the art can make numerous possible variations and modifications to the present teachings, or modify equivalent embodiments to equivalent variations, without departing from the scope of the present teachings, using the methods and techniques disclosed above. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention are still within the scope of the protection of the technical solution of the present invention, unless the contents of the technical solution of the present invention are departed.
Claims (10)
1. An actor identity re-identification method based on near-infrared anti-counterfeiting ink in a theater environment is characterized by comprising the following steps:
(1) collecting stage performance data to construct a training set, comprising:
(1.1) coating marks of specific patterns on target actor clothes by using near-infrared anti-counterfeiting ink, wherein the marks of different patterns represent identities of different actors;
(1.2) supplementing light to a theater environment by using a near-infrared light supplementing lamp, and collecting an image sequence of actors performing on the stage by using a near-infrared camera;
(1.3) marking the body area of an actor, the ink marking area and the corresponding category of the collected near-infrared image to form a data set of a detector;
(1.4) cutting an ink marking area of the near-infrared image, and dividing the cut ink marking image into N types according to different ink marking styles to form a data set of a classifier;
(2) training a detector and a classifier model based on a deep learning method, wherein the detector is used for detecting coordinate positions and corresponding categories of target actors and ink marks in a near-infrared image, and the classifier is used for identifying and classifying different styles of ink marks;
(3) sending the current frame to a detector to detect all target actors and ink marks in the image, cutting the detected ink mark area from the original image, and sending the cut ink mark area to a classifier;
(4) the classifier classifies the input ink marking images according to the ink marking patterns, different class numbers represent different actor IDs, if the confidence coefficient of the classification is smaller than a preset threshold value, the result is considered to be unreliable, and the class number of the ink marking detection frame with the highest IOU (input output unit) in the previous frame and the current frame is taken as the class number of the current frame low confidence coefficient detection frame;
(5) performing one-to-one matching on the target actor and the ink marks detected in each frame, calculating IOUs of each actor detection frame and all ink mark detection frames in the current frame, and taking the ink mark detection frame type number with the largest IOU as the actor ID of the target actor detection frame;
(6) and linking the actor tracks of the video frames according to the actor IDs, and sequentially linking the actor detection frames with the same IDs to form the multi-target tracked actor tracks.
2. The method according to claim 1, characterized in that in the step (1.1), the target actor clothing is marked with a specific pattern of marks in front of and behind the target actor with near-infrared anti-counterfeiting ink, so that the near-infrared camera can shoot the ink marks on the actor as much as possible when the actor moves; the clothing of the N target actors is painted with N patterns of different patterns, and the pattern patterns select numbers with enough view points, so that the ID of the target actors can be determined by identifying and classifying the ink mark patterns on the clothing.
3. The method according to claim 1, characterized in that in step (1.2), a plurality of high-power near-infrared fill lamps are installed obliquely above the theater so that ink marks applied to the actor's apparel can be photographed with a near-infrared camera.
4. The method according to claim 1, wherein in step (1.2), two near infrared cameras are arranged in the oblique front of the stage, which form an angle of 90 degrees with the center of the stage, and in the case that near infrared anti-counterfeiting ink marks are coated on the front and the back of the actor clothes, at least one of the two cameras can acquire the ink mark information on the actor clothes, and images acquired by the two cameras are subjected to multi-angle image registration, so that actor ID information can be acquired in each frame.
5. The method according to claim 1, wherein in the step (1.4), the ink mark area of the near-infrared image is expanded outward by a plurality of pixels and then cut to obtain a more complete near-infrared anti-counterfeiting ink mark.
6. The method according to claim 1, wherein in the step (3), since the real-time acquired near-infrared image contains only 1 channel, the near-infrared channel is copied to be a three-channel image, and then the three-channel image is sent to the detector.
7. The method according to claim 1, wherein in the step (4), whether the classification result is accurate is determined according to a confidence level, and if the confidence level corresponding to the classification result of the ink mark detection frame is smaller than a preset threshold, the result is considered to be unreliable, a false detection may occur, and the target classification should be readjusted, and the adjustment method is: and searching the ink mark detection frame with the largest ink mark detection frame IOU in the previous frame image and the current frame image, and assigning the type number of the ink mark detection frame to the current ink mark detection frame, so as to stabilize the type number of the ink mark detection frame which is frequently switched in the tracking process and improve the tracking accuracy.
8. The method according to claim 1, wherein in step (5), the IOU is calculated for each actor detection frame and all ink mark detection frames in the current frame, and if a plurality of matching IOUs are greater than or equal to a preset threshold, the ink mark detection frame with the largest IOU and the actor detection frame are taken as the best match, and the actor ID is the category number corresponding to the best matching ink mark; and for the actor detection frames in which the matched ink marks are not found, namely the IOUs of a certain actor detection frame and all the ink mark detection frames in the current frame are smaller than a preset threshold value, the actor is considered not to be the target actor to be tracked, and the actor detection frames are not endowed with IDs.
9. The method according to claim 1, wherein in the step (6), for the case that a new target appears in the tracking process: firstly, obtaining ID of a new approach actor through the steps (1) to (5), then associating each target actor detection frame of a current frame with each target actor detection frame of a previous frame according to ID matching, and directly linking the actor detection frame to a previous existing tracking track if the same ID can be found in the tracked actors; and if no tracked actor with the same ID as the new approach actor exists, establishing a tracking track head for the new approach actor.
10. The method according to claim 1, characterized in that in step (6), for the case that the actor leaves the scene during tracking: and (3) keeping the tracking tracks of all actors all the time, and if the actors are on the scene again after leaving the scene, adding the actor detection frame information on the scene again into the previous tracking tracks through the steps (1) to (5).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111252678.5A CN113920164B (en) | 2021-10-27 | 2021-10-27 | Actor identity re-identification method based on near infrared anti-counterfeiting ink in theatre environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111252678.5A CN113920164B (en) | 2021-10-27 | 2021-10-27 | Actor identity re-identification method based on near infrared anti-counterfeiting ink in theatre environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113920164A true CN113920164A (en) | 2022-01-11 |
| CN113920164B CN113920164B (en) | 2024-05-24 |
Family
ID=79243169
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111252678.5A Active CN113920164B (en) | 2021-10-27 | 2021-10-27 | Actor identity re-identification method based on near infrared anti-counterfeiting ink in theatre environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113920164B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE19806646C1 (en) * | 1998-02-18 | 1999-08-12 | Gmd Gmbh | Camera tracking system for virtual television or video studio |
| US20080100622A1 (en) * | 2006-11-01 | 2008-05-01 | Demian Gordon | Capturing surface in motion picture |
| US9176987B1 (en) * | 2014-08-26 | 2015-11-03 | TCL Research America Inc. | Automatic face annotation method and system |
| CN106909955A (en) * | 2017-02-22 | 2017-06-30 | 上海交通大学 | A kind of method for anti-counterfeit based on infrared external reflection |
| CN109886372A (en) * | 2019-01-29 | 2019-06-14 | 深圳市象形字科技股份有限公司 | It is a kind of that this recognition methods is drawn based on image recognition and stealthy infrared printing ink technology |
| CN110084831A (en) * | 2019-04-23 | 2019-08-02 | 江南大学 | Based on the more Bernoulli Jacob's video multi-target detecting and tracking methods of YOLOv3 |
| CN110298214A (en) * | 2018-03-23 | 2019-10-01 | 苏州启铭臻楠电子科技有限公司 | A kind of stage multi-target tracking and classification method based on combined depth neural network |
| CN111882586A (en) * | 2020-06-23 | 2020-11-03 | 浙江工商大学 | Multi-actor target tracking method oriented to theater environment |
| CN111914664A (en) * | 2020-07-06 | 2020-11-10 | 同济大学 | Vehicle multi-target detection and track tracking method based on re-identification |
| CN112926410A (en) * | 2021-02-03 | 2021-06-08 | 深圳市维海德技术股份有限公司 | Target tracking method and device, storage medium and intelligent video system |
-
2021
- 2021-10-27 CN CN202111252678.5A patent/CN113920164B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE19806646C1 (en) * | 1998-02-18 | 1999-08-12 | Gmd Gmbh | Camera tracking system for virtual television or video studio |
| US20080100622A1 (en) * | 2006-11-01 | 2008-05-01 | Demian Gordon | Capturing surface in motion picture |
| US9176987B1 (en) * | 2014-08-26 | 2015-11-03 | TCL Research America Inc. | Automatic face annotation method and system |
| CN106909955A (en) * | 2017-02-22 | 2017-06-30 | 上海交通大学 | A kind of method for anti-counterfeit based on infrared external reflection |
| CN110298214A (en) * | 2018-03-23 | 2019-10-01 | 苏州启铭臻楠电子科技有限公司 | A kind of stage multi-target tracking and classification method based on combined depth neural network |
| CN109886372A (en) * | 2019-01-29 | 2019-06-14 | 深圳市象形字科技股份有限公司 | It is a kind of that this recognition methods is drawn based on image recognition and stealthy infrared printing ink technology |
| CN110084831A (en) * | 2019-04-23 | 2019-08-02 | 江南大学 | Based on the more Bernoulli Jacob's video multi-target detecting and tracking methods of YOLOv3 |
| CN111882586A (en) * | 2020-06-23 | 2020-11-03 | 浙江工商大学 | Multi-actor target tracking method oriented to theater environment |
| CN111914664A (en) * | 2020-07-06 | 2020-11-10 | 同济大学 | Vehicle multi-target detection and track tracking method based on re-identification |
| CN112926410A (en) * | 2021-02-03 | 2021-06-08 | 深圳市维海德技术股份有限公司 | Target tracking method and device, storage medium and intelligent video system |
Non-Patent Citations (2)
| Title |
|---|
| 方岚;于凤芹;: "自适应在线判别外观学习的分层关联多目标跟踪", 中国图象图形学报, no. 04, 15 April 2020 (2020-04-15) * |
| 田彦;王慧燕;王勋;黄刚;章国峰;: "多任务网络融合多层信息的目标定位", 计算机辅助设计与图形学学报, no. 07, 15 July 2017 (2017-07-15) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113920164B (en) | 2024-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ogale | A survey of techniques for human detection from video | |
| Ge et al. | Real-time pedestrian detection and tracking at nighttime for driver-assistance systems | |
| Buehler et al. | Upper body detection and tracking in extended signing sequences | |
| CN100390811C (en) | Method for tracking multiple human faces from video in real time | |
| Lee et al. | Near-infrared-based nighttime pedestrian detection using grouped part models | |
| CN111460884A (en) | Multi-face recognition method based on human body tracking | |
| CN106570490A (en) | Pedestrian real-time tracking method based on fast clustering | |
| Liang et al. | Methods of moving target detection and behavior recognition in intelligent vision monitoring. | |
| CN107103301B (en) | Method and system for matching discriminant color regions with maximum video target space-time stability | |
| Lu et al. | A cross-scale and illumination invariance-based model for robust object detection in traffic surveillance scenarios | |
| Miller et al. | Person tracking in UAV video | |
| Nosheen et al. | Efficient Vehicle Detection and Tracking using Blob Detection and Kernelized Filter | |
| CN117475353A (en) | Video-based abnormal smoke identification method and system | |
| Vural et al. | Multi-view fast object detection by using extended haar filters in uncontrolled environments | |
| Weng et al. | Robust hand posture recognition integrating multi-cue hand tracking | |
| Song et al. | Combined face-body tracking in indoor environment | |
| CN113920164B (en) | Actor identity re-identification method based on near infrared anti-counterfeiting ink in theatre environment | |
| Sharma et al. | Multi-object tracking using TLD framework | |
| Pelisson et al. | Brand identification using gaussian derivative histograms | |
| Wen et al. | People tracking and counting for applications in video surveillance system | |
| Kovács et al. | Shape-and-motion-fused multiple flying target recognition and tracking | |
| Xian et al. | Target detection in mine based on fusion of color and edge information | |
| Jeong et al. | Moving shadow detection using a combined geometric and color classification approach | |
| Lin et al. | Smoking Behavior Detection Based on Hand Trajectory Tracking and Mouth Saturation Changes | |
| Lu et al. | Object tracking based on stable feature mining using intraframe clustering and interframe association |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |