CN111291692B - Video scene recognition method and device, electronic equipment and storage medium - Google Patents
Video scene recognition method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111291692B CN111291692B CN202010096738.8A CN202010096738A CN111291692B CN 111291692 B CN111291692 B CN 111291692B CN 202010096738 A CN202010096738 A CN 202010096738A CN 111291692 B CN111291692 B CN 111291692B
- Authority
- CN
- China
- Prior art keywords
- video
- layer structure
- scene
- target
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000013528 artificial neural network Methods 0.000 claims abstract description 48
- 230000009471 action Effects 0.000 claims abstract description 13
- 230000006870 function Effects 0.000 claims description 25
- 238000012549 training Methods 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 abstract description 4
- 238000004891 communication Methods 0.000 description 11
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000033001 locomotion Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 2
- 238000011423 initialization method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
- G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Mathematical Physics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention provides a video scene identification method, electronic equipment and a storage medium. The video scene recognition method comprises the following steps: inputting a target video of a scene to be identified into a target neural network, wherein the target neural network comprises a first layer structure and a second layer structure, the first layer structure is used for determining a position thermodynamic image of a key person according to the target video, and the action of the key person in the target video represents the candidate identification scene type of the target video; the second layer structure is used for determining the characteristic type of the target recognition scene from the candidate recognition scene types according to the target video and the thermal image of the position of the key person; and carrying out scene recognition on the video according to the target recognition scene type output by the target neural network. According to the video scene recognition method provided by the embodiment of the invention, as the target neural network can combine local characteristics and global information, the secondary scene of the video can be recognized under the condition that the video comprises multiple activities.
Description
Technical Field
The present invention relates to the field of video technologies, and in particular, to a method and apparatus for identifying a video scene, an electronic device, and a storage medium.
Background
In the classification process of the video, a video identification method is needed to determine a video scene, wherein the video scene can represent contents expressed by the video to a certain extent, such as whether the current video is a video of a football scene or a football scene, wherein the football scene is a primary scene of the video, and the football scene and the corner scene are secondary scenes of the video.
The recognition method in the prior art focuses on the primary scene recognition of a video, i.e. only the primary scene of the video can be recognized, for example, the video is recognized as a basketball game scene, a football game scene or a volleyball game scene, but it is difficult to recognize the secondary scene of the video, for example, it is difficult to recognize the secondary scene of the video scene of a basketball game, for example, a basketball shot scene, a robbery scene or a cap scene.
Disclosure of Invention
The embodiment of the invention provides a video scene recognition method, a device, electronic equipment and a storage medium, which are used for solving the problem that the prior art is difficult to use for secondary scene recognition including multi-person activities in videos.
In one aspect, an embodiment of the present invention provides a video scene recognition method, including: inputting a target video of a scene to be identified into a target neural network, wherein the target neural network comprises a first layer structure and a second layer structure, the first layer structure is used for determining a position thermal image of a key person according to the target video, and the action of the key person in the target video represents the candidate identification scene type of the target video; the second layer structure is used for determining a target recognition scene characteristic type from the candidate recognition scene types according to the target video and the position thermal image of the key person; and carrying out scene recognition on the video according to the target recognition scene type output by the target neural network.
According to one embodiment of the invention, the first layer structure comprises a first sub-layer structure and at least one second sub-layer structure; the first sublayer structure is used for determining a position thermodynamic image of a key person according to the target video; the second sub-layer structure is used for optimizing the position thermodynamic image of the key person output by the first sub-layer structure according to the target video; or the second sub-layer structure is used for re-optimizing the optimized position thermal image of the key person output by the previous second sub-layer structure according to the target video.
According to one embodiment of the invention, the first layer structure and the second layer structure are trained based on a loss function.
According to one embodiment of the invention, the loss function is a formula
Wherein Y is 1 、Y 2 、Y 3 Respectively representing the output of the first sublayer structure, the second sublayer structure and the second layer structure, representing the sample label of the position thermodynamic image of the input key character, representing the video scene by T, representing the image frame number in the video sample of the scene to be identified by N, representing the image length in the video sample of the scene to be identified by W, representing the image width in the video sample of the scene to be identified by H,output Y characterizing the kth image sample 1 The value of the pixel point at the position where the x-coordinate is i and the y-coordinate is j,/>Output Y characterizing the kth image sample 2 The value of the pixel point at the position where the x coordinate is i and the y coordinate is j; />The value of the pixel point at the j position with the x coordinate being i and the y coordinate being j in the kth image sample in the input sample label G; />Is the value of the s-th dimension data in T; />Is Y 3 The values of the s-th dimension data, p and m are preset parameters.
According to one embodiment of the present invention, a pixel point of the thermal image of the location of the key person, where the key person is located, is marked with 1.
In another aspect, an embodiment of the present invention provides a video recognition apparatus, including: the scene feature type recognition unit is used for inputting a target video of a scene to be recognized into a target neural network, wherein the target neural network comprises a first layer structure and a second layer structure, the first layer structure is used for determining a position thermodynamic image of a key person according to the target video, and the action of the key person in the target video represents the candidate recognition scene type of the target video; the second layer structure is used for determining a target recognition scene characteristic type from the candidate recognition scene types according to the target video and the position thermal image of the key person; and the video scene recognition unit is used for recognizing the video according to the target recognition scene type output by the target neural network.
According to one embodiment of the invention, the first layer structure comprises a first sub-layer structure and at least one second sub-layer structure; the first sublayer structure is used for determining a position thermodynamic image of a key person according to the target video; the second sub-layer structure is used for optimizing the position thermodynamic image of the key person output by the first sub-layer structure according to the target video; or the second sub-layer structure is used for re-optimizing the optimized position thermal image of the key person output by the previous second sub-layer structure according to the target video.
According to one embodiment of the invention, the first layer structure and the second layer structure are trained based on a loss function.
In yet another aspect, an embodiment of the present invention provides an electronic device including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the video scene recognition method described above when the program is executed.
In yet another aspect, embodiments of the present invention provide a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the video scene recognition method described above.
According to the video scene recognition method, the device, the electronic equipment and the storage medium, the target neural network is designed to extract the position characteristics of the key characters in the time domain and the space domain, and the target neural network is combined with the local characteristics and the comprehensive judgment of the global information, so that the secondary scene of the video can be recognized under the condition that the video comprises multiple people.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a video scene recognition method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a first layer structure and a second layer structure in a video scene recognition method according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a video recognition device according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The following describes a video scene recognition method according to an embodiment of the present invention with reference to fig. 1 to 2.
It should be noted that, the video scene may represent what is expressed by the video, such as whether the current video is a shooting scene about a football or a corner scene about a football, where the football scene is a primary scene of the video, and the shooting scene and the corner scene are secondary scenes of the video. The video scene recognition method provided by the embodiment of the invention can recognize the secondary scene corresponding to the video.
As shown in fig. 1, the video scene recognition method includes:
step S100, inputting a target video of a scene to be identified into a target neural network, wherein the target neural network comprises a first layer structure and a second layer structure, the first layer structure is used for determining a position thermal image of a key person according to the target video, and the action of the key person in the target video represents the candidate identification scene type of the target video; the second layer structure is used for determining the characteristic type of the target recognition scene from the candidate recognition scene types according to the target video and the thermal images of the positions of the key characters.
It can be understood that in the practice of applying the method, the target video input into the first layer structure is a video segment to be identified, the video segment includes N frames, N position thermal images of the key person output by the first layer structure are in one-to-one correspondence with the N frames of the target video, and each frame of the input target video is corresponding to a position thermal image of the key person.
The position thermal images of the key characters are respectively used for representing the position information of the key characters in the video frames corresponding to the position thermal images of the key characters, such as for a football video, and the position thermal images of the key characters are used for representing the position information of goalkeepers in the corresponding video frames, and the actions of the goalkeepers represent candidate identification scene types of the target video, such as putting out a ball, putting out a goal, and the like.
The first layer structure is obtained by training with video sample data as a sample and predetermined position thermal image sample data of a key person corresponding to the video sample data as a sample label.
In other words, the training samples of the first layer structure are: video sample data; the sample label is: the method comprises the steps of presetting position thermal image sample data of a key person corresponding to video sample data; for the video sample data of N frames, there are N positions of thermal image sample data.
The key character position thermal image sample data is a thermal image (hectmap) predetermined based on the video sample data, each video frame (image) sample data corresponds to one key character position thermal image sample data, and the key character position thermal image sample data is used for representing the position information of the key character in the corresponding video sample data.
The thermal image sample data of the positions of the key characters serving as sample labels can be manually marked or obtained one by one based on other single-frame picture identification methods.
It should be noted that, the first layer structure identifies local features in the video, that is, the step may extract time domain and space domain information in the video, and obtain information of the key person in the time domain and the space domain.
The target video input to the second layer structure is the same as the target video input to the first layer structure.
The second layer structure is obtained by training by taking target video sample data and a position thermal image of a key person output by the first layer structure as samples and taking a predetermined scene feature type corresponding to the video sample data as a sample label.
In other words, the training samples of the second layer structure are: target video sample data and a position thermal image of a key person output in the first layer structure; the sample label is: a predetermined scene feature type corresponding to the video sample data.
It should be noted that the scene feature type may be a secondary type of video, such as a shot video of football or a basketball shot video.
The scene feature type as a sample tag may be manually annotated.
It can be understood that the target video and the output result of the first layer structure need to be input in the second layer structure, wherein the output result of the first layer structure, namely the position thermal image of the key person is used for representing the position information of the key person, is local information, the action of the key person in the target video represents candidate identification scene types of the target video, and the target video can be used for representing global information, so that the second layer structure can capture all environmental information while focusing on the local features, and the identification result is more accurate.
Step 200, performing scene recognition on the video according to the target recognition scene type output by the target neural network.
Therefore, the method can be used for accurately identifying the key characters (local characteristics) in the multi-person scene by identifying the local characteristics through the first layer structure, and accurately identifying the real actions of the key characters by taking the local characteristics and the global characteristics into consideration through the second layer structure, so that the fine classification of the video is realized.
According to the video scene identification method, the first layer structure can acquire the information of the local features in the time domain and the space domain, the second layer structure can combine the local features and the global information, and for the case that the video comprises multi-person activities, the method can identify the secondary scene of the video.
In one embodiment of the present invention, the pixel point of the thermal image of the location of the key person in the step S100 is marked with 1.
For example, the size of the input target video is n×w×h×3, N denotes the number of frames, W denotes the length of an image (video frame), H denotes the width of an image (video frame), and 3 denotes RGB three-channel data; the sizes of the output N thermal images of the positions of the key characters are N, W, H and N, wherein N represents the number of frames, W represents the length of the image, H represents the width of the image, the positions of the pixels where the local features are located are marked as 1, and the other positions are marked as 0. Thus, the position of the local feature can be represented through the pixel point image.
In one embodiment of the invention, the first layer structure comprises a first sub-layer structure and at least one second sub-layer structure; the first sublayer structure is used for determining a position thermodynamic image of a key person according to the target video; the second sub-layer structure is used for optimizing the position thermodynamic image of the key person output by the first sub-layer structure according to the target video; or the second sub-layer structure is used for re-optimizing the optimized position thermodynamic image of the key character output by the previous second sub-layer structure according to the target video.
The first sublayer structure is obtained by training with video sample data as a sample and position thermal image sample data of a key person corresponding to the video sample data as a sample label.
In other words, the training samples of the first sublayer structure are: video sample data; the sample label is: and the position thermal image sample data of the key person corresponds to the video sample data.
The key character position thermal image sample data is a thermal image (hectmap) predetermined based on the video sample data, each video frame (image) sample data corresponds to one key character position thermal image sample data, and the key character position thermal image sample data is used for representing position information of the key character in the corresponding video frame sample data.
The thermal image sample data of the positions of the key characters serving as sample labels can be manually marked or obtained one by one based on other single-frame picture identification methods.
The target video input to the second sub-layer structure is the same as the target video input to the first sub-layer structure.
The second sub-layer structure connected with the first sub-layer structure is obtained by training with the target video sample data and the position thermal image of the key person output by the first sub-layer structure as samples and the position thermal image sample data of the key person corresponding to the video sample data as sample labels.
In other words, the training samples of the second sub-layer structure are: the video sample data and the position thermal image of the key person output by the first sublayer structure; the sample label is: and the position thermal image sample data of the key person corresponds to the video sample data.
The second sub-layer structure connected with the second sub-layer structure is obtained by training with the target video sample data and the position thermal image of the key person output by the previous second sub-layer structure as samples and the position thermal image sample data of the key person corresponding to the video sample data as sample labels.
The thermal image sample data of the positions of the key characters serving as sample labels can be manually marked or obtained one by one based on other single-frame picture identification methods.
It will be appreciated that the target video (video to be identified) and the output result of the first sub-layer structure or the previous second sub-layer structure need to be input in the second sub-layer structure.
In this embodiment, since the first layer structure is iterated at least twice to obtain the local feature, accuracy of local feature recognition can be greatly improved. The number of iterations is not limited to two in this embodiment, and more iterations may be performed, and the more iterations, the more accurate the calculation, and of course, the more calculation time and calculation resources are consumed.
It should be noted that, taking the first sub-layer structure and the second sub-layer structure of the first layer structure as the first-level neural network, the second layer structure is also the first-level neural network as an example, the embodiment is implemented through cascaded three-level neural networks, the first sub-layer structure and the second sub-layer structure are used for identifying and capturing local features, and the second layer structure is used for identifying the scene of the video.
As shown in fig. 3, in this embodiment, the first layer structure and the second layer structure form a cascaded three-level neural network, wherein the target video is input to a first sub-layer structure of the first layer structure, a second sub-layer structure of the first layer structure, and the second layer structure, and an output of the first sub-layer structure is also an input of the second sub-layer structure, and an output of the second sub-layer structure is also an input of the second layer structure.
In some embodiments of the invention, the first layer structure and the second layer structure are trained based on a loss function, and the loss function is determined based on the first loss function of the first layer structure and the second loss function of the second layer structure.
The loss function comprises a first loss function and a second loss function, wherein the first loss function aims at a first layer structure, namely the first loss function aims at the first sub-layer structure and a second sub-layer structure (a first two-stage neural network), and the deviation of the output of the first two-stage neural network and the identification of the key character features is calculated; and the second loss function aims at the second layer structure, and calculates the deviation between the video scene output by the second layer structure and the real result.
Because the output results of the first layer structure and the second layer structure have strong correlation, the first two-stage neural network outputs local features (key characters) in the video, and the second layer structure (second layer structure) is a video scene corresponding to the key characters.
The larger the residuals of the first and second sub-layer structures, the larger the error that accounts for local feature recognition. Similarly, the larger the deviation of video scene recognition is, the more the recognition results of the local features of the first sub-layer structure and the second sub-layer structure are, and the classification results of the second layer structure are directly affected. And the residual error of the second layer structure result can act on the first sub-layer structure and the second sub-layer structure in the same way, so that the recognition result of the local feature is optimized.
Therefore, in this embodiment, the loss function optimizes the first layer structure and the second layer structure at the same time, and the more accurate the local feature recognition of the first two-stage network, the better the classification result of the second layer structure; and the residual error of the classification result of the second layer structure also promotes the increase of the recognition accuracy of the first two stages.
The solution can promote the rapid convergence of the network and improve the accuracy of the classification algorithm.
Specifically, the target video includes N images with a size w×h×3, the size of the thermal image of the position of the key person is w×h×1, W represents the length of the image, H represents the width of the image, 3 represents RGB three-channel data, 1 represents the pixel value, the result of the second layer structure output is an (m+1) dimensional vector, and m represents the number of video scenes.
The loss function is a formula
Wherein Y is 1 、Y 2 、Y 3 Respectively representing the output of the first sublayer structure, the second sublayer structure and the second layer structure, wherein G represents the sample label of the position thermodynamic image of the input key character, T represents the video scene, N represents the image frame number in the video sample of the scene to be identified, W represents the image length in the video sample of the scene to be identified, H represents the image width in the video sample of the scene to be identified,output Y characterizing the kth image sample 1 The value of the pixel point at the position where the x-coordinate is i and the y-coordinate is j,/>Output Y characterizing the kth image sample 2 The value of the pixel point at the position where the x coordinate is i and the y coordinate is j; />The value of the pixel point at the j position with the x coordinate being i and the y coordinate being j in the kth image sample in the input sample label G; />Is the value of the s-th dimension data in T; />Is Y 3 The values of the s-th dimension data, p and m are preset parameters.
It can be understood that in the training method, the first layer structure is trained first, and then the first layer structure and the second layer structure are combined and trained, so that the first layer structure which is generally accurate is trained to a certain extent, the speed of the step S320 is increased, and in the step S320, the first layer structure and the second layer structure are combined and trained, so that the recognition accuracy of two recognition models can be rapidly improved at the same time based on the strong correlation of the first layer structure and the second layer structure.
In the following, the method for identifying a video scene according to the embodiment of the present invention includes:
step S310, initializing a first layer structure, taking video sample data as a sample, taking predetermined position thermal image sample data of a key person corresponding to the video sample data as a sample label, and training the first layer structure.
Step S320, initializing a second layer structure, taking video sample data and a position thermal image of a key person output by the first layer structure as samples, taking a predetermined video scene corresponding to the video sample data as a sample label, and training the first layer structure and the second layer structure.
In embodiments in which the first layer structure comprises a first sub-layer structure and a second sub-layer structure, the training of the first layer structure and the second layer structure comprises:
step S301, initializing a first sub-layer structure and a second sub-layer structure by adopting a random initialization method, inputting video sample data extracted from a video, wherein the size of the video sample data is N, W, H and 3, and inputting position thermal image sample data of a key character with the label of N, W, H and 1; training the first sub-layer structure and the second sub-layer structure by adopting an Adam Optimaizer, initializing lr to 0.001, and carrying out iterative training 10 times.
Step S302, setting training parameters of the first sublayer structure, the second sublayer structure and the second layer structure, including learning rate lr (0.001), p=0.5 of the Loss function, optimization algorithm, maximum iteration times, learning rate attenuation parameters, and the like, and randomly initializing the second layer structure.
In step S303, video sample data is input, the size is n×w×h×3, the position thermal image sample data of the key person with the label of n×w×h×1 is input, and the scene T to which the video belongs is input.
Step S304, inputting video sample data into the target neural network, and calculating Loss function L ((G, Y) according to the output results of the first stage, the second stage and the third stage of the network 1 )，(G，Y 2 )，(T，Y 3 ) According to the back propagation algorithm, the residual error of each stage is obtained.
Step S305, updating the weight value in the target neural network by adopting a back propagation algorithm based on the calculated residual error.
Step S306, return to step S304.
In a specific embodiment, the video scene recognition method can be used for recognizing a multi-person motion video, namely, an input target video expresses a multi-person motion scene.
The embodiment of the invention constructs a cascaded three-level neural network for processing the video classification problem of the motions of multiple people. Taking football recognition as an example, assuming that a scene to be recognized comprises m types of angular balls, shot balls, shooting, playing cards and the like, the input of a model is a target video, the size of each video frame is W.times.H.times.3, the width of an image is W, the height is H, and 3 is RGB three-channel data. The size of the model input is N x W x H x 3. The result of the final output of the model is an (m+1) dimensional vector (containing a background class, i.e. all but the scene to be identified are called background classes), each dimension representing the probability of belonging to that scene, the sum of which is 1. In addition, in order to capture a local feature, namely a key character, in a video, the first sub-layer structure and the second sub-layer structure of the model output a thermal image of the position of the key character with the ratio of N to W to H to 1, wherein the pixel point value in the i (1.ltoreq.i.ltoreq.N) frame represents the probability that the point belongs to the key character.
The model structure is shown in fig. 2, and the model is composed of 3 cascaded neural networks, a first layer structure 410 (a first sublayer structure 411 and a second sublayer structure 412) is used for identifying and capturing key characters in a video, and a second layer structure 420 is used for identifying a secondary scene to which a target video (video clip) belongs.
Model input is a group of N, W, H,3 images, which are target videos extracted from videos and marked as X; meanwhile, in the training process, the sample data of the thermal image of the position of the key person with the number of N, W and H is still required to be input, namely, the image of the position of the key person is marked and marked as G, wherein the position of the key person is marked as 1, otherwise, the position of the key person is marked as 0; in order to identify the scene to which the target video belongs, a scene T needs to be marked. The ith stage output of the model is denoted Y i Wherein i is more than or equal to 1 and less than or equal to 3.
The first sub-layer structure and the second sub-layer structure can be the deduction of the structure of Hourslass from 2D convolution to 3D convolution. I.e. by repeating the two-dimensional convolution kernel, the two-dimensional convolution kernel is converted into a 3D convolution kernel. The two-dimensional Hoursclass convolution network can be used for human body posture estimation, human face key point detection and the like by repeatedly carrying out bottom-up and top-down and combining supervision of intermediate results, so that different spatial positions of local features can be well utilized, and the recognition result can be optimized. The invention deduces the 2D convolution to the 3D convolution, so that not only the spatial position of the local information of the single-frame picture can be captured, but also the spatial position information of the target video can be further captured according to the continuity and the relativity of the images in the video, and the key character area of the target video can be identified.
According to the video scene recognition method, according to the key character information captured by the first sub-layer structure and the second sub-layer structure, global information in an original video (target video) is combined to serve as input of the second layer structure, surrounding environment information is captured while local characteristics are focused, and for the condition that the video comprises multiple people, the method can recognize the secondary scene of the video.
It will be appreciated that the prior art identification methods either have only partial features, which makes it difficult to distinguish between motions that are similar, such as a goal in a football, and a goal, and motions of the player are similar, but the background environments are different, one being just opposite the goal and the other being at a corner of the course.
Therefore, the video scene recognition method provided by the embodiment of the invention can accurately recognize the secondary classification of the video scene by combining the capturing of global (background) and key local characteristics aiming at the complexity of the secondary classification labels in the multi-person recognition scene.
For this cascaded three-level neural network, the loss function design is as follows, where p (0 < p < 1), here we choose p=0.5:
wherein Y is 1 、Y 2 、Y 3 The first sublayer structure, the second sublayer structure and the output of the second layer structure are respectively, G is a sample label of the input position thermodynamic image of the key person, T is a video scene,output Y for the kth frame sample l Values of pixel points at positions where x coordinates are i and y coordinates are j in (l=1, 2); />The value of the pixel point at the position where the x coordinate is i and the y coordinate is j in the kth frame sample in the input label G; />Is the T thValues of s-dimensional data; />Is Y 3 Values of the s-th dimensional data.
The implementation method of the invention is described below with reference to a network structure diagram and football scene recognition, and in order to make the model convergence effect better, the pre-neural two-stage network is initially trained to obtain the approximate convergence result, and then L ((G, Y) is utilized 1 )，(G，Y 2 )，(T，Y 3 ) Training a cascaded three-level neural network:
step S301, initializing a first sub-layer structure and a second sub-layer structure by adopting a random initialization method, inputting target video sample data extracted from a video, wherein the size of the target video sample data is N, W, H and 3, and inputting position thermal image sample data of a key character with a label of N, W, H and 1; training the first sub-layer structure and the second sub-layer structure by adopting an Adam Optimaizer, initializing lr to 0.001, and carrying out iterative training 10 times.
Step S302, setting training parameters of the first sublayer structure, the second sublayer structure and the second layer structure, including learning rate lr (0.001), p=0.5 of the Loss function, optimization algorithm, maximum iteration times, learning rate attenuation parameters, and the like, and randomly initializing the second layer structure.
In step S303, target video sample data with size n×w×h×3 is input, and the position thermal image sample data of the key person with label n×w×h×1 is input, and the scene T to which the target video belongs is input. The scene tag of the video is set according to the requirement, for example, football can be set as follows: 0 for background class, 1 for corner balls, 2 for arbitrary balls, 3 for shooting, etc.
Step S304, inputting the target video sample data into the cascade three-level neural network, and calculating Loss function L ((G, Y) according to the output results of the first, second and third levels of the network 1 )，(G，Y 2 )，(T，Y 3 ) According to the back propagation algorithm, the residual error of each stage is obtained.
And step 305, updating the weight value in the cascade three-level neural network by adopting a back propagation algorithm based on the calculated residual error.
Step S306, return to step S304.
The video recognition device provided by the embodiment of the invention is described below, and the video recognition device described below and the video scene recognition method described above can be referred to correspondingly.
As shown in fig. 3, the video recognition apparatus according to the embodiment of the present invention includes: scene feature type recognition unit 510, video scene recognition unit 520.
The scene feature type recognition unit 510 is configured to input a target video of a scene to be recognized into a target neural network, where the target neural network includes a first layer structure and a second layer structure, the first layer structure is configured to determine a thermal image of a position of a key person according to the target video, and an action of the key person in the target video characterizes a candidate recognition scene type of the target video; the second layer structure is used for determining a target recognition scene characteristic type from the candidate recognition scene types according to the target video and the position thermal image of the key person;
the first layer structure is obtained by training with video sample data as a sample and predetermined position thermal image sample data of a key person corresponding to the video sample data as a sample label.
The second layer structure is obtained by training by taking video sample data and the position thermal image of the key person output by the first layer structure as samples and taking a predetermined video scene corresponding to the video sample data as a sample label.
The video scene recognition unit 520 is configured to perform scene recognition on the video according to the target recognition scene type output by the target neural network.
In some embodiments, for the scene feature type identification unit 510, the first layer structure includes a first sub-layer structure and at least one second sub-layer structure; the first sublayer structure is used for determining a position thermodynamic image of a key person according to the target video; the second sub-layer structure is used for optimizing the position thermodynamic image of the key person output by the first sub-layer structure according to the target video; or the second sub-layer structure is used for re-optimizing the optimized position thermodynamic image of the key character output by the previous second sub-layer structure according to the target video.
In some embodiments, for the scene feature type identification unit 510, the first layer structure and the second layer structure are trained based on a loss function.
Fig. 4 illustrates a physical schematic diagram of an electronic device, as shown in fig. 4, which may include: processor 810, communication interface (Communications Interface) 820, memory 830, and communication bus 840, wherein processor 810, communication interface 820, memory 830 accomplish communication with each other through communication bus 840. The processor 810 may call logic instructions in the memory 830 to perform the following video scene recognition method: inputting a target video of a scene to be identified into a target neural network, wherein the target neural network comprises a first layer structure and a second layer structure, the first layer structure is used for determining a position thermodynamic image of a key person according to the target video, and the action of the key person in the target video represents the candidate identification scene type of the target video; the second layer structure is used for determining the characteristic type of the target recognition scene from the candidate recognition scene types according to the target video and the thermal image of the position of the key person; and carrying out scene recognition on the video according to the target recognition scene type output by the target neural network.
It should be noted that, in this embodiment, the electronic device may be a server, a PC, or other devices in the specific implementation, so long as the structure of the electronic device includes a processor 810, a communication interface 820, a memory 830, and a communication bus 840 as shown in fig. 4, where the processor 810, the communication interface 820, and the memory 830 complete communication with each other through the communication bus 840, and the processor 810 may call logic instructions in the memory 830 to execute the above method. The embodiment does not limit a specific implementation form of the electronic device.
Further, the logic instructions in the memory 830 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Further, embodiments of the present invention disclose a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the video scene recognition method provided by the above-described method embodiments, for example, the video scene recognition method comprising: inputting a target video of a scene to be identified into a target neural network, wherein the target neural network comprises a first layer structure and a second layer structure, the first layer structure is used for determining a position thermodynamic image of a key person according to the target video, and the action of the key person in the target video represents the candidate identification scene type of the target video; the second layer structure is used for determining the characteristic type of the target recognition scene from the candidate recognition scene types according to the target video and the thermal image of the position of the key person; and carrying out scene recognition on the video according to the target recognition scene type output by the target neural network.
In another aspect, embodiments of the present invention further provide a non-transitory computer readable storage medium having stored thereon a computer program that, when executed by a processor, is implemented to perform the video scene recognition method provided in the above embodiments, for example, the video scene recognition method includes: inputting a target video of a scene to be identified into a target neural network, wherein the target neural network comprises a first layer structure and a second layer structure, the first layer structure is used for determining a position thermodynamic image of a key person according to the target video, and the action of the key person in the target video represents the candidate identification scene type of the target video; the second layer structure is used for determining the characteristic type of the target recognition scene from the candidate recognition scene types according to the target video and the thermal image of the position of the key person; and carrying out scene recognition on the video according to the target recognition scene type output by the target neural network.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (5)
1. A method for identifying video scenes, comprising:
inputting a target video of a scene to be identified into a target neural network, wherein the target neural network comprises a first layer structure and a second layer structure, the first layer structure is used for determining a position thermal image of a key person according to the target video, and the action of the key person in the target video represents the candidate identification scene type of the target video; the second layer structure is used for determining a target recognition scene characteristic type from the candidate recognition scene types according to the target video and the position thermal image of the key person;
performing scene recognition on the video according to the target recognition scene feature type output by the target neural network;
the first layer structure comprises a first sub-layer structure and at least one second sub-layer structure;
the first sublayer structure is used for determining a position thermodynamic image of a key person according to the target video; the second sub-layer structure is used for optimizing the position thermodynamic image of the key person output by the first sub-layer structure according to the target video; or the second sub-layer structure is used for re-optimizing the optimized position thermal image of the key person output by the previous second sub-layer structure according to the target video;
training to obtain the first layer structure and the second layer structure based on a loss function;
the loss function is a formula
Wherein Y is 1 、Y 2 、Y 3 Representing the output of the first sublayer structure, the second sublayer structure and the second layer structure respectively, and representing the bit of the input key character by GPlacing a sample tag of a thermal image, T representing a video scene, N representing the number of image frames in a video sample of the scene to be identified, W representing the length of the image in the video sample of the scene to be identified, H representing the width of the image in the video sample of the scene to be identified,output Y characterizing the kth image sample 1 The value of the pixel point at the position where the x-coordinate is i and the y-coordinate is j,/>Output Y characterizing the kth image sample 2 The value of the pixel point at the position where the x coordinate is i and the y coordinate is j; />The value of the pixel point at the j position with the x coordinate being i and the y coordinate being j in the kth image sample in the input sample label G; />Is the value of the s-th dimension data in T; />Is Y 3 The values of the s-th dimension data, p and m are preset parameters.
2. The video scene recognition method according to claim 1, wherein a pixel point of the thermal image of the location of the key person where the key person is located is marked with 1.
3. A video recognition device, comprising:
the scene feature type recognition unit is used for inputting a target video of a scene to be recognized into a target neural network, wherein the target neural network comprises a first layer structure and a second layer structure, the first layer structure is used for determining a position thermodynamic image of a key person according to the target video, and the action of the key person in the target video represents the candidate recognition scene type of the target video; the second layer structure is used for determining a target recognition scene characteristic type from the candidate recognition scene types according to the target video and the position thermal image of the key person;
the video scene recognition unit is used for recognizing the video according to the target recognition scene characteristic type output by the target neural network;
the first layer structure comprises a first sub-layer structure and at least one second sub-layer structure;
the first sublayer structure is used for determining a position thermodynamic image of a key person according to the target video; the second sub-layer structure is used for optimizing the position thermodynamic image of the key person output by the first sub-layer structure according to the target video; or the second sub-layer structure is used for re-optimizing the optimized position thermal image of the key person output by the previous second sub-layer structure according to the target video;
the first layer structure and the second layer structure are obtained based on loss function training;
the loss function is a formula
Wherein Y is 1 、Y 2 、Y 3 Respectively representing the output of the first sublayer structure, the second sublayer structure and the second layer structure, representing the sample label of the position thermodynamic image of the input key character, representing the video scene by T, representing the image frame number in the video sample of the scene to be identified by N, representing the image length in the video sample of the scene to be identified by W, representing the image width in the video sample of the scene to be identified by H,output Y characterizing the kth image sample 1 The value of the pixel point at the position where the x-coordinate is i and the y-coordinate is j,/>Output Y characterizing the kth image sample 2 The value of the pixel point at the position where the x coordinate is i and the y coordinate is j; />The value of the pixel point at the j position with the x coordinate being i and the y coordinate being j in the kth image sample in the input sample label G; />Is the value of the s-th dimension data in T; />Is Y 3 The values of the s-th dimension data, p and m are preset parameters.
4. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the video scene recognition method according to any one of claims 1 to 2 when the program is executed by the processor.
5. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor performs the steps of the video scene recognition method according to any of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010096738.8A CN111291692B (en) | 2020-02-17 | 2020-02-17 | Video scene recognition method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010096738.8A CN111291692B (en) | 2020-02-17 | 2020-02-17 | Video scene recognition method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111291692A CN111291692A (en) | 2020-06-16 |
| CN111291692B true CN111291692B (en) | 2023-10-20 |
Family
ID=71023615
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010096738.8A Active CN111291692B (en) | 2020-02-17 | 2020-02-17 | Video scene recognition method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111291692B (en) |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845487A (en) * | 2016-12-30 | 2017-06-13 | 佳都新太科技股份有限公司 | A kind of licence plate recognition method end to end |
| CN107886069A (en) * | 2017-11-10 | 2018-04-06 | 东北大学 | A kind of multiple target human body 2D gesture real-time detection systems and detection method |
| CN108648224A (en) * | 2018-05-18 | 2018-10-12 | 杭州电子科技大学 | A method of the real-time scene layout identification based on artificial neural network and reconstruction |
| CN108710847A (en) * | 2018-05-15 | 2018-10-26 | 北京旷视科技有限公司 | Scene recognition method, device and electronic equipment |
| CN108830208A (en) * | 2018-06-08 | 2018-11-16 | Oppo广东移动通信有限公司 | Method for processing video frequency and device, electronic equipment, computer readable storage medium |
| CN109117703A (en) * | 2018-06-13 | 2019-01-01 | 中山大学中山眼科中心 | It is a kind of that cell category identification method is mixed based on fine granularity identification |
| CN109145840A (en) * | 2018-08-29 | 2019-01-04 | 北京字节跳动网络技术有限公司 | video scene classification method, device, equipment and storage medium |
| CN109271854A (en) * | 2018-08-07 | 2019-01-25 | 北京市商汤科技开发有限公司 | Based on method for processing video frequency and device, video equipment and storage medium |
| CN109508681A (en) * | 2018-11-20 | 2019-03-22 | 北京京东尚科信息技术有限公司 | The method and apparatus for generating human body critical point detection model |
| CN109598234A (en) * | 2018-12-04 | 2019-04-09 | 深圳美图创新科技有限公司 | Critical point detection method and apparatus |
| CN109740522A (en) * | 2018-12-29 | 2019-05-10 | 广东工业大学 | A kind of personnel's detection method, device, equipment and medium |
| CN110166826A (en) * | 2018-11-21 | 2019-08-23 | 腾讯科技（深圳）有限公司 | Scene recognition method, device, storage medium and the computer equipment of video |
| CN110348463A (en) * | 2019-07-16 | 2019-10-18 | 北京百度网讯科技有限公司 | The method and apparatus of vehicle for identification |
| CN110443969A (en) * | 2018-05-03 | 2019-11-12 | 中移(苏州)软件技术有限公司 | A kind of fire point detecting method, device, electronic equipment and storage medium |
| CN110766096A (en) * | 2019-10-31 | 2020-02-07 | 北京金山云网络技术有限公司 | Video classification method and device and electronic equipment |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10162308B2 (en) * | 2016-08-01 | 2018-12-25 | Integem Inc. | Methods and systems for photorealistic human holographic augmented reality communication with interactive control in real-time |
| US10798313B2 (en) * | 2017-08-22 | 2020-10-06 | Alarm.Com Incorporated | Preserving privacy in surveillance |
-
2020
- 2020-02-17 CN CN202010096738.8A patent/CN111291692B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845487A (en) * | 2016-12-30 | 2017-06-13 | 佳都新太科技股份有限公司 | A kind of licence plate recognition method end to end |
| CN107886069A (en) * | 2017-11-10 | 2018-04-06 | 东北大学 | A kind of multiple target human body 2D gesture real-time detection systems and detection method |
| CN110443969A (en) * | 2018-05-03 | 2019-11-12 | 中移(苏州)软件技术有限公司 | A kind of fire point detecting method, device, electronic equipment and storage medium |
| CN108710847A (en) * | 2018-05-15 | 2018-10-26 | 北京旷视科技有限公司 | Scene recognition method, device and electronic equipment |
| CN108648224A (en) * | 2018-05-18 | 2018-10-12 | 杭州电子科技大学 | A method of the real-time scene layout identification based on artificial neural network and reconstruction |
| CN108830208A (en) * | 2018-06-08 | 2018-11-16 | Oppo广东移动通信有限公司 | Method for processing video frequency and device, electronic equipment, computer readable storage medium |
| CN109117703A (en) * | 2018-06-13 | 2019-01-01 | 中山大学中山眼科中心 | It is a kind of that cell category identification method is mixed based on fine granularity identification |
| CN109271854A (en) * | 2018-08-07 | 2019-01-25 | 北京市商汤科技开发有限公司 | Based on method for processing video frequency and device, video equipment and storage medium |
| CN109145840A (en) * | 2018-08-29 | 2019-01-04 | 北京字节跳动网络技术有限公司 | video scene classification method, device, equipment and storage medium |
| CN109508681A (en) * | 2018-11-20 | 2019-03-22 | 北京京东尚科信息技术有限公司 | The method and apparatus for generating human body critical point detection model |
| CN110166826A (en) * | 2018-11-21 | 2019-08-23 | 腾讯科技（深圳）有限公司 | Scene recognition method, device, storage medium and the computer equipment of video |
| CN109598234A (en) * | 2018-12-04 | 2019-04-09 | 深圳美图创新科技有限公司 | Critical point detection method and apparatus |
| CN109740522A (en) * | 2018-12-29 | 2019-05-10 | 广东工业大学 | A kind of personnel's detection method, device, equipment and medium |
| CN110348463A (en) * | 2019-07-16 | 2019-10-18 | 北京百度网讯科技有限公司 | The method and apparatus of vehicle for identification |
| CN110766096A (en) * | 2019-10-31 | 2020-02-07 | 北京金山云网络技术有限公司 | Video classification method and device and electronic equipment |
Non-Patent Citations (6)
| Title |
|---|
| Hao Guo 等.Human attribute recognition by refining attention heat map.《ELSEVIER:Pattern Recognition Letters》.2017,38-45. * |
| Mohammad Ashraf Russo 等.Sports Classification in Sequential Frames Using CNN and RNN.《2018 International Conference on Information and Communication Technology Robotics (ICT-ROBOT)》.2018,全文. * |
| 周永生.基于多尺度CNN特征的人体行为识别算法研究.《中国优秀硕士学位论文全文数据库 信息科技辑》.2019,I138-3964. * |
| 林露.智能安防的感知和识别关键技术研究.《中国博士论文全文数据库 工程科技II辑》.2020,C038-89. * |
| 王雨廷."个体-群组"关联描述的层次LSTM行为识别.《中国优秀硕士学位论文全文数据库 信息科技辑》.2020,I138-1457. * |
| 站春儒.基于卷积神经网络的图像场景分类方法研究.《中国优秀硕士学位论文全文数据库 信息科技辑》.2019,I138-3185. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111291692A (en) | 2020-06-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110176027B (en) | Video target tracking method, device, equipment and storage medium | |
| Kreiss et al. | Pifpaf: Composite fields for human pose estimation | |
| CN112597941B (en) | Face recognition method and device and electronic equipment | |
| Tran et al. | Two-stream flow-guided convolutional attention networks for action recognition | |
| CN111127308B (en) | Mirror image feature rearrangement restoration method for single sample face recognition under partial shielding | |
| CN111626925B (en) | Method and device for generating counterwork patch | |
| CN114331829A (en) | Countermeasure sample generation method, device, equipment and readable storage medium | |
| WO2021218671A1 (en) | Target tracking method and device, and storage medium and computer program | |
| CN111144314B (en) | Method for detecting tampered face video | |
| CN111814744A (en) | Face detection method and device, electronic equipment and computer storage medium | |
| CN111401192B (en) | Model training method and related device based on artificial intelligence | |
| CN113822254B (en) | Model training method and related device | |
| CN112084952B (en) | Video point location tracking method based on self-supervision training | |
| CN110633004A (en) | Interaction method, device and system based on human body posture estimation | |
| CN112036260A (en) | Expression recognition method and system for multi-scale sub-block aggregation in natural environment | |
| CN111105443A (en) | Video group figure motion trajectory tracking method based on feature association | |
| WO2021103474A1 (en) | Image processing method and apparatus, storage medium and electronic apparatus | |
| CN111353325A (en) | Key point detection model training method and device | |
| CN114694261A (en) | Video three-dimensional human body posture estimation method and system based on multi-level supervision graph convolution | |
| Ruiz-Santaquiteria et al. | Improving handgun detection through a combination of visual features and body pose-based data | |
| CN113435264A (en) | Face recognition attack resisting method and device based on black box substitution model searching | |
| CN111274985B (en) | Video text recognition system, video text recognition device and electronic equipment | |
| CN111291692B (en) | Video scene recognition method and device, electronic equipment and storage medium | |
| CN112150464A (en) | Image detection method and device, electronic equipment and storage medium | |
| CN113544701B (en) | Method and device for detecting associated object, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |