WO2016062095A1 - 视频分类方法和装置 - Google Patents

视频分类方法和装置 Download PDF

Info

Publication number
WO2016062095A1
WO2016062095A1 PCT/CN2015/080871 CN2015080871W WO2016062095A1 WO 2016062095 A1 WO2016062095 A1 WO 2016062095A1 CN 2015080871 W CN2015080871 W CN 2015080871W WO 2016062095 A1 WO2016062095 A1 WO 2016062095A1
Authority
WO
WIPO (PCT)
Prior art keywords
neural network
classification model
weight matrix
network classification
layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2015/080871
Other languages
English (en)
French (fr)
Inventor
姜育刚
吴祖煊
薛向阳
顾子晨
柴振华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Fudan University
Original Assignee
Huawei Technologies Co Ltd
Fudan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd, Fudan University filed Critical Huawei Technologies Co Ltd
Publication of WO2016062095A1 publication Critical patent/WO2016062095A1/zh
Priority to US15/495,541 priority Critical patent/US20170228618A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7834—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using audio features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/73—Querying
    • G06F16/735—Filtering based on additional data, e.g. user or group profiles
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/75—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7847—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7847—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content
    • G06F16/786—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content using motion, e.g. object motion or camera motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items

Definitions

  • the embodiments of the present invention relate to computer technologies, and in particular, to a video classification method and apparatus.
  • Video classification refers to the processing and analysis of video using visual, auditory, and motion information of video, and determining and recognizing actions and events occurring in the video. Video classification applications are very broad, such as: intelligent monitoring, video data management, and so on.
  • the video is classified by the early fusion technology. Specifically, the kernels of different features or different features extracted from the video file are linearly combined and input into the classifier for analysis, thereby performing video on the video. classification.
  • the relationship between features and semantics is neglected, and therefore, the accuracy of video classification is not high.
  • Embodiments of the present invention provide a video classification method and apparatus to improve the accuracy of video classification.
  • a first aspect of the embodiments of the present invention provides a video classification method, including:
  • the video file to be classified is classified by using the feature combination of the neural network classification model and the video file to be classified.
  • the feature according to the video sample The relationship between the relationship and the semantics establishes a neural network classification model, including:
  • a classification model of the neural network is established according to the weight matrix of the neural network classification model fusion layer and the weight matrix of the neural network classification layer.
  • the weight of the fusion layer of the neural network classification model is obtained according to the relationship between the relationship between the features of the video samples and the semantics A matrix and a weight matrix of the classification layer of the neural network classification model, including:
  • the objective function is:
  • ⁇ represents the deviation between the predicted value and the true value of the video sample
  • ⁇ 1 represents a preset first weight coefficient
  • ⁇ 2 represents a preset second weight coefficient
  • W E represents the neural network classification model fusion layer a weight matrix
  • W L-1 represents a weight matrix of the classifier layer of the neural network classification model
  • 2,1 represents the 2,1 norm of W E
  • ⁇ represents a semi-positive symmetric matrix for characterizing the relationship between semantics
  • the weighting matrix of the neural network classification model fusion layer and the neural network classification model classification layer are obtained by optimizing the objective function Weight matrix, including:
  • the near-end gradient algorithm is used to optimize the objective function, and the weight matrix of the fusion layer of the neural network classification model and the weight matrix of the classification layer of the neural network classification model are obtained.
  • the using the near-end gradient algorithm to optimize the objective function includes:
  • a second aspect of the embodiments of the present invention provides a video classification apparatus, including:
  • a model building module configured to establish a neural network classification model according to a relationship between features of the video samples and a relationship between semantics
  • a feature extraction module configured to acquire a feature combination of the video file to be classified
  • a classification module configured to classify the video file to be classified by using the feature combination of the neural network classification model and the video file to be classified.
  • the model establishing module is specifically configured to acquire a weight matrix of a fusion layer of a neural network classification model according to a relationship between a feature of a video sample and a relationship between semantics
  • the neural network classifies a weight matrix of the classification layer of the model; and establishes a classification model of the neural network according to the weight matrix of the fusion layer of the neural network classification model and the weight matrix of the neural network classification layer.
  • the model building module is specifically configured to obtain a weight matrix of the neural network classification model fusion layer and the neural network by optimizing an objective function Weight matrix of the classification layer of the network classification model;
  • the objective function is:
  • ⁇ represents the deviation between the predicted value and the true value of the video sample
  • ⁇ 1 represents a preset first weight coefficient
  • ⁇ 2 represents a preset second weight coefficient
  • W E represents the neural network classification model fusion layer a weight matrix
  • W L-1 represents a weight matrix of the classifier layer of the neural network classification model
  • 2,1 represents the 2,1 norm of W E
  • ⁇ represents a semi-positive symmetric matrix for characterizing the relationship between semantics
  • the model establishing module is specifically configured to use an near-end gradient algorithm to optimize an objective function, and obtain a weight matrix of a neural network classification model fusion layer And a weight matrix of the classification layer of the neural network classification model.
  • the model establishing module is specifically configured to initialize a weight matrix of the neural network classification model fusion layer in the objective function
  • the neural network classifies a weight matrix of the classification layer of the model; obtains a deviation between the predicted value and the actual value of the output by inputting characteristics of the video sample; and adjusts a weight matrix of the fusion layer of the neural network classification model and the nerve according to the deviation Weighting moments of the classification layer of the network classification model Array until the deviation is less than a preset threshold.
  • the video classification method and apparatus provided by the embodiments of the present invention establish a neural network classification model according to the relationship between the characteristics of the video samples and the semantics; acquire the feature combination of the video files to be classified; use the neural network classification And combining the feature of the model and the video file to be classified, and classifying the video file to be classified. Since the neural network classification model is established based on the relationship between the features of the video samples and the semantics, the relationship between the features and the semantics are fully considered, and thus the accuracy of the video classification can be improved.
  • Embodiment 1 is a schematic flowchart of Embodiment 1 of a video classification method according to the present invention
  • Embodiment 2 is a schematic flowchart of Embodiment 2 of a video classification method according to the present invention
  • Embodiment 3 is a schematic structural diagram of Embodiment 1 of a video classification apparatus according to the present invention.
  • FIG. 4 is a schematic structural diagram of Embodiment 2 of a video classification apparatus according to the present invention.
  • the invention trains the neural network classification model by combining the relationship between the features of the video samples and the semantics, and obtains the optimal weight of each connection in the neural network classification model, thereby improving the accuracy of the video classification.
  • FIG. 1 is a schematic flowchart of Embodiment 1 of a video classification method according to the present invention. As shown in FIG. 1 , the method in this embodiment is as follows:
  • S101 Establish a neural network classification model according to the relationship between the characteristics of the video samples and the relationship between the semantics.
  • the neural network described in the embodiments of the present invention refers to an artificial neural network, which is a computational model simulating a biological nervous system, including multiple layers, each layer is a nonlinear change of the upper layer, and an artificial neural network.
  • an artificial neural network which is a computational model simulating a biological nervous system, including multiple layers, each layer is a nonlinear change of the upper layer, and an artificial neural network.
  • deep neural networks can obtain complex features from low to high levels compared with traditional neural networks.
  • the structure of deep neural networks is very similar to the multilayer perceptual structure of human cerebral cortex. Therefore, it has a certain biological theory foundation and is a hot spot of current research.
  • a neural network is a set of connected input/output units, each of which is called a neuron, where each connection is associated with a weight.
  • the prediction results can be output more accurately by adjusting the associated weights of each connection.
  • the video samples described in the embodiments of the present invention refer to video files used when training a neural network classification model.
  • the weight matrix of the fusion layer of the neural network classification model and the weight matrix of the classification layer of the neural network classification model are obtained according to the relationship between the characteristics of the video samples and the semantic relationship;
  • a classification model of the neural network is established according to the weight matrix of the neural network classification model fusion layer and the weight matrix of the neural network classification layer.
  • the weight matrix of the fusion layer of the neural network classification model and the weight matrix of the classification layer of the neural network classification model are obtained according to the relationship between the characteristics of the video samples and the semantic relationship, and the neural network is obtained by optimizing the objective function.
  • the weight matrix of the network classification model fusion layer and the weight matrix of the classification layer of the neural network classification model, wherein the objective function has a well-designed regularization constraint condition, so that the feature can be fully considered in the same neural network classification model The relationship between the relationship and the semantics, thereby improving the accuracy of the video classification.
  • ⁇ represents the deviation between the predicted value and the true value of the video sample
  • ⁇ 1 represents a preset first weight coefficient
  • ⁇ 2 represents a preset second weight coefficient
  • W E represents the neural network classification model fusion layer a weight matrix
  • W L-1 represents a weight matrix of the classifier layer of the neural network classification model
  • 2,1 represents the 2,1 norm of W E
  • ⁇ represents a semi-positive symmetric matrix for characterizing the relationship between semantics
  • the weight matrix of the neural network classification model is generally randomly initialized.
  • the forward-propagation algorithm continuously performs nonlinear mapping on the features (original input) of the video samples, thereby obtaining the predicted values of the video samples. There is often a certain deviation between the predicted value and the true value of the video sample.
  • ⁇ is used to measure the true value of all video samples on the entire data set and the empirical loss of the predicted value deviation through the network forward propagation.
  • the present invention improves the accuracy of video classification, and
  • W E representing the weights of the neural network classifiers fusion layer weight matrix
  • W L-1 represents the weights of the neural network classifiers classify layer weight matrix.
  • ⁇ is a semi-positive symmetric matrix used to characterize the relationship between semantics. It is initially initialized as a unit matrix, which is updated by the weight of the classifier layer during the training process of the neural network classification model. Relationship, each element of its off-diagonal measure is the relationship between different semantics.
  • the above objective function can optimize the objective function by using a Proximal Gradient Method (PGM) in the frame of backward propagation.
  • PGM Proximal Gradient Method
  • the near-end gradient algorithm is the most commonly used optimization algorithm for solving large-scale data. It can usually converge faster and solve optimization problems efficiently.
  • the weights of the connections in the neural network classification model are obtained.
  • the weight matrix of the neural network classification model fusion layer in the objective function and the weight matrix of the classification layer of the neural network classification model are initialized; and the deviation of the predicted value and the actual value of the output is obtained by inputting the characteristics of the video sample. And adjusting a weight matrix of the neural network classification model fusion layer and a weight matrix of the neural network classification model classification layer according to the deviation, until the deviation is less than a preset threshold.
  • a neural network classification model capable of accurately performing video classification can be trained.
  • S102 Acquire a feature combination of the video file to be classified.
  • the improved dense trajectory features are extracted as visual features.
  • the dense trajectory features include 30-dimensional trajectory features, 96-dimensional histogram of gradients, 108-dimensional histogram of optical flow and The motion binary histogram feature of the 192-dimensional motion. These four features are further converted into feature representations of 4000-dimensional bag-of-words. Audio characteristics such as Mel-Frequency Cepstral Coefficients (MFCC) and Scale Invariant Feature Transform (SIFT) based on Spectrogram are also extracted.
  • MFCC Mel-Frequency Cepstral Coefficients
  • SIFT Scale Invariant Feature Transform
  • S103 classify the video files to be classified by using a combination of a neural network classification model and a video file to be classified.
  • the feature combination of the video file to be classified is used as an input of the neural network classification model, and the classification of the video file to be classified is output through the neural network classification model.
  • the neural network classification model is used for video classification processing, which can be completed almost in real time and has high efficiency.
  • a neural network classification model is established according to a relationship between a relationship between features of a video sample and a semantic; a feature combination of a video file to be classified is acquired; and the neural network classification model and the to-be-classified The feature combination of the video files is used to classify the video files to be classified. Since the neural network classification model is established based on the relationship between the features of the video samples and the semantics, the relationship between the features and the semantics are fully considered, and thus the accuracy of the video classification can be improved.
  • the results of the video classification generated by the technical solution of the present invention can be applied to other video related technologies, such as video summary and video retrieval.
  • video summary the video can be divided into multiple segments, and then the video classification technology of the present invention is used to perform semantic analysis on the video to extract meaningful meanings.
  • the video clip is the result of a video summary.
  • the video classification technology in the present invention can be used to extract the semantic information of the video content, thereby searching the video.
  • FIG. 2 is a schematic flowchart of Embodiment 2 of a video classification method according to the present invention, as shown in FIG. 2:
  • the video classification processing can be completed almost in real time, the efficiency is high, and the accuracy of the video classification is high.
  • Embodiment 3 is a schematic structural diagram of Embodiment 1 of a video classification apparatus according to the present invention.
  • the apparatus of this embodiment includes a model creation module 301, a feature extraction module 302, and a classification module 303, wherein the model establishment module 301 is configured to The relationship between the relationship and the semantics establishes a neural network classification model;
  • the feature extraction module 302 is configured to acquire a feature combination of the video files to be classified
  • the classification module 303 is configured to classify the video files to be classified by using the feature combination of the neural network classification model and the video file to be classified.
  • the model establishing module 301 is specifically configured to acquire a weight matrix of a neural network classification model fusion layer and a classification layer of the neural network classification model according to a relationship between a feature of a video sample and a semantic relationship. a weight matrix; a classification model of the neural network is established according to the weight matrix of the fusion layer of the neural network classification model and the weight matrix of the neural network classification layer.
  • the model establishing module 301 is specifically configured to acquire a weight matrix of a neural network classification model fusion layer and a weight matrix of the neural network classification model classification layer by optimizing an objective function;
  • the objective function is:
  • ⁇ represents the deviation between the predicted value and the true value of the video sample
  • ⁇ 1 represents a preset first weight coefficient
  • ⁇ 2 represents a preset second weight coefficient
  • W E represents the neural network classification model fusion layer a weight matrix
  • W L-1 represents a weight matrix of the classifier layer of the neural network classification model
  • 2,1 represents the 2,1 norm of W E
  • ⁇ represents a semi-positive symmetric matrix for characterizing the relationship between semantics
  • the model building module 301 is specifically configured to optimize the objective function by using a near-end gradient algorithm, and obtain a weight matrix of the neural network classification model fusion layer and a weight matrix of the neural network classification model classification layer.
  • the model establishing module 301 is specifically configured to initialize a weight matrix of the neural network classification model fusion layer and a weight matrix of the neural network classification model classification layer in the objective function; a feature, obtaining a deviation between the predicted value and the actual value of the output; adjusting a weight matrix of the fusion layer of the neural network classification model and a weight matrix of the classification layer of the neural network classification model according to the deviation, until the deviation is less than a preset Threshold.
  • the apparatus of the embodiment shown in FIG. 3 establishes a neural network classification model according to the relationship between the characteristics of the video samples and the semantics by the model building module; the feature extraction module acquires the feature combination of the video files to be classified; the classification module adopts And combining the feature of the neural network classification model and the video file to be classified, and classifying the video files to be classified. Since the neural network classification model is established based on the relationship between the features of the video samples and the semantics, the relationship between the features and the semantics are fully considered, and thus the accuracy of the video classification can be improved.
  • the apparatus of this embodiment includes a memory 410 and a processor 420.
  • the memory 410 may include a random access memory, a flash memory, a read only memory, and a programmable only Read memory, non-volatile memory or registers, etc.
  • the processor 420 can be a Central Processing Unit (CPU).
  • the memory 410 is used to store executable instructions.
  • the processor 420 can execute executable instructions stored in the memory 410.
  • the processor 420 is configured to establish a neural network classification model according to a relationship between features and semantics of the features of the video samples; and acquire features of the video file to be classified. And combining the neural network classification model and the feature combination of the video files to be classified to classify the video files to be classified.
  • the processor 420 is configured to acquire, according to a relationship between the relationship between the features of the video samples and the semantics, a weight matrix of the neural network classification model fusion layer and the classification layer of the neural network classification model. Weight matrix; fusion layer according to the neural network classification model The weight matrix and the weight matrix of the neural network classification layer establish a classification model of the neural network.
  • the processor 420 is configured to obtain, by optimizing an objective function, a weight matrix of a neural network classification model fusion layer and a weight matrix of the neural network classification model classification layer;
  • the objective function is:
  • ⁇ represents the deviation between the predicted value and the true value of the video sample
  • ⁇ 1 represents a preset first weight coefficient
  • ⁇ 2 represents a preset second weight coefficient
  • W E represents the neural network classification model fusion layer a weight matrix
  • W L-1 represents a weight matrix of the classifier layer of the neural network classification model
  • 2,1 represents the 2,1 norm of W E
  • ⁇ represents a semi-positive symmetric matrix for characterizing the relationship between semantics
  • the processor 420 is configured to optimize a target function by using a near-end gradient algorithm, obtain a weight matrix of the neural network classification model fusion layer, and a weight matrix of the neural network classification model classification layer.
  • the processor 420 is configured to initialize a weight matrix of the neural network classification model fusion layer in the objective function and a weight matrix of the neural network classification model classification layer;
  • the aforementioned program can be stored in a computer readable storage medium.
  • the program when executed, performs the steps including the foregoing method embodiments; and the foregoing storage medium includes various media that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Library & Information Science (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Medical Informatics (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Algebra (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种视频分类方法和装置，通过根据视频样本的特征之间的关系和语义之间的关系建立神经网络分类模型（S101）；获取待分类的视频文件的特征组合（S102）；采用所述神经网络分类模型和所述待分类的视频文件的特征组合，对所述待分类的视频文件进行分类（S103）。由于神经网络分类模型是根据视频样本的特征之间的关系和语义之间的关系建立的，充分考虑了特征之间的关系和语义之间的关系，因此，可以提高视频分类的准确性。

Description

视频分类方法和装置
本申请要求于2014年10月24日提交中国专利局、申请号为201410580006.0、发明名称为“视频分类方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明实施例涉及计算机技术，尤其涉及一种视频分类方法和装置。
背景技术
视频分类是指利用视频的视觉信息、听觉信息以及动作信息对视频进行处理和分析，并判断和识别出视频中出现的动作和事件。视频分类应用非常广泛，例如：进行智能监控、视频数据管理等。
现有技术中，通过早期融合的技术进行视频分类，具体地，将从视频文件中提取出的不同特征或者不同特征的核矩阵线性组合起来，输入到分类器中进行分析，从而，对视频进行分类。然而，采用现有技术的方法，忽略了特征之间和语义之间的关系，因此，视频分类的准确性不高。
发明内容
本发明实施例提供一种视频分类方法和装置，以提高视频分类的准确性。
本发明实施例第一方面提供一种视频分类方法，包括：
根据视频样本的特征之间的关系和语义之间的关系建立神经网络分类模型；
获取待分类的视频文件的特征组合；
采用所述神经网络分类模型和所述待分类的视频文件的特征组合，对所述待分类的视频文件进行分类。
结合第一方面，在第一种可能的实现方式中，所述根据视频样本的特征 之间的关系和语义之间的关系建立神经网络分类模型，包括：
根据视频样本的特征之间的关系和语义之间的关系，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；
根据所述神经网络分类模型融合层的权重矩阵和所述神经网络分类层的权重矩阵建立神经网络的分类模型。
结合第一方面的第一种可能的实现方式，在第二种可能的实现方式中，所述根据视频样本的特征之间的关系和语义之间的关系，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵，包括：
通过优化目标函数，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；
所述目标函数为：
Figure PCTCN2015080871-appb-000001
s.t  Ω≥0  tr(Ω)＝1
其中，ζ表示视频样本的预测值和真实值之间的偏差，λ1表示预设的第一权重系数，λ2表示预设的第二权重系数，WE表示所述神经网络分类模型融合层的权重矩阵，WE的每一列对应一种特征，WL-1表示所述神经网络分类模型分类器层的权重矩阵，
Figure PCTCN2015080871-appb-000002
表示所述WL-1的转置，||WE||2,1表示WE的2,1范数，Ω表示一个半正定的对称矩阵，用于表征语义之间的关系，Ω初始值为单位矩阵。
结合第一方面的第二种可能的实现方式，在第三种可能的实现方式中，所述通过优化目标函数，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵，包括：
采用近端梯度算法优化目标函数，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵。
结合第一方面的第三种可能的实现方式，在第四种可能的实现方式中，所述采用近端梯度算法优化目标函数，包括：
初始化所述目标函数中的所述神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；
通过输入视频样本的特征，获取输出的预测值和实际值的偏差；
根据所述偏差调整所述神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵，直到所述偏差小于预设阈值。
本发明实施例第二方面提供一种视频分类装置，包括：
模型建立模块，用于根据视频样本的特征之间的关系和语义之间的关系建立神经网络分类模型；
特征提取模块，用于获取待分类的视频文件的特征组合；
分类模块，用于采用所述神经网络分类模型和所述待分类的视频文件的特征组合，对所述待分类的视频文件进行分类。
结合第二方面，在第一种可能的实现方式中，所述模型建立模块具体用于根据视频样本的特征之间的关系和语义之间的关系，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；根据所述神经网络分类模型融合层的权重矩阵和所述神经网络分类层的权重矩阵建立神经网络的分类模型。
结合第二方面的第一种可能的实现方式，在第二种可能的实现方式中，所述模型建立模块具体用于通过优化目标函数，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；
所述目标函数为：
Figure PCTCN2015080871-appb-000003
s.t  Ω≥0  tr(Ω)＝1
其中，ζ表示视频样本的预测值和真实值之间的偏差，λ1表示预设的第一权重系数，λ2表示预设的第二权重系数，WE表示所述神经网络分类模型融合层的权重矩阵，WE的每一列对应一种特征，WL-1表示所述神经网络分类模型分类器层的权重矩阵，
Figure PCTCN2015080871-appb-000004
表示所述WL-1的转置，||WE||2,1表示WE的2,1范数，Ω表示一个半正定的对称矩阵，用于表征语义之间的关系，Ω初始值为单位矩阵。
结合第二方面的第二种可能的实现方式，在第三种可能的实现方式中，所述模型建立模块具体用于采用近端梯度算法优化目标函数，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵。
结合第二方面的第三种可能的实现方式，在第四种可能的实现方式中，所述模型建立模块具体用于初始化所述目标函数中的所述神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；通过输入视频样本的特征，获取输出的预测值和实际值的偏差；根据所述偏差调整所述神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩 阵，直到所述偏差小于预设阈值。
本发明实施例提供的视频分类方法和装置，通过根据视频样本的特征之间的关系和语义之间的关系建立神经网络分类模型；获取待分类的视频文件的特征组合；采用所述神经网络分类模型和所述待分类的视频文件的特征组合，对所述待分类的视频文件进行分类。由于神经网络分类模型是根据视频样本的特征之间的关系和语义之间的关系建立的，充分考虑了特征之间的关系和语义之间的关系，因此，可以提高视频分类的准确性。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本发明视频分类方法实施例一的流程示意图；
图2为本发明视频分类方法实施例二的流程示意图；
图3为本发明视频分类装置实施例一的结构示意图；
图4为本发明视频分类装置实施例二的结构示意图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本发明通过结合视频样本的特征之间的关系和语义之间的关系训练神经网络分类模型，获取神经网络分类模型中各连接的最优的权重，从而，提高视频分类的准确性。
下面以具体地实施例对本发明的技术方案进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某 些实施例不再赘述。
图1为本发明视频分类方法实施例一的流程示意图，如图1所示，本实施例的方法如下：
S101：根据视频样本的特征之间的关系和语义之间的关系建立神经网络分类模型。
本发明实施例中所描述的神经网络是指人工神经网络，人工神经网络是一种模拟生物神经系统的计算模型，包括多层，每一层都是上一层的非线性变化，人工神经网络包括深度神经网络和传统的神经网络，深度神经网络和传统的神经网络相比可以得到从低到高不同层次的复杂特征表达，深度神经网络的结构和人体大脑皮层的多层感知结构非常类似，从而具有一定的生物理论基础，是目前研究的热点。
神经网络是一组连接的输入/输出单元，每一个输入/输出单元叫做神经元，其中，每个连接都与一个权重相关联。在神经网络的训练阶段，通过调整每个连接的相关的权重，能够较为准确的输出预测结果。
本发明实施例中所描述的视频样本是指用于训练神经网络分类模型时，所采用的视频文件。
本发明实施例借助深度神经网络的结构，根据视频样本的特征之间的关系和语义之间的关系，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；根据所述神经网络分类模型融合层的权重矩阵和所述神经网络分类层的权重矩阵建立神经网络的分类模型。
其中，根据视频样本的特征之间的关系和语义之间的关系，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵具体地，通过优化目标函数，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵，其中，目标函数带有精心设计的正则化约束条件，从而，能够在同一个神经网络分类模型中充分考虑特征之间的关系和语义之间的关系，从而，提高视频分类的准确性。
本发明实施例带有正则化约束条件的目标函数如下所示：
Figure PCTCN2015080871-appb-000005
s.t  Ω≥0  tr(Ω)＝1
其中，ζ表示视频样本的预测值和真实值之间的偏差，λ1表示预设的第一权重系数，λ2表示预设的第二权重系数，WE表示所述神经网络分类模型融合层的权重矩阵，WE的每一列对应一种特征，WL-1表示所述神经网络分类模型分类器层的权重矩阵，
Figure PCTCN2015080871-appb-000006
表示所述WL-1的转置，||WE||2,1表示WE的2,1范数，Ω表示一个半正定的对称矩阵，用于表征语义之间的关系，Ω初始值为单位矩阵。
通常情况下，神经网络分类模型的权重矩阵一般随机进行初始化，在训练阶段，通过前向传播算法不断的对视频样本的特征(原始输入)进行非线性的映射，从而得到视频样本的预测值，视频样本的预测值和真实值之间往往有一定的偏差，通过不断的调整融合层的权重矩阵和分类器层的权重矩阵，使得针对不同的视频样本，预测值和真实值之间的偏差最小，ζ即是用来衡量整个数据集上所有视频样本的真实值和通过网络前向传播得到的预测值偏差的经验损失。
本发明为了充分利用特征之间的关系和语义之间的关系，提高视频分类的准确性，目标函数中增加了||WE||2,1项和
Figure PCTCN2015080871-appb-000007
项，其中，WE表示所述神经网络分类模型融合层的权重矩阵，WE的每一列对应一种特征，WL-1表示所述神经网络分类模型分类器层的权重矩阵。
最小化不同范数的含义如下所示：
特征之间的关系(融合层权重)：
Figure PCTCN2015080871-appb-000008
语义之间的关系(分类器层权重)
Figure PCTCN2015080871-appb-000009
Figure PCTCN2015080871-appb-000010
||WE||2,1即先对矩阵的每一行求2范数得到一个向量，再对这个向量求1范 数。当最小化这一范数的时候，在极少行为非0的情况下对应的目标函数会最小，从而使得矩阵行稀疏，于是保留下来的非零行即为所有不同特征间所共享的一个具有相同的模式，可反映出特征之间的一致性。
Ω是一个半正定的对称矩阵用来刻画语义之间的关系，它最初初始化为一个单位矩阵，在神经网络分类模型的训练过程中利用分类器层的权重对其进行更新，从而得到语义间的关系，它的非对角线上的每一元素衡量的是不同语义之间的关系。
上述目标函数，可在后向传播的框架中采用基于近端梯度算法(Proximal Gradient Method，以下简称：PGM)优化目标函数。近端梯度算法是求解大规模数据时最为常用的优化算法，通常可以较快收敛，高效地求解优化问题。从而，获取神经网络分类模型中各连接的权重。通常是初始化所述目标函数中的所述神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；通过输入视频样本的特征，获取输出的预测值和实际值的偏差；根据所述偏差调整所述神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵，直到所述偏差小于预设阈值。
更具体求解算法的详细步骤如下：
1：随机初始化网络权重；
2：训练过程，重复下述步骤K次；
21)不同的特征首先通过多层非线性变换抽象到同一维度；
22)不同特征在神经网络分类模型中融合在一起；
23)融合后的特征进行分类，得到前向传播的误差，即实际值和预测值之间的偏差；
24)将误差从第L层往后传递，固定Ω，利用Ω的约束使用梯度下降更新分类器层的权重矩阵WL-1，从而在更新WL-1时考虑语义之间的关系；对融合层的权重矩阵WE，在2-1范数的约束下更新WE，从而利用特征之间的关系， 在WE更新后，利用更新后的WE，学习得到Ω。
结束。
通过S101的步骤，可以训练出能够准确进行视频分类的神经网络分类模型。
S102：获取待分类的视频文件的特征组合。
获取视频文件的特征组合的方式有多种，本发明对此不作限制。
通常，会获取待分类的视频文件的多种特征从而提高分类效果。一般提取改进的密集轨迹特征作为视觉特征，密集轨迹特征包括30维的轨迹特征，96维的梯度直方图(histogram of gradients)的特征，108维的光流直方图(histogram of optical flow)特征以及192维的运动的二元直方图(motion binary histogram)特征。这四种特征进一步被转换为了4000维的词袋(bag-of-words)的特征表达。还会提取梅尔倒谱系数(Mel-Frequency Cepstral Coefficients，以下简称：MFCC)以及基于频谱图(Spectrogram)的尺度不变特征(Scale Invariant Feature Transform，以下简称：SIFT)等音频特征。
S103：采用神经网络分类模型和待分类的视频文件的特征组合，对待分类的视频文件进行分类。
即，将待分类的视频文件的特征组合作为神经网络分类模型的输入，通过神经网络分类模型输出待分类的视频文件所属的分类。
采用神经网络分类模型进行视频分类处理，几乎可以实时完成，效率较高。
本实施例中，通过根据视频样本的特征之间的关系和语义之间的关系建立神经网络分类模型；获取待分类的视频文件的特征组合；采用所述神经网络分类模型和所述待分类的视频文件的特征组合，对所述待分类的视频文件进行分类。由于神经网络分类模型是根据视频样本的特征之间的关系和语义之间的关系建立的，充分考虑了特征之间的关系和语义之间的关系，因此，可以提高视频分类的准确性。
利用本发明的技术方案产生视频分类的结果可应用与其他视频相关技术之中，如视频摘要和视频检索等。在视频摘要中，可以将视频分成多个片段，之后利用本发明中的视频分类技术对视频进行语义分析，提取出具有意义的 视频片段作为视频摘要的结果。在视频检索中，可以利用本发明中的视频分类技术提取出视频内容的语义信息，从而对视频进行检索。
本发明还提供一种一个实施例，如图2所示，图2为本发明视频分类方法实施例二的流程示意图，如图2所示：
S201：从给定的视频文件中提取视觉特征和听觉特征；
S202：对提取的特征进行量化，得到特征对应的词袋模型；
S203：将每个词袋模型表征为对应的向量，对向量进行前向特征变换；
S204：对进行前向特征变换后的特征进行融合特征处理。
S205：输出视频分类结果。
采用本发明的方法，视频分类处理，几乎可以实时完成，效率较高，并且视频分类的准确性较高。
图3为本发明视频分类装置实施例一的结构示意图，本实施例的装置包括模型建立模块301、特征提取模块302和分类模块303，其中，模型建立模块301用于根据视频样本的特征之间的关系和语义之间的关系建立神经网络分类模型；
特征提取模块302用于获取待分类的视频文件的特征组合；
分类模块303用于采用所述神经网络分类模型和所述待分类的视频文件的特征组合，对所述待分类的视频文件进行分类。
在上述实施例中，所述模型建立模块301具体用于根据视频样本的特征之间的关系和语义之间的关系，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；根据所述神经网络分类模型融合层的权重矩阵和所述神经网络分类层的权重矩阵建立神经网络的分类模型。
在上述实施例中，所述模型建立模块301具体用于通过优化目标函数，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；
所述目标函数为：
Figure PCTCN2015080871-appb-000011
s.t  Ω≥0  tr(Ω)＝1
其中，ζ表示视频样本的预测值和真实值之间的偏差，λ1表示预设的第一权重系数，λ2表示预设的第二权重系数，WE表示所述神经网络分类模型融合层的权重矩阵，WE的每一列对应一种特征，WL-1表示所述神经网络分类模 型分类器层的权重矩阵，
Figure PCTCN2015080871-appb-000012
表示所述WL-1的转置，||WE||2,1表示WE的2,1范数，Ω表示一个半正定的对称矩阵，用于表征语义之间的关系，Ω初始值为单位矩阵。
在上述实施例中，所述模型建立模块301具体用于采用近端梯度算法优化目标函数，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵。
在上述实施例中，所述模型建立模块301具体用于初始化所述目标函数中的所述神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；通过输入视频样本的特征，获取输出的预测值和实际值的偏差；根据所述偏差调整所述神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵，直到所述偏差小于预设阈值。
图3的装置的其它功能和操作可以参照上面图1的方法实施例的过程，为了避免重复，此处不再赘述。
图3所示实施例的装置，通过模型建立模块根据视频样本的特征之间的关系和语义之间的关系建立神经网络分类模型；特征提取模块获取待分类的视频文件的特征组合；分类模块采用所述神经网络分类模型和所述待分类的视频文件的特征组合，对所述待分类的视频文件进行分类。由于神经网络分类模型是根据视频样本的特征之间的关系和语义之间的关系建立的，充分考虑了特征之间的关系和语义之间的关系，因此，可以提高视频分类的准确性。
图4为本发明视频分类装置实施例二的结构示意图，如图4所示，本实施例的装置包括存储器410和处理器420，存储器410可以包括随机存储器、闪存、只读存储器、可编程只读存储器、非易失性存储器或寄存器等。处理器420可以是中央处理器(Central Processing Unit，CPU)。存储器410用于存储可执行指令。处理器420可以执行存储器410中存储的可执行指令，例如，处理器420用于根据视频样本的特征之间的关系和语义之间的关系建立神经网络分类模型；获取待分类的视频文件的特征组合；采用所述神经网络分类模型和所述待分类的视频文件的特征组合，对所述待分类的视频文件进行分类。
可选地，作为一个实施例，处理器420可用于根据视频样本的特征之间的关系和语义之间的关系，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；根据所述神经网络分类模型融合层 的权重矩阵和所述神经网络分类层的权重矩阵建立神经网络的分类模型。
可选地，作为一个实施例，处理器420可用于通过优化目标函数，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；
所述目标函数为：
Figure PCTCN2015080871-appb-000013
s.t  Ω≥0  tr(Ω)＝1
其中，ζ表示视频样本的预测值和真实值之间的偏差，λ1表示预设的第一权重系数，λ2表示预设的第二权重系数，WE表示所述神经网络分类模型融合层的权重矩阵，WE的每一列对应一种特征，WL-1表示所述神经网络分类模型分类器层的权重矩阵，
Figure PCTCN2015080871-appb-000014
表示所述WL-1的转置，||WE||2,1表示WE的2,1范数，Ω表示一个半正定的对称矩阵，用于表征语义之间的关系，Ω初始值为单位矩阵。
可选地，作为一个实施例，处理器420可用于采用近端梯度算法优化目标函数，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵。
可选地，作为一个实施例，处理器420可用于初始化所述目标函数中的所述神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；
通过输入视频样本的特征，获取输出的预测值和实际值的偏差；
根据所述偏差调整所述神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵，直到所述偏差小于预设阈值。
图4的装置的其它功能和操作可以参照上面图1的方法实施例的过程，为了避免重复，此处不再赘述。
本领域普通技术人员可以理解：实现上述各方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成。前述的程序可以存储于一计算机可读取存储介质中。该程序在执行时，执行包括上述各方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术 方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。

Claims (10)

  1. 一种视频分类方法，其特征在于，包括：
    根据视频样本的特征之间的关系和语义之间的关系建立神经网络分类模型；
    获取待分类的视频文件的特征组合；
    采用所述神经网络分类模型和所述待分类的视频文件的特征组合，对所述待分类的视频文件进行分类。
  2. 根据权利要求1所述的方法，其特征在于，所述根据视频样本的特征之间的关系和语义之间的关系建立神经网络分类模型，包括：
    根据视频样本的特征之间的关系和语义之间的关系，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；
    根据所述神经网络分类模型融合层的权重矩阵和所述神经网络分类层的权重矩阵建立神经网络的分类模型。
  3. 根据权利要求2所述的方法，其特征在于，所述根据视频样本的特征之间的关系和语义之间的关系，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵，包括：
    通过优化目标函数，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；
    所述目标函数为：
    Figure PCTCN2015080871-appb-100001
    s.t Ω≥0 tr(Ω)＝1
    其中，ζ表示视频样本的预测值和真实值之间的偏差，λ1表示预设的第一权重系数，λ2表示预设的第二权重系数，WE表示所述神经网络分类模型融合层的权重矩阵，WE的每一列对应一种特征，WL-1表示所述神经网络分类模型分类器层的权重矩阵，
    Figure PCTCN2015080871-appb-100002
    表示所述WL-1的转置，||WE||2,1表示WE的2,1范数，Ω表示一个半正定的对称矩阵，用于表征语义之间的关系，Ω初始值为单位矩阵。
  4. 根据权利要求3所述的方法，其特征在于，所述通过优化目标函数，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵，包括：
    采用近端梯度算法优化目标函数，获取神经网络分类模型融合层的权重 矩阵和所述神经网络分类模型分类层的权重矩阵。
  5. 根据权利要求4所述的方法，其特征在于，所述采用近端梯度算法优化目标函数，包括：
    初始化所述目标函数中的所述神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；
    通过输入视频样本的特征，获取输出的预测值和实际值的偏差；
    根据所述偏差调整所述神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵，直到所述偏差小于预设阈值。
  6. 一种视频分类装置，其特征在于，包括：
    模型建立模块，用于根据视频样本的特征之间的关系和语义之间的关系建立神经网络分类模型；
    特征提取模块，用于获取待分类的视频文件的特征组合；
    分类模块，用于采用所述神经网络分类模型和所述待分类的视频文件的特征组合，对所述待分类的视频文件进行分类。
  7. 根据权利要求6所述的装置，其特征在于，所述模型建立模块具体用于根据视频样本的特征之间的关系和语义之间的关系，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；根据所述神经网络分类模型融合层的权重矩阵和所述神经网络分类层的权重矩阵建立神经网络的分类模型。
  8. 根据权利要求7所述的装置，其特征在于，所述模型建立模块具体用于通过优化目标函数，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；
    所述目标函数为：
    Figure PCTCN2015080871-appb-100003
    s.t Ω≥0 tr(Ω)＝1
    其中，ζ表示视频样本的预测值和真实值之间的偏差，λ1表示预设的第一权重系数，λ2表示预设的第二权重系数，WE表示所述神经网络分类模型融合层的权重矩阵，WE的每一列对应一种特征，WL-1表示所述神经网络分类模型分类器层的权重矩阵，
    Figure PCTCN2015080871-appb-100004
    表示所述WL-1的转置，||WE||2,1表示WE的2,1范数，Ω表示一个半正定的对称矩阵，用于表征语义之间的关系，Ω初始值为单位 矩阵。
  9. 根据权利要求8所述的装置，其特征在于，所述模型建立模块具体用于采用近端梯度算法优化目标函数，获取神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵。
  10. 根据权利要求9所述的装置，其特征在于，所述模型建立模块具体用于初始化所述目标函数中的所述神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵；通过输入视频样本的特征，获取输出的预测值和实际值的偏差；根据所述偏差调整所述神经网络分类模型融合层的权重矩阵和所述神经网络分类模型分类层的权重矩阵，直到所述偏差小于预设阈值。
PCT/CN2015/080871 2014-10-24 2015-06-05 视频分类方法和装置 Ceased WO2016062095A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/495,541 US20170228618A1 (en) 2014-10-24 2017-04-24 Video classification method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410580006.0A CN104331442A (zh) 2014-10-24 2014-10-24 视频分类方法和装置
CN201410580006.0 2014-10-24

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/495,541 Continuation US20170228618A1 (en) 2014-10-24 2017-04-24 Video classification method and apparatus

Publications (1)

Publication Number Publication Date
WO2016062095A1 true WO2016062095A1 (zh) 2016-04-28

Family

ID=52406169

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/080871 Ceased WO2016062095A1 (zh) 2014-10-24 2015-06-05 视频分类方法和装置

Country Status (3)

Country Link
US (1) US20170228618A1 (zh)
CN (1) CN104331442A (zh)
WO (1) WO2016062095A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107890348A (zh) * 2017-11-21 2018-04-10 郑州大学 一种基于深度学习法心电节拍特征自动化提取及分类方法
CN108304479A (zh) * 2017-12-29 2018-07-20 浙江工业大学 一种基于图结构过滤的快速密度聚类双层网络推荐方法
CN111033520A (zh) * 2017-08-21 2020-04-17 诺基亚技术有限公司 用于模式识别的方法、系统及装置
CN111401464A (zh) * 2020-03-25 2020-07-10 北京字节跳动网络技术有限公司 分类方法、装置、电子设备及计算机可读存储介质
CN112966646A (zh) * 2018-05-10 2021-06-15 北京影谱科技股份有限公司 基于双路模型融合的视频分割方法、装置、设备及介质
CN111259919B (zh) * 2018-11-30 2024-01-23 杭州海康威视数字技术股份有限公司 一种视频分类方法、装置及设备、存储介质

Families Citing this family (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104331442A (zh) * 2014-10-24 2015-02-04 华为技术有限公司 视频分类方法和装置
CN104966104B (zh) * 2015-06-30 2018-05-11 山东管理学院 一种基于三维卷积神经网络的视频分类方法
CN106503723A (zh) * 2015-09-06 2017-03-15 华为技术有限公司 一种视频分类方法及装置
CN108319888B (zh) * 2017-01-17 2023-04-07 阿里巴巴集团控股有限公司 视频类型的识别方法及装置、计算机终端
CN212708031U (zh) 2017-03-15 2021-03-16 卡本有限公司 集成增材制造系统
US11037330B2 (en) * 2017-04-08 2021-06-15 Intel Corporation Low rank matrix compression
CN107491782B (zh) * 2017-07-22 2020-11-20 复旦大学 利用语义空间信息的针对少量训练数据的图像分类方法
CN109508584B (zh) * 2017-09-15 2022-12-02 腾讯科技（深圳）有限公司 视频分类的方法、信息处理的方法以及服务器
CN107911755B (zh) * 2017-11-10 2020-10-20 天津大学 一种基于稀疏自编码器的多视频摘要方法
CN108763325B (zh) * 2018-05-04 2019-10-01 北京达佳互联信息技术有限公司 一种网络对象处理方法及装置
US10805029B2 (en) * 2018-09-11 2020-10-13 Nbcuniversal Media, Llc Real-time automated classification system
CN109124635B (zh) * 2018-09-25 2022-09-02 上海联影医疗科技股份有限公司 模型生成方法、磁共振成像扫描方法及系统
CN109522450B (zh) 2018-11-29 2023-04-07 腾讯科技（深圳）有限公司 一种视频分类的方法以及服务器
CN110070067B (zh) * 2019-04-29 2021-11-12 北京金山云网络技术有限公司 视频分类方法及其模型的训练方法、装置和电子设备
CN110135386B (zh) * 2019-05-24 2021-09-03 长沙学院 一种基于深度学习的人体动作识别方法和系统
CN110188668B (zh) * 2019-05-28 2020-09-25 复旦大学 一种面向小样本视频动作分类的方法
CN110263217A (zh) * 2019-06-28 2019-09-20 北京奇艺世纪科技有限公司 一种视频片段标签识别方法及装置
CN110598733A (zh) * 2019-08-05 2019-12-20 南京智谷人工智能研究院有限公司 一种基于交互建模的多标记距离度量学习方法
CN110503076B (zh) * 2019-08-29 2023-06-30 腾讯科技（深圳）有限公司 基于人工智能的视频分类方法、装置、设备和介质
CN110740343B (zh) * 2019-09-11 2022-08-26 深圳壹账通智能科技有限公司 基于视频类型的播放控制实现方法、装置及计算机设备
EP3997625A4 (en) * 2019-10-29 2022-11-09 Samsung Electronics Co., Ltd. ELECTRONIC DEVICE AND CONTROL METHOD THEREOF
CN111339362B (zh) * 2020-02-05 2023-07-18 天津大学 一种基于深度协同矩阵分解的短视频多标签分类方法
CN111737521B (zh) * 2020-08-04 2020-11-24 北京微播易科技股份有限公司 一种视频分类方法和装置
KR102504321B1 (ko) * 2020-08-25 2023-02-28 한국전자통신연구원 온라인 행동 탐지 장치 및 방법
CN112633263B (zh) * 2021-03-09 2021-06-08 中国科学院自动化研究所 海量音视频情感识别系统
CN115714915B (zh) 2021-08-12 2026-04-03 蒂普爱可斯有限公司 基于人工智能的图像稳定化方法及其相机模块
CN114398983B (zh) * 2022-01-14 2024-11-05 腾讯科技（深圳）有限公司 分类预测方法、装置、设备、存储介质及计算机程序产品
CN114969439B (zh) * 2022-06-27 2024-08-30 北京爱奇艺科技有限公司 一种模型训练、信息检索方法及装置
US12316647B1 (en) 2024-01-31 2025-05-27 Netskope, Inc. Video data loss prevention (vDLP)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101593273A (zh) * 2009-08-13 2009-12-02 北京邮电大学 一种基于模糊综合评价的视频情感内容识别方法
CN102436583A (zh) * 2011-09-26 2012-05-02 哈尔滨工程大学 基于对标注图像学习的图像分割方法
CN102930302A (zh) * 2012-10-18 2013-02-13 山东大学 基于在线序贯极限学习机的递增式人体行为识别方法
US20130138436A1 (en) * 2011-11-26 2013-05-30 Microsoft Corporation Discriminative pretraining of deep neural networks
CN104331442A (zh) * 2014-10-24 2015-02-04 华为技术有限公司 视频分类方法和装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8165407B1 (en) * 2006-10-06 2012-04-24 Hrl Laboratories, Llc Visual attention and object recognition system
CN101866339A (zh) * 2009-04-16 2010-10-20 周矛锐 基于图像的多内容信息在互联网上的识别,及对已识别内容信息中的商品引导购买的应用
CN101894125B (zh) * 2010-05-13 2012-05-09 复旦大学 一种基于内容的视频分类方法
CN101902617B (zh) * 2010-06-11 2011-12-07 公安部第三研究所 一种用dsp和fpga实现视频结构化描述的装置及方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101593273A (zh) * 2009-08-13 2009-12-02 北京邮电大学 一种基于模糊综合评价的视频情感内容识别方法
CN102436583A (zh) * 2011-09-26 2012-05-02 哈尔滨工程大学 基于对标注图像学习的图像分割方法
US20130138436A1 (en) * 2011-11-26 2013-05-30 Microsoft Corporation Discriminative pretraining of deep neural networks
CN102930302A (zh) * 2012-10-18 2013-02-13 山东大学 基于在线序贯极限学习机的递增式人体行为识别方法
CN104331442A (zh) * 2014-10-24 2015-02-04 华为技术有限公司 视频分类方法和装置

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111033520A (zh) * 2017-08-21 2020-04-17 诺基亚技术有限公司 用于模式识别的方法、系统及装置
CN111033520B (zh) * 2017-08-21 2024-03-19 诺基亚技术有限公司 用于模式识别的方法、系统及装置
CN107890348A (zh) * 2017-11-21 2018-04-10 郑州大学 一种基于深度学习法心电节拍特征自动化提取及分类方法
CN108304479A (zh) * 2017-12-29 2018-07-20 浙江工业大学 一种基于图结构过滤的快速密度聚类双层网络推荐方法
CN108304479B (zh) * 2017-12-29 2022-05-03 浙江工业大学 一种基于图结构过滤的快速密度聚类双层网络推荐方法
CN112966646A (zh) * 2018-05-10 2021-06-15 北京影谱科技股份有限公司 基于双路模型融合的视频分割方法、装置、设备及介质
CN112966646B (zh) * 2018-05-10 2024-01-09 北京影谱科技股份有限公司 基于双路模型融合的视频分割方法、装置、设备及介质
CN111259919B (zh) * 2018-11-30 2024-01-23 杭州海康威视数字技术股份有限公司 一种视频分类方法、装置及设备、存储介质
CN111401464A (zh) * 2020-03-25 2020-07-10 北京字节跳动网络技术有限公司 分类方法、装置、电子设备及计算机可读存储介质

Also Published As

Publication number Publication date
CN104331442A (zh) 2015-02-04
US20170228618A1 (en) 2017-08-10

Similar Documents

Publication Publication Date Title
WO2016062095A1 (zh) 视频分类方法和装置
CN113326852B (zh) 模型训练方法、装置、设备、存储介质及程序产品
CN108288067B (zh) 图像文本匹配模型的训练方法、双向搜索方法及相关装置
KR102570278B1 (ko) 교사 모델로부터 학생 모델을 트레이닝하는데 사용되는 학습 데이터를 생성하는 장치 및 방법
JP6755849B2 (ja) 人工ニューラルネットワークのクラスに基づく枝刈り
CN107506761B (zh) 基于显著性学习卷积神经网络的脑部图像分割方法及系统
CN106919951B (zh) 一种基于点击与视觉融合的弱监督双线性深度学习方法
CN107633522B (zh) 基于局部相似性活动轮廓模型的脑部图像分割方法和系统
US10691971B2 (en) Method and apparatus for recognizing object
CN104573669B (zh) 图像物体检测方法
CN112529005A (zh) 基于语义特征一致性监督金字塔网络的目标检测方法
Ma et al. Lightweight attention convolutional neural network through network slimming for robust facial expression recognition
CN112784778B (zh) 生成模型并识别年龄和性别的方法、装置、设备和介质
CN112990427A (zh) 域自适应的神经网络实现的装置和方法
WO2016062044A1 (zh) 一种模型参数训练方法、装置及系统
US20230316733A1 (en) Video behavior recognition method and apparatus, and computer device and storage medium
WO2022228425A1 (zh) 一种模型训练方法及装置
CN110622175A (zh) 神经网络分类
CN108280058A (zh) 基于强化学习的关系抽取方法和装置
CN112529149B (zh) 一种数据处理方法及相关装置
US12039449B2 (en) Method and apparatus with neural network meta-training and class vector training
CN112749737A (zh) 图像分类方法及装置、电子设备、存储介质
CN113704528B (zh) 聚类中心确定方法、装置和设备及计算机存储介质
CN114008666B (zh) 动态图像分辨率评估
CN111401521A (zh) 神经网络模型训练方法及装置、图像识别方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15853077

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15853077

Country of ref document: EP

Kind code of ref document: A1