WO2023016290A1 - 视频分类方法、装置、可读介质和电子设备 - Google Patents

视频分类方法、装置、可读介质和电子设备 Download PDF

Info

Publication number
WO2023016290A1
WO2023016290A1 PCT/CN2022/109470 CN2022109470W WO2023016290A1 WO 2023016290 A1 WO2023016290 A1 WO 2023016290A1 CN 2022109470 W CN2022109470 W CN 2022109470W WO 2023016290 A1 WO2023016290 A1 WO 2023016290A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
transformation
target
transformed
videos
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/109470
Other languages
English (en)
French (fr)
Inventor
佘琪
沈铮阳
王长虎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Youzhuju Network Technology Co Ltd
Original Assignee
Beijing Youzhuju Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Youzhuju Network Technology Co Ltd filed Critical Beijing Youzhuju Network Technology Co Ltd
Publication of WO2023016290A1 publication Critical patent/WO2023016290A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the present disclosure relates to the technical field of video processing, and in particular, to a video classification method, device, readable medium and electronic equipment.
  • video classification usually uses an end-to-end CNN (English: Convolutional Neural Networks, Chinese: Convolutional Neural Networks) model to learn the implicit spatiotemporal relationship in the video for video classification.
  • CNN International: Convolutional Neural Networks
  • Chinese Convolutional Neural Networks
  • the present disclosure provides a video classification method, the method comprising:
  • the video to be classified is transformed to obtain multiple transformed videos
  • the video classification model is used to determine the target video feature corresponding to the converted video according to the converted video, and determine the video classification result according to the target video feature; the target video feature is the converted video has transformation invariance properties.
  • the present disclosure provides a video classification device, the device comprising:
  • the transformation module is used to transform the video to be classified by the target transformation group to obtain a plurality of transformed videos
  • a determining module configured to determine the video classification result of the video to be classified through the trained video classification model according to a plurality of transformed videos
  • the video classification model is used to determine the target video feature corresponding to the converted video according to the converted video, and determine the video classification result according to the target video feature; the target video feature is the converted video has transformation invariance properties.
  • the present disclosure provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processing device, the steps of the method described in the first aspect of the present disclosure are implemented.
  • an electronic device including:
  • a processing device configured to execute the computer program in the storage device to implement the steps of the method described in the first aspect of the present disclosure.
  • Fig. 1 is a flow chart of a video classification method shown according to an exemplary embodiment
  • Fig. 2 is a flow chart showing a step 101 according to the embodiment shown in Fig. 1;
  • Fig. 3 is a flow chart showing a step 102 according to the embodiment shown in Fig. 1;
  • Fig. 4 is a flow chart showing a training video classification model according to an exemplary embodiment
  • Fig. 5 is a block diagram of a video classification device according to an exemplary embodiment
  • Fig. 6 is a block diagram of a transformation module shown according to the embodiment shown in Fig. 5;
  • Fig. 7 is a block diagram of a determination module according to the embodiment shown in Fig. 5;
  • Fig. 8 is a block diagram of an electronic device according to an exemplary embodiment.
  • the term “comprise” and its variations are open-ended, ie “including but not limited to”.
  • the term “based on” is “based at least in part on”.
  • the term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one further embodiment”; the term “some embodiments” means “at least some embodiments.” Relevant definitions of other terms will be given in the description below.
  • Fig. 1 is a flowchart of a video classification method according to an exemplary embodiment. As shown in Figure 1, the method may include the following steps:
  • Step 101 Transform the video to be classified through the target transformation group to obtain multiple transformed videos.
  • the preset transformation group corresponding to each specified type of transformation is generated.
  • the preset transformation group corresponding to each specified type of transformation is a group formed by multiple transformations of the specified type.
  • the preset transformation group when the specified type of transformation is a rotation transformation, can be a rotation group composed of multiple rotation transformations; when the specified type of transformation is a scaling transformation, the preset transformation group can be multiple A scaling group composed of multiple scaling transformations.
  • the preset transformation group when the specified type of transformation is an affine transformation, can be an affine transformation group composed of multiple affine transformations.
  • the video to be classified may be acquired, and a target transformation group is determined from a plurality of pre-generated preset transformation groups according to the video to be classified.
  • a target transformation group is determined from a plurality of pre-generated preset transformation groups according to the video to be classified.
  • One possible way is to select the target transformation group from a plurality of preset transformation groups according to the characteristics of the video to be classified. For example, if the object in the video to be classified (the object For example, it may be that the size of a person or an object is relatively large, then a scaling group may be selected as the target transformation group.
  • the video to be classified can be transformed respectively through multiple transformations included in the target transformation group to obtain the transformed video corresponding to each transformation.
  • Step 102 Determine the video classification result of the video to be classified through the trained video classification model according to the plurality of transformed videos.
  • the video classification model is used to determine the target video feature corresponding to the transformed video according to the transformed video, and determine the video classification result according to the target video feature, and the target video feature is a transform invariant feature in the transformed video.
  • the multiple transformed videos may be input into a trained video classification model.
  • the video features of each transformed video are extracted by the video classification model, and the maximum pooling process is performed on the video features of all the extracted transformed videos to obtain the target video features with transformation invariance in the transformed videos.
  • the target transformation group is a rotation group
  • multiple transformed videos are actually obtained after performing different rotation transformations on the videos to be classified.
  • the obtained target video features are actually invariant to rotation transformations in the videos to be classified. Characteristics.
  • the video classification model can determine a video classification result of the video to be classified from a plurality of preset video types according to the characteristics of the target video.
  • the multiple preset video types may include normal videos and multiple abnormal video types.
  • the present disclosure actually takes the video to be classified as a whole, and obtains the video classification result of the video to be classified through the target transformation group and the video classification model.
  • the classification result is to determine the video classification result of the video to be classified.
  • the video classification method of the present disclosure may not only be applied to classify videos, but also may be applied to classify images, which is not specifically limited in the present disclosure.
  • the present disclosure first transforms the video to be classified through the target transformation group to obtain multiple transformed videos, and then determines the video classification result of the video to be classified according to the multiple transformed videos through the trained video classification model, wherein , the video classification model is used to determine the target video features corresponding to the transformed video according to the transformed video, and determine the video classification result according to the target video features.
  • the target video features are features with transformation invariance in the transformed video.
  • the disclosure uses a video classification model to extract transformation-invariant target video features from multiple transformed videos, and performs video classification through the target video features, which can avoid the influence of rotation, scaling or affine transformation on video classification, and improves the efficiency of video classification. Accuracy of video classification.
  • Fig. 2 is a flow chart showing a step 101 according to the embodiment shown in Fig. 1 .
  • step 101 may include the following steps:
  • Step 1011 determine a target transformation group from a plurality of preset transformation groups.
  • the plurality of preset transformation groups include rotation group, scaling group and affine transformation group, and each preset transformation group includes a plurality of transformation matrices.
  • the object information corresponding to the target object in the video to be classified may be determined first by using a preset recognition algorithm. Then, the target transformation group can be determined from multiple preset transformation groups according to the object information.
  • the object information may include the position, direction and size of the target object, and each preset transformation group may include multiple transformations, and each transformation corresponds to a transformation matrix.
  • a standard image can be preset. If the direction of the target object in the video to be classified is quite different from the direction of the object in the standard image, the rotation group can be selected as the target transformation group. If the direction of the target object in the video to be classified is If the size of the target object in the target object differs greatly from the size of the object in the standard image, the scaling group can be selected as the target transformation group.
  • Step 1012 Transform the video to be classified by using each target transformation matrix in the target transformation group to obtain the transformed video corresponding to each target transformation matrix.
  • the target transformation group is the rotation group, and the rotation group includes 4 rotation transformations as an example. If the 4 rotation transformations respectively rotate the video to be classified by 45°, 90°, 135° and 180° clockwise, Then, after the target transformation group is determined, the video to be classified can be transformed through the target transformation matrix corresponding to each rotation transformation in the target transformation group (the transformation at this time is a rotation transformation), and the video to be classified can be rotated clockwise by 45 4 transformation videos of °, 90°, 135° and 180°.
  • Fig. 3 is a flow chart showing a step 102 according to the embodiment shown in Fig. 1 .
  • the video classification model includes a Siamese network, a maximum pooling layer, and a classifier.
  • the Siamese network includes multiple neural sub-networks, and the transformed video is in one-to-one correspondence with the neural sub-networks.
  • Step 102 may include the following steps:
  • Step 1021 input each transformed video into the neural sub-network corresponding to the transformed video, so as to extract video features of the transformed video.
  • Step 1022 Perform maximum pooling processing on the video features of each transformed video through the maximum pooling layer to obtain target video features.
  • Step 1023 using the classifier to determine the video classification result according to the characteristics of the target video.
  • the video classification model can be constructed through Siamese network, maximum pooling layer and classifier.
  • the twin network includes multiple neural sub-networks, and these neural sub-networks share network weights and network parameters.
  • the Siamese network can use 3D-CNN or two-stream CNN, and the classifier can use a linear classifier.
  • each transformed video may be input into a neural sub-network corresponding to the transformed video to obtain n-dimensional video features of the transformed video. Then, the video features of each transformed video can be input into the maximum pooling layer, and the maximum pooling layer performs element-by-element maximum pooling operation on the video features of the transformed video obtained by each neural sub-network, and outputs the target video features. Afterwards, the target video feature can be input into the classifier, and the classifier determines the video classification result according to the target video feature.
  • Fig. 4 is a flowchart showing a training video classification model according to an exemplary embodiment. As shown in Figure 4, the video classification model is obtained by:
  • Step 201 acquire a training sample set.
  • the training sample set includes training videos and training video classification results corresponding to the training videos.
  • Step 202 Transform the training video through each preset transformation group to obtain a plurality of training transformed videos corresponding to each preset transformation group.
  • Step 203 according to multiple training transformation videos and training video classification results, train the preset model to obtain a video classification model.
  • videos may be collected from business lines, and the collected videos may be divided into a training sample set and a test sample set.
  • the training sample set includes training videos and training video classification results corresponding to the training videos
  • the testing sample set includes testing videos and testing video classification results corresponding to the testing videos.
  • the training video can be transformed through each preset transformation group to obtain multiple training transformation videos corresponding to each preset transformation group.
  • the preset model can include a Siamese network, a maximum pooling layer, and a classifier.
  • the Siamese network can include multiple neural sub-networks.
  • the test sample set can be used to perform a performance test on the obtained video classification model (for example, the performance of the video classification model can be judged by the accuracy of the video classification result output by the video classification model), if the performance of the video classification model cannot reach requirements, retrain the video classification model until the performance of the video classification model meets the requirements.
  • the present disclosure first transforms the video to be classified through the target transformation group to obtain multiple transformed videos, and then determines the video classification result of the video to be classified according to the multiple transformed videos through the trained video classification model, wherein , the video classification model is used to determine the target video features corresponding to the transformed video according to the transformed video, and determine the video classification result according to the target video features.
  • the target video features are features with transformation invariance in the transformed video.
  • the disclosure uses a video classification model to extract transformation-invariant target video features from multiple transformed videos, and performs video classification through the target video features, which can avoid the influence of rotation, scaling or affine transformation on video classification, and improves the efficiency of video classification. Accuracy of video classification.
  • Fig. 5 is a block diagram of a device for classifying videos according to an exemplary embodiment. As shown in Figure 5, the device 300 includes:
  • the transformation module 301 is configured to transform the video to be classified through the target transformation group to obtain multiple transformed videos.
  • the determination module 302 is configured to determine the video classification result of the video to be classified according to the multiple converted videos and through the trained video classification model.
  • the video classification model is used to determine the target video feature corresponding to the transformed video according to the transformed video, and determine the video classification result according to the target video feature, and the target video feature is a transform invariant feature in the transformed video.
  • Fig. 6 is a block diagram of a transformation module according to the embodiment shown in Fig. 5 .
  • the transformation module 301 includes:
  • the determination sub-module 3011 is used to determine a target transformation group from a plurality of preset transformation groups, the plurality of preset transformation groups include a rotation group, a scaling group and an affine transformation group, and each preset transformation group includes a plurality of transformations matrix.
  • the transformation sub-module 3012 is configured to respectively transform the video to be classified through each target transformation matrix in the target transformation group to obtain the transformed video corresponding to each target transformation matrix.
  • the determining submodule 3011 is used for:
  • Object information corresponding to the target object in the video to be classified is determined, and the object information includes the position, direction and size of the target object.
  • a target transformation group is determined from a plurality of preset transformation groups.
  • Fig. 7 is a block diagram of a determining module according to the embodiment shown in Fig. 5 .
  • the video classification model includes a twin network, a maximum pooling layer and a classifier
  • the twin network includes a plurality of neural sub-networks
  • the transformed video is in one-to-one correspondence with the neural sub-networks
  • the determination module 302 includes:
  • the feature extraction sub-module 3021 is configured to input each transformed video into a neural sub-network corresponding to the transformed video, so as to extract video features of the transformed video.
  • the pooling sub-module 3022 is configured to perform maximum pooling processing on the video features of each converted video through a maximum pooling layer to obtain target video features.
  • the classification sub-module 3023 is configured to use a classifier to determine a video classification result according to the characteristics of the target video.
  • the determination module 302 is used to train the video classification model in the following manner:
  • the training sample set includes training videos, and training video classification results corresponding to the training videos.
  • the training video is transformed through each preset transformation group to obtain a plurality of training transformation videos corresponding to each preset transformation group.
  • the preset model is trained to obtain a video classification model.
  • the present disclosure first transforms the video to be classified through the target transformation group to obtain multiple transformed videos, and then determines the video classification result of the video to be classified according to the multiple transformed videos through the trained video classification model, wherein , the video classification model is used to determine the target video features corresponding to the transformed video according to the transformed video, and determine the video classification result according to the target video features.
  • the target video features are features with transformation invariance in the transformed video.
  • the disclosure uses a video classification model to extract transformation-invariant target video features from multiple transformed videos, and performs video classification through the target video features, which can avoid the influence of rotation, scaling or affine transformation on video classification, and improves the efficiency of video classification. Accuracy of video classification.
  • FIG. 8 it shows a schematic structural diagram of an electronic device (such as the terminal device or server in FIG. 1 ) 400 suitable for implementing the embodiments of the present disclosure.
  • the terminal equipment in the embodiment of the present disclosure may include but not limited to such as mobile phone, notebook computer, digital broadcast receiver, PDA (personal digital assistant), PAD (tablet computer), PMP (portable multimedia player), vehicle terminal (such as mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers and the like.
  • the electronic device shown in FIG. 8 is only an example, and should not limit the functions and scope of use of the embodiments of the present disclosure.
  • an electronic device 400 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) Various appropriate actions and processes are executed by programs in the memory (RAM) 403 .
  • RAM random access memory
  • various programs and data necessary for the operation of the electronic device 400 are also stored.
  • the processing device 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404.
  • An input/output (I/O) interface 405 is also connected to bus 404 .
  • the following devices can be connected to the I/O interface 405: input devices 406 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speaker, vibration an output device 407 such as a computer; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409.
  • the communication means 409 may allow the electronic device 400 to perform wireless or wired communication with other devices to exchange data. While FIG. 8 shows electronic device 400 having various means, it should be understood that implementing or having all of the means shown is not a requirement. More or fewer means may alternatively be implemented or provided.
  • embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer readable medium, where the computer program includes program code for executing the method shown in the flowchart.
  • the computer program may be downloaded and installed from a network via communication means 409, or from storage means 408, or from ROM 402.
  • the processing device 401 When the computer program is executed by the processing device 401, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed.
  • the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • a computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device .
  • Program code embodied on a computer readable medium may be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
  • the client and the server can communicate using any currently known or future network protocols such as HTTP (HyperText Transfer Protocol, Hypertext Transfer Protocol), and can communicate with digital data in any form or medium
  • HTTP HyperText Transfer Protocol
  • the communication eg, communication network
  • Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network of.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.
  • the above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: transforms the video to be classified through the target transformation group to obtain multiple transformed videos; A plurality of the converted videos, through the trained video classification model, determine the video classification result of the video to be classified; wherein, the video classification model is used to determine the target video corresponding to the converted video according to the converted video feature, and determine the video classification result according to the target video feature; the target video feature is a feature with transformation invariance in the transformed video.
  • Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, or combinations thereof, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Includes conventional procedural programming languages - such as "C" or similar programming languages.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer may be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, using an Internet service provider to connected via the Internet).
  • LAN local area network
  • WAN wide area network
  • Internet service provider for example, using an Internet service provider to connected via the Internet.
  • each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.
  • the modules involved in the embodiments described in the present disclosure may be implemented by software or by hardware. Wherein, the name of the module does not constitute a limitation of the module itself under certain circumstances.
  • the transformation module can also be described as "a module for transforming the video to be classified to obtain multiple transformed videos".
  • FPGAs Field Programmable Gate Arrays
  • ASICs Application Specific Integrated Circuits
  • ASSPs Application Specific Standard Products
  • SOCs System on Chips
  • CPLD Complex Programmable Logical device
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing.
  • machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read only memory
  • EPROM or flash memory erasable programmable read only memory
  • CD-ROM compact disk read only memory
  • magnetic storage or any suitable combination of the foregoing.
  • Example 1 provides a video classification method, including: transforming the video to be classified through the target transformation group to obtain multiple transformed videos; according to the multiple transformed videos, through training A good video classification model is used to determine the video classification result of the video to be classified; wherein, the video classification model is used to determine the target video features corresponding to the converted video according to the converted video, and to determine the corresponding target video features according to the target video features. Determining the video classification result; the target video feature is a feature with transformation invariance in the transformed video.
  • Example 2 provides the method of Example 1.
  • the video to be classified is transformed through the target transformation group to obtain multiple transformed videos, including: from a plurality of preset transformation groups, determining the target transformation group; the multiple preset transformation groups include a rotation group, a scaling group and an affine transformation group, each of the preset transformation groups includes a plurality of transformation matrices; through the target transformation group Each target transformation matrix respectively transforms the video to be classified to obtain a transformed video corresponding to each target transformation matrix.
  • Example 3 provides the method of Example 2, the determining the target transformation group from a plurality of preset transformation groups includes: determining the target object in the video to be classified Corresponding object information, the object information includes the position, direction and size of the target object; according to the object information, the target transformation group is determined from a plurality of the preset transformation groups.
  • Example 4 provides the method of Example 1, the video classification model includes a Siamese network, a maximum pooling layer and a classifier, the Siamese network includes a plurality of neural sub-networks, the Transformed videos are in one-to-one correspondence with the neural sub-network; according to a plurality of transformed videos, through a trained video classification model, determining the video classification result of the video to be classified includes: converting each transformed video Input into the neural sub-network corresponding to the transformed video to extract the video features of the transformed video; through the maximum pooling layer, perform maximum pooling processing on the video features of each transformed video to obtain the target video features ; Using the classifier to determine the video classification result according to the target video features.
  • Example 5 provides the method of Example 1, the video classification model is obtained by: obtaining a training sample set; the training sample set includes training videos, and the training The training video classification result corresponding to the video; the training video is transformed by each preset transformation group to obtain a plurality of training transformation videos corresponding to each of the preset transformation groups; according to the multiple training transformation videos and the obtained The classification result of the training video is used to train the preset model to obtain the video classification model.
  • Example 6 provides a video classification device, the device includes: a transformation module, which is used to transform the video to be classified through a target transformation group to obtain multiple transformed videos; a determination module , for determining the video classification result of the video to be classified by using a trained video classification model according to a plurality of transformed videos; wherein, the video classification model is used for determining the transformed video according to the transformed video corresponding target video feature, and determine the video classification result according to the target video feature; the target video feature is a feature with transformation invariance in the transformed video.
  • Example 7 provides the device of Example 6, the transformation module includes: a determining submodule, configured to determine the target transformation group from a plurality of preset transformation groups; A plurality of preset transformation groups include a rotation group, a scaling group and an affine transformation group, and each of the preset transformation groups includes a plurality of transformation matrices; a transformation sub-module is used to pass each target in the target transformation group Transformation matrices, respectively transforming the videos to be classified to obtain transformed videos corresponding to each of the target transformation matrices.
  • Example 8 provides the apparatus of Example 6, the video classification model includes a Siamese network, a maximum pooling layer and a classifier, the Siamese network includes a plurality of neural subnetworks, the Transformed videos correspond one-to-one to the neural sub-network; the determination module includes: a feature extraction sub-module, which is used to input each transformed video into the neural sub-network corresponding to the transformed video, so as to extract the features of the transformed video Video features; pooling sub-module, for performing maximum pooling processing on the video features of each of the transformed videos through the maximum pooling layer, to obtain the target video features; classification sub-module, for passing the classification The device determines the video classification result according to the target video feature.
  • the video classification model includes a Siamese network, a maximum pooling layer and a classifier, the Siamese network includes a plurality of neural subnetworks, the Transformed videos correspond one-to-one to the neural sub-network; the determination module includes: a feature extraction sub-module
  • Example 9 provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processing device, the steps of the methods described in Example 1 to Example 5 are implemented.
  • Example 10 provides an electronic device, including: a storage device, on which a computer program is stored; a processing device, configured to execute the computer program in the storage device, to Implement the steps of the method described in Example 1 to Example 5.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Image Analysis (AREA)

Abstract

本公开涉及一种视频分类方法、装置、可读介质和电子设备，涉及视频处理技术领域，该方法包括：通过目标变换群，对待分类视频进行变换，得到多个变换视频，根据多个变换视频，通过训练好的视频分类模型，确定待分类视频的视频分类结果，其中，视频分类模型用于根据变换视频，确定变换视频对应的目标视频特征，并根据目标视频特征确定视频分类结果，目标视频特征为变换视频中具有变换不变性的特征。本公开利用视频分类模型从多个变换视频中提取具有变换不变性的目标视频特征，并通过目标视频特征来进行视频分类，能够避免旋转、放缩或仿射变换对视频分类的影响，提高了视频分类的准确性。

Description

视频分类方法、装置、可读介质和电子设备
相关申请的交叉引用
本申请基于申请号为202110926870.1、申请日为2021年8月12日，名称为“视频分类方法、装置、可读介质和电子设备”的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本公开涉及视频处理技术领域，具体地，涉及一种视频分类方法、装置、可读介质和电子设备。
背景技术
随着互联网技术的不断发展，用户上传到互联网中的视频也越来越多。为了更好地对互联网中的视频进行分析管理，需要对视频进行分类。当前，对视频进行分类，通常是使用端对端的CNN(英文：Convolutional Neural Networks，中文：卷积神经网络)模型来学习视频中隐含的时空关系，来进行视频分类。然而，在实际情况中，由于视频采集装置的摆放朝向和距离远近是任意的，因此获取到的视频图像相对于标准图像可能会产生一些旋转、放缩或仿射变换。这些变换会导致CNN模型很难统一进行特征学习，并影响CNN模型的泛化能力，降低了视频分类的准确性。
发明内容
提供该发明内容部分以便以简要的形式介绍构思，这些构思将在后面的具体实施方式部分被详细描述。该发明内容部分并不旨在标识要求保护的技术方案的关键特征或必要特征，也不旨在用于限制所要求的保护的技术方案的范围。
第一方面，本公开提供一种视频分类方法，所述方法包括：
通过目标变换群，对待分类视频进行变换，得到多个变换视频；
根据多个所述变换视频，通过训练好的视频分类模型，确定所述待分类视频的视频分类结果；
其中，所述视频分类模型用于根据所述变换视频，确定所述变换视频对应的目标视频特征，并根据所述目标视频特征确定所述视频分类结果；所述目标视频特征为所述变换视频中具有变换不变性的特征。
第二方面，本公开提供一种视频分类装置，所述装置包括：
变换模块，用于通过目标变换群，对待分类视频进行变换，得到多个变换视频；
确定模块，用于根据多个所述变换视频，通过训练好的视频分类模型，确定所述待分类视频的视频分类结果；
其中，所述视频分类模型用于根据所述变换视频，确定所述变换视频对应的目标视频特征，并根据所述目标视频特征确定所述视频分类结果；所述目标视频特征为所述变换视频中具有变换不变性的特征。
第三方面，本公开提供一种计算机可读介质，其上存储有计算机程序，该程序被处理装置执行时实现本公开第一方面所述方法的步骤。
第四方面，本公开提供一种电子设备，包括：
存储装置，其上存储有计算机程序；
处理装置，用于执行所述存储装置中的所述计算机程序，以实现本公开第一方面所述方法的步骤。
本公开的其他特征和优点将在随后的具体实施方式部分予以详细说明。
附图说明
结合附图并参考以下具体实施方式，本公开各实施例的上述和其他特征、优点及方面将变得更加明显。贯穿附图中，相同或相似的附图标记表示相同或相似的元素。应当理解附图是示意性的，原件和元素不一定按照比例绘制。在附图中：
图1是根据一示例性实施例示出的一种视频分类方法的流程图；
图2是根据图1所示实施例示出的一种步骤101的流程图；
图3是根据图1所示实施例示出的一种步骤102的流程图；
图4是根据一示例性实施例示出的一种训练视频分类模型的流程图；
图5是根据一示例性实施例示出的一种视频分类装置的框图；
图6是根据图5所示实施例示出的一种变换模块的框图；
图7是根据图5所示实施例示出的一种确定模块的框图；
图8是根据一示例性实施例示出的一种电子设备的框图。
具体实施方式
下面将参照附图更详细地描述本公开的实施例。虽然附图中显示了本公开的某些实施例，然而应当理解的是，本公开可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实施例，相反提供这些实施例是为了更加透彻和完整地理解本公开。应当理解的是，本公开的附图及实施例仅用于示例性作用，并非用于限制本公开的保护范围。
应当理解，本公开的方法实施方式中记载的各个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本公开的范围在此方面不受限制。
本文使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。
需要注意，本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分，并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。
需要注意，本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有明确指出，否则应该理解为“一个或多个”。
本公开实施方式中的多个装置之间所交互的消息或者信息的名称 仅用于说明性的目的，而并不是用于对这些消息或信息的范围进行限制。
图1是根据一示例性实施例示出的一种视频分类方法的流程图。如图1所示，该方法可以包括以下步骤：
步骤101，通过目标变换群，对待分类视频进行变换，得到多个变换视频。
示例地，为了避免旋转、放缩以及仿射变换对视频分类的影响，可以通过提取视频中对于指定类型的变换具有变换不变性的特征(指定类型的变换例如可以是旋转变换、放缩变换或一般的仿射变换等)，来进行视频分类。具体的，首先可以预先根据先验知识，确定需要针对哪些指定类型的变换提取具有变换不变性的特征，并根据这些指定类型的变换，生成每个指定类型的变换对应的预设变换群。其中，每个指定类型的变换对应的预设变换群为该指定类型的多个变换所构成的群。例如，在指定类型的变换为旋转变换的情况下，预设变换群可以为由多个旋转变换构成的旋转群，在指定类型的变换为放缩变换的情况下，预设变换群可以为多个放缩变换构成的放缩群，在指定类型的变换为仿射变换的情况下，预设变换群可以为多个仿射变换构成的仿射变换群。
然后，可以获取待分类视频，并根据待分类视频从预先生成的多个预设变换群中，确定目标变换群。确定目标变换群有多种实现方式，一种可实现的方式为根据待分类视频的特点，从多个预设变换群中，选取目标变换群，例如，如果待分类视频中的对象(该对象例如可以是人物、物品)的尺寸较大，则可以选取放缩群作为目标变换群。之后可以通过目标变换群包括的多个变换，分别对待分类视频进行变换，得到每个变换对应的变换视频。
步骤102，根据多个变换视频，通过训练好的视频分类模型，确定待分类视频的视频分类结果。
其中，视频分类模型用于根据变换视频，确定变换视频对应的目标视频特征，并根据目标视频特征确定视频分类结果，目标视频特征为变换视频中具有变换不变性的特征。
举例来说，在获取到多个变换视频后，可以将多个变换视频输入到训练好的视频分类模型中。由视频分类模型分别提取每个变换视频的视频特征，并对提取到的全部变换视频的视频特征进行最大池化处理，以得到变换视频中具有变换不变性的目标视频特征。例如，当目标变换群为旋转群时，多个变换视频实际上是对待分类视频进行不同旋转变换后得到的，此时，得到的目标视频特征实际上为待分类视频中对旋转变换具有不变性的特征。然后，视频分类模型可以根据目标视频特征，从多个预设的视频类型中，确定待分类视频的视频分类结果。例如，在对待分类视频进行视频内容安全分类的场景下，多个预设的视频类型可以包括正常视频和多种非正常视频类型。
需要说明的是，本公开实际上是将待分类视频作为一个整体，通过目标变换群以及视频分类模型，来得到待分类视频的视频分类结果。另外，还可以从待分类视频抽取指定数量帧视频图像，并针对每帧视频图像，通过目标变换群以及视频分类模型，得到每帧视频图像的视频分类结果，然后可以根据全部帧视频图像的视频分类结果，确定待分类视频的视频分类结果。进一步地，本公开的视频分类方法不仅可以应用于对视频进行分类，还可以应用于对图像进行分类，本公开对此不作具体限定。
综上所述，本公开首先通过目标变换群，对待分类视频进行变换，得到多个变换视频，之后根据多个变换视频，通过训练好的视频分类模型，确定待分类视频的视频分类结果，其中，视频分类模型用于根据变换视频，确定变换视频对应的目标视频特征，并根据目标视频特征确定视频分类结果，目标视频特征为变换视频中具有变换不变性的特征。本公开利用视频分类模型从多个变换视频中提取具有变换不变性的目标视频特征，并通过目标视频特征来进行视频分类，能够避免旋转、放缩或仿射变换对视频分类的影响，提高了视频分类的准确性。
图2是根据图1所示实施例示出的一种步骤101的流程图。如图2所示，步骤101可以包括以下步骤：
步骤1011，从多个预设变换群中，确定目标变换群。其中，多个预设变换群包括旋转群、放缩群和仿射变换群，每个预设变换群包括 多个变换矩阵。
示例地，可以先利用预设识别算法，确定待分类视频中的目标对象对应的对象信息。然后，可以根据对象信息，从多个预设变换群中，确定目标变换群。其中，对象信息可以包括目标对象的位置、方向和尺寸，每个预设变换群可以包括多个变换，每个变换对应一个变换矩阵。
举个例子，可以预设设置有一个标准图像，如果待分类视频中的目标对象的方向与标准图像中的对象的方向差别较大，则可以选择旋转群作为目标变换群，如果待分类视频中的目标对象的尺寸与标准图像中的对象的尺寸差别较大，则可以选择放缩群作为目标变换群。
步骤1012，通过目标变换群中的每个目标变换矩阵，分别对待分类视频进行变换，得到每个目标变换矩阵对应的变换视频。
具体地，以目标变换群为旋转群，且旋转群包括4个旋转变换为例进行说明，若4个旋转变换为分别将待分类视频顺时针旋转45°、90°、135°和180°，则可以在确定目标变换群后，分别通过目标变换群中每个旋转变换对应的目标变换矩阵，对待分类视频进行变换(此时的变换为旋转变换)，得到分别将待分类视频顺时针旋转45°、90°、135°和180°的4个变换视频。
图3是根据图1所示实施例示出的一种步骤102的流程图。如图3所示，视频分类模型包括孪生网络、最大池化层和分类器，孪生网络包括多个神经子网络，变换视频与神经子网络一一对应。步骤102可以包括以下步骤：
步骤1021，将每个变换视频输入到该变换视频对应的神经子网络中，以提取该变换视频的视频特征。
步骤1022，通过最大池化层对每个变换视频的视频特征进行最大池化处理，得到目标视频特征。
步骤1023，通过分类器根据目标视频特征，确定视频分类结果。
在一种场景中，为了确保视频分类结果的准确性，需要避免旋转、放缩以及仿射变换对视频分类的影响。因此，可以构建对于旋转变换、放缩变换以及仿射变换具有不变性的视频分类模型。具体的，可以通 过孪生网络、最大池化层和分类器来构建视频分类模型。其中，孪生网络包括多个神经子网络，这些神经子网络之间共享网络权值和网络参数。孪生网络例如可以采用3D-CNN或双流CNN，分类器可以采用线性分类器。
在获取到多个变换视频后，可以将每个变换视频输入到该变换视频对应的神经子网络中，得到该变换视频的n维的视频特征。然后可以将每个变换视频的视频特征输入到最大池化层中，由最大池化层对各个神经子网络得到的变换视频的视频特征逐元素进行最大池化操作，并输出目标视频特征。之后可以将目标视频特征输入到分类器中，由分类器根据目标视频特征，确定视频分类结果。
图4是根据一示例性实施例示出的一种训练视频分类模型的流程图。如图4所示，视频分类模型是通过以下方式得到的：
步骤201，获取训练样本集。
其中，训练样本集包括训练视频，以及训练视频对应的训练视频分类结果。
步骤202，通过每个预设变换群对训练视频进行变换，得到每个预设变换群对应的多个训练变换视频。
步骤203，根据多个训练变换视频和训练视频分类结果，对预设模型进行训练，得到视频分类模型。
举例来说，在对视频分类模型进行训练时，首先可以从业务线采集视频，并将采集到的视频分为训练样本集和测试样本集。其中，训练样本集包括训练视频，以及训练视频对应的训练视频分类结果，测试样本集包括测试视频，以及测试视频对应的测试视频分类结果。其次，可以通过每个预设变换群对训练视频进行变换，得到每个预设变换群对应的多个训练变换视频。之后可以针对每个预设变换群，将该预设变换群对应的多个训练变换视频作为预设模型的输入，将训练视频分类结果作为预设模型的输出，并利用SGD(英文：Stochastic Gradient Descent，中文：随机梯度下降)等优化器对预设模型进行训练，以得到视频分类模型。例如，预设模型可以包括孪生网络、最大池化层和分类器，孪生网络可以包括多个神经子网络，在对预设模型 进行训练时，可以将多个训练变换视频分别输入到不同的神经子网络中来进行视频特征提取，并将训练视频分类结果作为分类器的输出，来对预设模型进行训练。
然后可以使用测试样本集对得到的视频分类模型进行性能测试(例如，可以通过视频分类模型输出的视频分类结果的准确率，来判断视频分类模型的性能)，若视频分类模型的性能达不到要求，则重新对视频分类模型进行训练，直到视频分类模型的性能达到要求。
综上所述，本公开首先通过目标变换群，对待分类视频进行变换，得到多个变换视频，之后根据多个变换视频，通过训练好的视频分类模型，确定待分类视频的视频分类结果，其中，视频分类模型用于根据变换视频，确定变换视频对应的目标视频特征，并根据目标视频特征确定视频分类结果，目标视频特征为变换视频中具有变换不变性的特征。本公开利用视频分类模型从多个变换视频中提取具有变换不变性的目标视频特征，并通过目标视频特征来进行视频分类，能够避免旋转、放缩或仿射变换对视频分类的影响，提高了视频分类的准确性。
图5是根据一示例性实施例示出的一种视频分类装置的框图。如图5所示，该装置300包括：
变换模块301，用于通过目标变换群，对待分类视频进行变换，得到多个变换视频。
确定模块302，用于根据多个变换视频，通过训练好的视频分类模型，确定待分类视频的视频分类结果。
其中，视频分类模型用于根据变换视频，确定变换视频对应的目标视频特征，并根据目标视频特征确定视频分类结果，目标视频特征为变换视频中具有变换不变性的特征。
图6是根据图5所示实施例示出的一种变换模块的框图。如图6所示，变换模块301包括：
确定子模块3011，用于从多个预设变换群中，确定目标变换群，多个预设变换群包括旋转群、放缩群和仿射变换群，每个预设变换群包括多个变换矩阵。
变换子模块3012，用于通过目标变换群中的每个目标变换矩阵， 分别对待分类视频进行变换，得到每个目标变换矩阵对应的变换视频。
可选地，确定子模块3011用于：
确定待分类视频中的目标对象对应的对象信息，对象信息包括目标对象的位置、方向和尺寸。
根据对象信息，从多个预设变换群中，确定目标变换群。
图7是根据图5所示实施例示出的一种确定模块的框图。如图7所示，视频分类模型包括孪生网络、最大池化层和分类器，孪生网络包括多个神经子网络，变换视频与神经子网络一一对应，确定模块302包括：
特征提取子模块3021，用于将每个变换视频输入到该变换视频对应的神经子网络中，以提取该变换视频的视频特征。
池化子模块3022，用于通过最大池化层对每个变换视频的视频特征进行最大池化处理，得到目标视频特征。
分类子模块3023，用于通过分类器根据目标视频特征，确定视频分类结果。
可选地，确定模块302用于通过以下方式训练得到视频分类模型：
获取训练样本集，训练样本集包括训练视频，以及训练视频对应的训练视频分类结果。
通过每个预设变换群对训练视频进行变换，得到每个预设变换群对应的多个训练变换视频。
根据多个训练变换视频和训练视频分类结果，对预设模型进行训练，得到视频分类模型。
综上所述，本公开首先通过目标变换群，对待分类视频进行变换，得到多个变换视频，之后根据多个变换视频，通过训练好的视频分类模型，确定待分类视频的视频分类结果，其中，视频分类模型用于根据变换视频，确定变换视频对应的目标视频特征，并根据目标视频特征确定视频分类结果，目标视频特征为变换视频中具有变换不变性的特征。本公开利用视频分类模型从多个变换视频中提取具有变换不变性的目标视频特征，并通过目标视频特征来进行视频分类，能够避免旋转、放缩或仿射变换对视频分类的影响，提高了视频分类的准确性。
下面参考图8，其示出了适于用来实现本公开实施例的电子设备(例如图1中的终端设备或服务器)400的结构示意图。本公开实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图8示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图8所示，电子设备400可以包括处理装置(例如中央处理器、图形处理器等)401，其可以根据存储在只读存储器(ROM)402中的程序或者从存储装置408加载到随机访问存储器(RAM)403中的程序而执行各种适当的动作和处理。在RAM 403中，还存储有电子设备400操作所需的各种程序和数据。处理装置401、ROM 402以及RAM 403通过总线404彼此相连。输入/输出(I/O)接口405也连接至总线404。
通常，以下装置可以连接至I/O接口405：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置406；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置407；包括例如磁带、硬盘等的存储装置408；以及通信装置409。通信装置409可以允许电子设备400与其他设备进行无线或有线通信以交换数据。虽然图8示出了具有各种装置的电子设备400，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置409从网络上被下载和安装，或者从存储装置408被安装，或者从ROM 402被安装。在该计算机程序被处理装置401执行时，执行本公开实施例的方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如HTTP(HyperText Transfer Protocol，超文本传输协议)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(“LAN”)，广域网(“WAN”)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：通过目标变换群， 对待分类视频进行变换，得到多个变换视频；根据多个所述变换视频，通过训练好的视频分类模型，确定所述待分类视频的视频分类结果；其中，所述视频分类模型用于根据所述变换视频，确定所述变换视频对应的目标视频特征，并根据所述目标视频特征确定所述视频分类结果；所述目标视频特征为所述变换视频中具有变换不变性的特征。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言——诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)——连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的模块可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，模块的名称在某种情况下并不构成对该模块本身的限定，例如，变换模块还可以被描述为“对待分类视频进行变换，得到多个变换视频的模块”。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
根据本公开的一个或多个实施例，示例1提供了一种视频分类方法，包括：通过目标变换群，对待分类视频进行变换，得到多个变换视频；根据多个所述变换视频，通过训练好的视频分类模型，确定所述待分类视频的视频分类结果；其中，所述视频分类模型用于根据所述变换视频，确定所述变换视频对应的目标视频特征，并根据所述目标视频特征确定所述视频分类结果；所述目标视频特征为所述变换视频中具有变换不变性的特征。
根据本公开的一个或多个实施例，示例2提供了示例1的方法，所述通过目标变换群，对待分类视频进行变换，得到多个变换视频，包括：从多个预设变换群中，确定所述目标变换群；所述多个预设变换群包括旋转群、放缩群和仿射变换群，每个所述预设变换群包括多个变换矩阵；通过所述目标变换群中的每个目标变换矩阵，分别对所述待分类视频进行变换，得到每个所述目标变换矩阵对应的变换视频。
根据本公开的一个或多个实施例，示例3提供了示例2的方法，所述从多个预设变换群中，确定所述目标变换群，包括：确定所述待 分类视频中的目标对象对应的对象信息，所述对象信息包括所述目标对象的位置、方向和尺寸；根据所述对象信息，从多个所述预设变换群中，确定所述目标变换群。
根据本公开的一个或多个实施例，示例4提供了示例1的方法，所述视频分类模型包括孪生网络、最大池化层和分类器，所述孪生网络包括多个神经子网络，所述变换视频与所述神经子网络一一对应；所述根据多个所述变换视频，通过训练好的视频分类模型，确定所述待分类视频的视频分类结果，包括：将每个所述变换视频输入到该变换视频对应的神经子网络中，以提取该变换视频的视频特征；通过所述最大池化层对每个所述变换视频的视频特征进行最大池化处理，得到所述目标视频特征；通过所述分类器根据所述目标视频特征，确定所述视频分类结果。
根据本公开的一个或多个实施例，示例5提供了示例1的方法，所述视频分类模型是通过以下方式得到的：获取训练样本集；所述训练样本集包括训练视频，以及所述训练视频对应的训练视频分类结果；通过每个预设变换群对所述训练视频进行变换，得到每个所述预设变换群对应的多个训练变换视频；根据所述多个训练变换视频和所述训练视频分类结果，对预设模型进行训练，得到所述视频分类模型。
根据本公开的一个或多个实施例，示例6提供了一种视频分类装置，所述装置包括：变换模块，用于通过目标变换群，对待分类视频进行变换，得到多个变换视频；确定模块，用于根据多个所述变换视频，通过训练好的视频分类模型，确定所述待分类视频的视频分类结果；其中，所述视频分类模型用于根据所述变换视频，确定所述变换视频对应的目标视频特征，并根据所述目标视频特征确定所述视频分类结果；所述目标视频特征为所述变换视频中具有变换不变性的特征。
根据本公开的一个或多个实施例，示例7提供了示例6的装置，所述变换模块包括：确定子模块，用于从多个预设变换群中，确定所述目标变换群；所述多个预设变换群包括旋转群、放缩群和仿射变换群，每个所述预设变换群包括多个变换矩阵；变换子模块，用于通过所述目标变换群中的每个目标变换矩阵，分别对所述待分类视频进行 变换，得到每个所述目标变换矩阵对应的变换视频。
根据本公开的一个或多个实施例，示例8提供了示例6的装置，所述视频分类模型包括孪生网络、最大池化层和分类器，所述孪生网络包括多个神经子网络，所述变换视频与所述神经子网络一一对应；所述确定模块包括：特征提取子模块，用于将每个所述变换视频输入到该变换视频对应的神经子网络中，以提取该变换视频的视频特征；池化子模块，用于通过所述最大池化层对每个所述变换视频的视频特征进行最大池化处理，得到所述目标视频特征；分类子模块，用于通过所述分类器根据所述目标视频特征，确定所述视频分类结果。
根据本公开的一个或多个实施例，示例9提供了一种计算机可读介质，其上存储有计算机程序，该程序被处理装置执行时实现示例1至示例5中所述方法的步骤。
根据本公开的一个或多个实施例，示例10提供了一种电子设备，包括：存储装置，其上存储有计算机程序；处理装置，用于执行所述存储装置中的所述计算机程序，以实现示例1至示例5中所述方法的步骤。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了 本主题，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。

Claims (10)

  1. 一种视频分类方法，其特征在于，所述方法包括：
    通过目标变换群，对待分类视频进行变换，得到多个变换视频；
    根据多个所述变换视频，通过训练好的视频分类模型，确定所述待分类视频的视频分类结果；
    其中，所述视频分类模型用于根据所述变换视频，确定所述变换视频对应的目标视频特征，并根据所述目标视频特征确定所述视频分类结果；所述目标视频特征为所述变换视频中具有变换不变性的特征。
  2. 根据权利要求1所述的方法，其特征在于，所述通过目标变换群，对待分类视频进行变换，得到多个变换视频，包括：
    从多个预设变换群中，确定所述目标变换群；所述多个预设变换群包括旋转群、放缩群和仿射变换群，每个所述预设变换群包括多个变换矩阵；
    通过所述目标变换群中的每个目标变换矩阵，分别对所述待分类视频进行变换，得到每个所述目标变换矩阵对应的变换视频。
  3. 根据权利要求2所述的方法，其特征在于，所述从多个预设变换群中，确定所述目标变换群，包括：
    确定所述待分类视频中的目标对象对应的对象信息，所述对象信息包括所述目标对象的位置、方向和尺寸；
    根据所述对象信息，从多个所述预设变换群中，确定所述目标变换群。
  4. 根据权利要求1所述的方法，其特征在于，所述视频分类模型包括孪生网络、最大池化层和分类器，所述孪生网络包括多个神经子网络，所述变换视频与所述神经子网络一一对应；所述根据多个所述变换视频，通过训练好的视频分类模型，确定所述待分类视频的视频分类结果，包括：
    将每个所述变换视频输入到该变换视频对应的神经子网络中，以提取该变换视频的视频特征；
    通过所述最大池化层对每个所述变换视频的视频特征进行最大池化处理，得到所述目标视频特征；
    通过所述分类器根据所述目标视频特征，确定所述视频分类结果。
  5. 根据权利要求1所述的方法，其特征在于，所述视频分类模型是通过以下方式得到的：
    获取训练样本集；所述训练样本集包括训练视频，以及所述训练视频对应的训练视频分类结果；
    通过每个预设变换群对所述训练视频进行变换，得到每个所述预设变换群对应的多个训练变换视频；
    根据所述多个训练变换视频和所述训练视频分类结果，对预设模型进行训练，得到所述视频分类模型。
  6. 一种视频分类装置，其特征在于，所述装置包括：
    变换模块，用于通过目标变换群，对待分类视频进行变换，得到多个变换视频；
    确定模块，用于根据多个所述变换视频，通过训练好的视频分类模型，确定所述待分类视频的视频分类结果；
    其中，所述视频分类模型用于根据所述变换视频，确定所述变换视频对应的目标视频特征，并根据所述目标视频特征确定所述视频分类结果；所述目标视频特征为所述变换视频中具有变换不变性的特征。
  7. 根据权利要求6所述的装置，其特征在于，所述变换模块包括：
    确定子模块，用于从多个预设变换群中，确定所述目标变换群；所述多个预设变换群包括旋转群、放缩群和仿射变换群，每个所述预设变换群包括多个变换矩阵；
    变换子模块，用于通过所述目标变换群中的每个目标变换矩阵，分别对所述待分类视频进行变换，得到每个所述目标变换矩阵对应的 变换视频。
  8. 根据权利要求6所述的装置，其特征在于，所述视频分类模型包括孪生网络、最大池化层和分类器，所述孪生网络包括多个神经子网络，所述变换视频与所述神经子网络一一对应；所述确定模块包括：
    特征提取子模块，用于将每个所述变换视频输入到该变换视频对应的神经子网络中，以提取该变换视频的视频特征；
    池化子模块，用于通过所述最大池化层对每个所述变换视频的视频特征进行最大池化处理，得到所述目标视频特征；
    分类子模块，用于通过所述分类器根据所述目标视频特征，确定所述视频分类结果。
  9. 一种计算机可读介质，其上存储有计算机程序，其特征在于，该程序被处理装置执行时实现权利要求1-5中任一项所述方法的步骤。
  10. 一种电子设备，其特征在于，包括：
    存储装置，其上存储有计算机程序；
    处理装置，用于执行所述存储装置中的所述计算机程序，以实现权利要求1-5中任一项所述方法的步骤。
PCT/CN2022/109470 2021-08-12 2022-08-01 视频分类方法、装置、可读介质和电子设备 Ceased WO2023016290A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110926870.1 2021-08-12
CN202110926870.1A CN113705386A (zh) 2021-08-12 2021-08-12 视频分类方法、装置、可读介质和电子设备

Publications (1)

Publication Number Publication Date
WO2023016290A1 true WO2023016290A1 (zh) 2023-02-16

Family

ID=78652514

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/109470 Ceased WO2023016290A1 (zh) 2021-08-12 2022-08-01 视频分类方法、装置、可读介质和电子设备

Country Status (2)

Country Link
CN (1) CN113705386A (zh)
WO (1) WO2023016290A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113705386A (zh) * 2021-08-12 2021-11-26 北京有竹居网络技术有限公司 视频分类方法、装置、可读介质和电子设备

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105160358A (zh) * 2015-09-07 2015-12-16 苏州大学张家港工业技术研究院 一种图像分类方法及系统
CN108985217A (zh) * 2018-07-10 2018-12-11 常州大学 一种基于深度空间网络的交通标志识别方法及系统
CN111353548A (zh) * 2020-03-11 2020-06-30 中国人民解放军军事科学院国防科技创新研究院 一种基于对抗空间变换网络的鲁棒特征深度学习方法
CN111401452A (zh) * 2020-03-17 2020-07-10 北京大学 一种基于偏微分算子的等变卷积网络模型的图像分类方法
WO2020221278A1 (zh) * 2019-04-29 2020-11-05 北京金山云网络技术有限公司 视频分类方法及其模型的训练方法、装置和电子设备
CN112257753A (zh) * 2020-09-23 2021-01-22 北京大学 基于偏微分算子的广义等变卷积网络模型的图像分类方法
US20210034913A1 (en) * 2018-05-23 2021-02-04 Beijing Sensetime Technology Development Co., Ltd. Method and device for image processing, and computer storage medium
CN112990315A (zh) * 2021-03-17 2021-06-18 北京大学 基于偏微分算子的等变3d卷积网络的3d形状图像分类方法
CN113033677A (zh) * 2021-03-30 2021-06-25 北京有竹居网络技术有限公司 视频分类方法、装置、电子设备和存储介质
CN113705386A (zh) * 2021-08-12 2021-11-26 北京有竹居网络技术有限公司 视频分类方法、装置、可读介质和电子设备

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109145927A (zh) * 2017-06-16 2019-01-04 杭州海康威视数字技术股份有限公司 一种对形变图像的目标识别方法及装置
CN109840556B (zh) * 2019-01-24 2020-10-23 浙江大学 一种基于孪生网络的图像分类识别方法
CN110287836B (zh) * 2019-06-14 2021-10-15 北京迈格威科技有限公司 图像分类方法、装置、计算机设备和存储介质
CN110377787B (zh) * 2019-06-21 2022-03-25 北京奇艺世纪科技有限公司 一种视频分类方法、装置及计算机可读存储介质
CN110347876A (zh) * 2019-07-12 2019-10-18 Oppo广东移动通信有限公司 视频分类方法、装置、终端设备及计算机可读存储介质
CN111612093A (zh) * 2020-05-29 2020-09-01 Oppo广东移动通信有限公司 一种视频分类方法、视频分类装置、电子设备及存储介质

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105160358A (zh) * 2015-09-07 2015-12-16 苏州大学张家港工业技术研究院 一种图像分类方法及系统
US20210034913A1 (en) * 2018-05-23 2021-02-04 Beijing Sensetime Technology Development Co., Ltd. Method and device for image processing, and computer storage medium
CN108985217A (zh) * 2018-07-10 2018-12-11 常州大学 一种基于深度空间网络的交通标志识别方法及系统
WO2020221278A1 (zh) * 2019-04-29 2020-11-05 北京金山云网络技术有限公司 视频分类方法及其模型的训练方法、装置和电子设备
CN111353548A (zh) * 2020-03-11 2020-06-30 中国人民解放军军事科学院国防科技创新研究院 一种基于对抗空间变换网络的鲁棒特征深度学习方法
CN111401452A (zh) * 2020-03-17 2020-07-10 北京大学 一种基于偏微分算子的等变卷积网络模型的图像分类方法
CN112257753A (zh) * 2020-09-23 2021-01-22 北京大学 基于偏微分算子的广义等变卷积网络模型的图像分类方法
CN112990315A (zh) * 2021-03-17 2021-06-18 北京大学 基于偏微分算子的等变3d卷积网络的3d形状图像分类方法
CN113033677A (zh) * 2021-03-30 2021-06-25 北京有竹居网络技术有限公司 视频分类方法、装置、电子设备和存储介质
CN113705386A (zh) * 2021-08-12 2021-11-26 北京有竹居网络技术有限公司 视频分类方法、装置、可读介质和电子设备

Also Published As

Publication number Publication date
CN113705386A (zh) 2021-11-26

Similar Documents

Publication Publication Date Title
CN109816589B (zh) 用于生成漫画风格转换模型的方法和装置
WO2022252881A1 (zh) 图像处理方法、装置、可读介质和电子设备
CN111210485B (zh) 图像的处理方法、装置、可读介质和电子设备
CN112364860B (zh) 字符识别模型的训练方法、装置和电子设备
CN114494709B (zh) 特征提取模型的生成方法、图像特征提取方法和装置
WO2022171036A1 (zh) 视频目标追踪方法、视频目标追踪装置、存储介质及电子设备
CN113033682B (zh) 视频分类方法、装置、可读介质、电子设备
CN118097157B (zh) 基于模糊聚类算法的图像分割方法及系统
CN113033580B (zh) 图像处理方法、装置、存储介质及电子设备
WO2022105779A1 (zh) 图像处理方法、模型训练方法、装置、介质及设备
CN113222050B (zh) 图像分类方法、装置、可读介质及电子设备
CN115270981A (zh) 对象处理方法、装置、可读介质及电子设备
WO2023035877A1 (zh) 视频的识别方法、装置、可读介质和电子设备
CN114429658A (zh) 人脸关键点信息获取方法、生成人脸动画的方法及装置
CN110222829A (zh) 基于卷积神经网络的特征提取方法、装置、设备及介质
WO2023000782A1 (zh) 获取视频热点的方法、装置、可读介质和电子设备
WO2023030426A1 (zh) 息肉识别方法、装置、介质及设备
WO2023093481A1 (zh) 基于傅里叶域的超分图像处理方法、装置、设备及介质
CN113033707B (zh) 视频分类方法、装置、可读介质及电子设备
CN112418233B (zh) 图像处理方法、装置、可读介质及电子设备
CN112418249B (zh) 掩膜图像生成方法、装置、电子设备和计算机可读介质
CN116524532B (zh) 动作识别方法、装置、存储介质以及电子设备
WO2023130925A1 (zh) 字体识别方法、装置、可读介质及电子设备
WO2023016290A1 (zh) 视频分类方法、装置、可读介质和电子设备
CN116503596A (zh) 图片分割方法、装置、介质和电子设备

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22855273

Country of ref document: EP

Kind code of ref document: A1