CN110852195A - Video slice-based video type classification method - Google Patents
Video slice-based video type classification method Download PDFInfo
- Publication number
- CN110852195A CN110852195A CN201911015725.7A CN201911015725A CN110852195A CN 110852195 A CN110852195 A CN 110852195A CN 201911015725 A CN201911015725 A CN 201911015725A CN 110852195 A CN110852195 A CN 110852195A
- Authority
- CN
- China
- Prior art keywords
- video
- classification
- slice
- image
- classification method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000013528 artificial neural network Methods 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 2
- 238000001514 detection method Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000005034 decoration Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000003475 lamination Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a video slice-based video type classification method, which comprises the following steps of: 101) a setting and classifying step, 102) a slice image extracting step, 103) a category classifying step and 104) a category judging step; the invention provides a video slice-based video type classification method with performance improved by at least 100 times.
Description
Technical Field
The invention relates to the technical field of video classification, in particular to a video slice-based video type classification method.
Background
Video content analysis is a basic technology of video processing, and comprises the following steps: (1) lens switching detection; (2) type classification, determining comedy, war, suspicion, animation and the like; (3) specific object detection, such as detecting human faces, cars, billboards; (4) semantic analysis; (5) content ratio is equal. The video content analysis technology can be applied to various fields, such as video editing software, video recommendation, content verification, advertisement delivery, copyright maintenance and the like.
A video is composed of many images, a 10-minute video can contain 1 ten thousand multi-frame images, and it is difficult to simply analyze the video content based on the images, and assuming that one image is processed for 50ms (usually longer than this time), 1 ten thousand images take 8 minutes. Therefore, the current application limitation is large, the accuracy of the mature copyright maintenance (adopting a video comparison technology) is still inferior to that of manual work, and a large number of servers need to be put in. Therefore, many of the tasks related to video processing are basically completed by manual work, such as content review of video websites.
Disclosure of Invention
The invention overcomes the defects of the prior art and provides a video slice-based video type classification method.
The technical scheme of the invention is as follows:
a video slice-based video type classification method comprises the following steps:
101) setting and classifying: designing classification labels according to application scenes, wherein the classification labels comprise comedy, science fiction, cartoon, pornography and violence and terrorism;
102) extracting slice images: the video content is rapidly extracted by adopting a video slicing method, the video slicing method is to extract partial pixels from each frame of image and combine the partial pixels into an image which has a certain rule and can represent short video content, and the formula is as follows:
wherein x is a slice image, and I is a middle line of pixels of a video frame;
forming a new image by a video slicing method, extracting n 256 new images, wherein the size of n is in direct proportion to the total frame number of the video, and detecting scene switching in the video image through a video slice;
103) and (3) classification step: using self-developed neural networks as a classification method, i.e. convolutional layers LcAn active layer LrA pooling layer LpThe cascade combination is formed by adding an inner layer, so that the classification task can be quickly and effectively carried out, and the formula is expressed as follows;
f＝Lp(Lr(Lc(x)))
arg min|F-M·g|
wherein F is a neural network single-stage characteristic, g is a joint characteristic, M is an inner product matrix, and F is a classification probability;
104) a category judgment step: and (5) counting the labels of the n new images, and finding the classification result with the largest number as the classification label of the whole video.
Furthermore, the cut shot switching is judged according to the position of the crack, and the position of the fuzzy filtering is the gradual shot switching.
Further, a framework of the video type classification method is deployed at a server; and uploading the video by the client in an http request mode, and returning an obtained result after the server classifies.
Compared with the prior art, the invention has the advantages that:
compared with a classification method which adopts original frame images (sampling or full frames) or manual work, the method has the advantages that the performance is improved by at least 100 times; the video slice method effectively abstracts the characteristics of the video, reduces redundant information, and has the effect of completely replacing manual classification at present by the powerful classification capability of a self-developed network.
Drawings
FIG. 1 is a diagram of a video slice extracting partial content of a frame according to the present invention;
FIG. 2 is a new image of the present invention;
FIG. 3 is a schematic diagram of the classification process of the present invention;
FIG. 4 is a flow chart of video classification according to the present invention.
Detailed Description
The invention is further described with reference to the following figures and detailed description.
As shown in fig. 1 to 4, a video slice-based video type classification method includes the following steps:
101) setting and classifying: according to the application scene, the classification labels are designed, and the classification labels comprise labels of comedy, science fiction, cartoon, pornography, riot, and the like.
102) Extracting slice images: for continuous images in a video, a middle row of pixels in each frame of image is taken as a column to form a new image, and the new image can reflect the characteristics of some videos, such as shot switching of the videos, video content motion modes, content styles and the like. As shown in fig. 2, the break position of the shot-to-shot new image of the video is cut shot switching, and the position of the blur filter is gradual shot switching. The video content moves in a manner that the content is still as a static shot (as shown in the middle of the first drawing from left to right in fig. 2), and the content changes continuously (as shown in the middle of the second drawing from left to right in fig. 2, the shot moves upward). The content styles, as shown in fig. 2, the first and second images from left to right are animation videos (adventure type), the second two are drama videos (suspense type), and it can be clearly seen that the first two images are gorgeous in color, and the second two images are dull in color.
The video content is extracted quickly by adopting a video slicing method, namely, part of pixels are extracted from each frame of image and combined into an image which has a certain rule and can represent short video content, the formula is as follows, wherein x is a slice image, I is a line of pixels in the middle of a video frame, and the operation is splicing operation;
the method comprises the steps of obtaining n 256 new images after extraction through a video slicing method, wherein the size of n is in direct proportion to the total frame number of video, and detecting scene switching in a video image through video slice. For example, if there are 25600 frames in a video, 256 slice images are obtained, and the above feature detection is performed on each image.
103) And (3) classification step: using neural networks as classification methods, i.e. convolutional layers LcAn active layer LrA pooling layer LpThe three are combined in a cascade mode and formed by an inner laminated layer, classification tasks can be rapidly and effectively carried out according to feature detection, and a formula is expressed as follows;
f＝Lp(Lr(Lc(x)))
argmin|F-M·g|
f is the single-stage characteristic of the neural network, g is the joint characteristic, M is the inner product matrix, and F is the classification probability.
I.e. passing through the multi-stage convolutional layer L for each new imagecAn active layer LrA pooling layer LpThe three are combined in a cascade way, and the obtained data is subjected to an inner lamination layer to obtain a classification result; compared with a full convolution network (such as cascade cnn), the network mainly introduces a pooling layer and simultaneously comprises a very ingenious structure, and the final characteristics comprise the statistical values of all convolution layers, so that the front-back correlation is well kept, and under the condition of the same parameter number as the full convolution network, the calculated amount is smaller, and the representation capability is stronger. The convolutional layer, the activation layer and the pooling layer are all basic components of a common deep neural network.
104) A category judgment step: and (5) counting the labels of the n new images, and finding the classification result with the largest number as the classification label of the whole video.
Finally, deploying the framework of the video type classification method at a server; and the client uploads the video in an http request mode, and the server returns the obtained result after classification, so that complete application is realized.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and decorations can be made without departing from the spirit of the present invention, and these modifications and decorations should also be regarded as being within the scope of the present invention.
Claims (3)
1. A video slice-based video type classification method is characterized by comprising the following steps:
101) setting and classifying: designing classification labels according to application scenes, wherein the classification labels comprise comedy, science fiction, cartoon, pornography and violence and terrorism;
102) extracting slice images: the video content is rapidly extracted by adopting a video slicing method, the video slicing method is to extract partial pixels from each frame of image and combine the partial pixels into an image which has a certain rule and can represent short video content, and the formula is as follows:
wherein x is a slice image, and I is a middle line of pixels of a video frame;
forming a new image by a video slicing method, extracting n 256 new images, wherein the size of n is in direct proportion to the total frame number of the video, and detecting scene switching in the video image through a video slice;
103) and (3) classification step: using self-developed neural networks as a classification method, i.e. convolutional layers LcAn active layer LrA pooling layer LpThe cascade combination is formed by adding an inner layer, so that the classification task can be quickly and effectively carried out, and the formula is expressed as follows;
f＝Lp(Lr(Lc(x)))
arg min|F-M·g|
wherein F is a neural network single-stage characteristic, g is a joint characteristic, M is an inner product matrix, and F is a classification probability;
104) a category judgment step: and (5) counting the labels of the n new images, and finding the classification result with the largest number as the classification label of the whole video.
2. The video slice-based video type classification method according to claim 1, wherein the position of the crack is determined as cut shot cut, and the position of the blur filtering is gradual shot cut.
3. The video slice-based video type classification method according to claim 1, wherein a framework of the video type classification method is deployed at a server; and uploading the video by the client in an http request mode, and returning an obtained result after the server classifies.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911015725.7A CN110852195A (en) | 2019-10-24 | 2019-10-24 | Video slice-based video type classification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911015725.7A CN110852195A (en) | 2019-10-24 | 2019-10-24 | Video slice-based video type classification method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110852195A true CN110852195A (en) | 2020-02-28 |
Family
ID=69596990
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911015725.7A Pending CN110852195A (en) | 2019-10-24 | 2019-10-24 | Video slice-based video type classification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110852195A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111753790A (en) * | 2020-07-01 | 2020-10-09 | 武汉楚精灵医疗科技有限公司 | Video classification method based on random forest algorithm |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107330392A (en) * | 2017-06-26 | 2017-11-07 | 司马大大（北京）智能系统有限公司 | Video scene annotation equipment and method |
| CN107679462A (en) * | 2017-09-13 | 2018-02-09 | 哈尔滨工业大学深圳研究生院 | A kind of depth multiple features fusion sorting technique based on small echo |
| CN108594997A (en) * | 2018-04-16 | 2018-09-28 | 腾讯科技（深圳）有限公司 | Gesture framework construction method, apparatus, equipment and storage medium |
| CN109948721A (en) * | 2019-03-27 | 2019-06-28 | 北京邮电大学 | A kind of video scene classification method based on video presentation |
| CN110070067A (en) * | 2019-04-29 | 2019-07-30 | 北京金山云网络技术有限公司 | The training method of video classification methods and its model, device and electronic equipment |
-
2019
- 2019-10-24 CN CN201911015725.7A patent/CN110852195A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107330392A (en) * | 2017-06-26 | 2017-11-07 | 司马大大（北京）智能系统有限公司 | Video scene annotation equipment and method |
| CN107679462A (en) * | 2017-09-13 | 2018-02-09 | 哈尔滨工业大学深圳研究生院 | A kind of depth multiple features fusion sorting technique based on small echo |
| CN108594997A (en) * | 2018-04-16 | 2018-09-28 | 腾讯科技（深圳）有限公司 | Gesture framework construction method, apparatus, equipment and storage medium |
| CN109948721A (en) * | 2019-03-27 | 2019-06-28 | 北京邮电大学 | A kind of video scene classification method based on video presentation |
| CN110070067A (en) * | 2019-04-29 | 2019-07-30 | 北京金山云网络技术有限公司 | The training method of video classification methods and its model, device and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| ZEESHAN RASHEED ET AL；: "《On the Use of Computable Features for Film Classification》", 《IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111753790A (en) * | 2020-07-01 | 2020-10-09 | 武汉楚精灵医疗科技有限公司 | Video classification method based on random forest algorithm |
| CN111753790B (en) * | 2020-07-01 | 2023-12-12 | 武汉楚精灵医疗科技有限公司 | Video classification method based on random forest algorithm |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10779037B2 (en) | Method and system for identifying relevant media content | |
| CN102819528B (en) | The method and apparatus generating video frequency abstract | |
| Smeaton et al. | Video shot boundary detection: Seven years of TRECVid activity | |
| CN110309795B (en) | Video detection method, device, electronic equipment and storage medium | |
| Mussel Cirne et al. | VISCOM: A robust video summarization approach using color co-occurrence matrices | |
| CN111489372A (en) | Video foreground and background separation method based on cascade convolution neural network | |
| CN111598026A (en) | Action recognition method, device, equipment and storage medium | |
| CN112150450B (en) | Image tampering detection method and device based on dual-channel U-Net model | |
| CN109033476B (en) | Intelligent spatio-temporal data event analysis method based on event cue network | |
| CN112580523A (en) | Behavior recognition method, behavior recognition device, behavior recognition equipment and storage medium | |
| CN111222450A (en) | Model training method, model training device, model live broadcast processing equipment and storage medium | |
| Verde et al. | Video codec forensics based on convolutional neural networks | |
| CN112163488A (en) | Video false face detection method and electronic device | |
| CN110852195A (en) | Video slice-based video type classification method | |
| CN107301245B (en) | Power information video search system | |
| CN114996227A (en) | Monitoring video compression and restoration method | |
| US11830286B2 (en) | Data processing apparatus, data processing method, and non-transitory storage medium | |
| EP2345978B1 (en) | Detection of flash illuminated scenes in video clips and related ranking of video clips | |
| Qiu et al. | Dual focus attention network for video emotion recognition | |
| CN115410131A (en) | Method for intelligently classifying short videos | |
| O’Byrne et al. | Impact of video compression on the performance of object detection systems for surveillance applications | |
| CN116189027A (en) | Faster R-cnn context mechanism optimization method based on multilayer feature fusion | |
| Roka et al. | Deep stacked denoising autoencoder for unsupervised anomaly detection in video surveillance | |
| Andreadou et al. | Media REVEALr: A social multimedia monitoring and intelligence system for web multimedia verification | |
| WO2002095662A1 (en) | Method for detecting text zones in a video image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200228 |