CN111797912B - System and method for identifying film age type and construction method of identification model - Google Patents

System and method for identifying film age type and construction method of identification model Download PDF

Info

Publication number
CN111797912B
CN111797912B CN202010580262.5A CN202010580262A CN111797912B CN 111797912 B CN111797912 B CN 111797912B CN 202010580262 A CN202010580262 A CN 202010580262A CN 111797912 B CN111797912 B CN 111797912B
Authority
CN
China
Prior art keywords
film
type
picture
pictures
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010580262.5A
Other languages
Chinese (zh)
Other versions
CN111797912A (en
Inventor
杨唤晨
徐杰
谢恩鹏
刘永辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Inspur Ultra HD Video Industry Co Ltd
Original Assignee
Shandong Inspur Ultra HD Video Industry Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Inspur Ultra HD Video Industry Co Ltd filed Critical Shandong Inspur Ultra HD Video Industry Co Ltd
Priority to CN202010580262.5A priority Critical patent/CN111797912B/en
Publication of CN111797912A publication Critical patent/CN111797912A/en
Application granted granted Critical
Publication of CN111797912B publication Critical patent/CN111797912B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/32Normalisation of the pattern dimensions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/56Extraction of image or video features relating to colour
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Multimedia (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a system and a method for identifying film age types and a method for constructing an identification model. A system for film year type identification, comprising: a trained film age type identification model; the computing device, the storage device and the input-output device constitute a computer device for automatic identification processes; a convolutional neural network system formed by decoding a digital film, capturing a picture and preprocessing the picture; and a classifier capable of analyzing the film age type according to the model reasoning result. With the system and method, a computing device may automatically identify the chronological type of a movie after obtaining a digital movie from a storage device. Compared with manual identification, the identification by using the computing equipment has the advantages of reliability, rapidness, cheapness, batch processing and the like.

Description

System and method for identifying film age type and construction method of identification model
Technical Field
The invention relates to a system and a method for identifying film age types and a method for constructing an identification model, and belongs to the technical field of information technology.
Background
In recent years, video services have rapidly developed, and activities such as chasing dramas and watching digital televisions become an important entertainment mode for people. Video providers often store a large number of movies, and how to automatically, quickly, reliably, and batchwise label these movies with ages, titles, genres, etc. is of great importance in helping the providers provide good quality video services.
Disclosure of Invention
The invention aims to provide a system and a method for identifying film age types and a method for constructing an identification model.
The invention aims to achieve the aim, and the aim is achieved by the following technical scheme:
a system for film year type identification, comprising:
a trained film age type identification model;
the computing device, the storage device and the input-output device constitute a computer device for automatic identification processes;
a convolutional neural network system formed by decoding a digital film, capturing a picture and preprocessing the picture;
and a classifier capable of analyzing the film age type according to the model reasoning result.
Based on the film age type identification system, the convolutional neural network system is of a VGG-16 network structure.
A film age type identification method by the system comprises the following steps:
the computing equipment reads a film to be identified from the storage equipment, and cuts off the first a minute and the last b minute of the film, and then intercepts M pictures at uniform time intervals;
after preprocessing M pictures, respectively inputting the M pictures into a convolutional neural network;
for each picture input, the convolutional neural network can infer an N-dimensional probability vector, and each dimension corresponds to the probability that the picture belongs to the corresponding age type;
after all reasoning of the M pictures is completed, the system for identifying the film age type inputs M probability vectors into the classifier.
The method for constructing the film age type identification model is characterized by comprising the following steps of:
s1, constructing a training set and a verification set: preparing a large number of digital films belonging to N types of years, and uniformly intercepting pictures from each film time interval as training data and verification data of the type;
s2, building a convolutional neural network: using a deep convolutional network for image classification tasks as the main body of the network;
s3, training of a convolutional neural network: freezing parameters of a convolution base, training a classifier, wherein a first layer of the classifier uses Dropout; after the classifier reaches a better accuracy through multiple rounds of training, the bottom layer of the convolution base is thawed, fine tuning training is carried out, and the accuracy of the verification network on the verification set is verified.
According to the method for constructing the film age type identification model, all training pictures and verification pictures are preprocessed, and the method comprises the following steps: 1) Scaling to an input size required by the neural network; 2) Subtracting the average RGB value of the whole picture dataset from the pixels of each picture; 3) The pixel values of RGB are divided by 255 such that the RGB values are between 0 and 1.
6. A method of constructing a film year type identification model as claimed in claim 3, wherein: the classifier analyzes the chronology type of the film according to the following algorithm: any playing time t of the video stream corresponds to a frameTaking the frame as the input of the model, an N-dimensional probability vector can be obtained>Each dimension of the vector corresponds to a chronology type respectively;
introduction evaluationDividing a function into
Integrating the scoring functions of all playing moments to obtain total scoresThe total score is an N-dimensional vector whose dimensions are defined as +.>And each dimension corresponds to one year type;
and taking the age type with the largest total score as the classification result of the video.
The invention has the advantages that: by the system and the method, the computing device can automatically identify the age type of the film after acquiring the digital film from the storage device, and compared with manual identification, the method has the advantages of reliability, rapidness, low cost, batch processing and the like.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention.
Fig. 1 is a schematic flow chart of a method for constructing a film age type recognition model according to the present invention.
Fig. 2 is a flowchart of a film age type identification method according to the present invention.
Fig. 3 is a schematic diagram of a connection of the film age type identification system according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
A system for chronology type recognition, comprising: a trained film age type identification model; the computing device, the storage device and the input-output device constitute a computer device for automatic identification processes; a convolutional neural network system formed by decoding a digital film, capturing a picture and preprocessing the picture; and a classifier capable of analyzing the film age type according to the model reasoning result.
A method for identifying the film age type by the system, which marks the film to be identified with an age type label, wherein the actual age type of the film to be identified must be one of N preset age types, which can be ancient dress, republic, contemporary and the like, includes the following steps:
the computing equipment reads a film to be identified from the storage equipment, and cuts off the first a minute and the last b minute of the film, and then intercepts M pictures at uniform time intervals;
after preprocessing M pictures, respectively inputting the M pictures into a convolutional neural network;
picture preprocessing includes, but is not limited to, subtracting the average RGB value of all pixels from the RGB value of each pixel of the picture, multiplying the RGB values of all pixels by 1/255 so that the values fall within the interval of 0-1, scaling the picture to a certain fixed size, and so on;
for each picture input, the convolutional neural network can infer an N-dimensional probability vector, and each dimension corresponds to the probability that the picture belongs to the corresponding age type;
after all reasoning of the M pictures is completed, the system for identifying the film age type inputs M probability vectors into the classifier.
The method for constructing the film age type identification model comprises the following steps:
s1, constructing a training set and a verification set: for each type of year, S digital films are prepared, and S is large enough; for each film, firstly pinching off the first a minutes and the last b minutes to ensure that the film head and the film tail are pinched off; intercepting T pictures at equal time intervals as the data of the type; thus, each type can obtain S multiplied by T pictures, and the S multiplied by T pictures are distributed to a training set and a verification set according to a certain proportion;
s2, building a convolutional neural network: using a deep convolutional network for image classification tasks as the main body of the network, for example, a VGG16 network, initializing the network by using parameters of the VGG16 network trained on a large picture data set, for example, downloading the trained VGG16 network by using Keras;
s3, training of a convolutional neural network: freezing parameters of a convolution base, training a classifier, wherein a first layer of the classifier uses Dropout for reducing the problem of overfitting; after the classifier reaches a better accuracy through multiple rounds of training, the bottom layer of the convolution base is thawed, fine tuning training is carried out, and the accuracy of the verification network on the verification set is verified.
All training pictures and verification pictures should be preprocessed, including the following steps: 1) Scaling to an input size required by the neural network; 2) Subtracting the average RGB value of the whole picture dataset from the pixels of each picture; 3) The pixel values of RGB are divided by 255 such that the RGB values are between 0 and 1.
The classifier analyzes the chronology type of the film according to the following algorithm: introducing an N-dimensional probability vectorFunction of->Wherein->Hereafter will be->Called scoring function;
for a video stream S with a duration T, each playing time T corresponds to a frameF is input into the network to obtain a probability vector +.>The scoring function is a unitary function of the probability vector, and as known from the chain rule, the scoring function is also a unitary function r=r (t) at the playing time t;
the scores of all moments of the video stream S are integrated to obtain an overall scoreThe function is an N-dimensional vector, and
thus, the first and second light sources are connected,i.e. film type, wherein->The operation finds the subscript of the vector.
For digital video, the playing time is discrete, in other words, a digital film with a frame rate of 25fps contains 25 frames per second, instead of a myriad of frames. In this case, the integral expression degenerates into a sum expression, i.e
T on the left of the equal sign is a constant and has no effect on the classification result, and for convenience the expression of the total scoring function can be redefined as
The movie is tagged with the corresponding year type if the value of that dimension is large.
Finally, it should be noted that: the foregoing description is only a preferred embodiment of the present invention, and the present invention is not limited thereto, but it is to be understood that modifications and equivalents of some of the technical features described in the foregoing embodiments may be made by those skilled in the art, although the present invention has been described in detail with reference to the foregoing embodiments. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (3)

1. A film year type identification method comprising a film year type identification system, said system comprising:
the computing device, the storage device and the input-output device constitute a computer device for automatic identification processes;
a convolutional neural network system formed by a digital film decoding, screenshot, picture preprocessing and trained film age type recognition model; the recognition model is a VGG-16 network structure;
the classifier can analyze the film age type according to the model reasoning result;
the method comprises the following steps:
the computing equipment reads a film to be identified from the storage equipment, and cuts off the first a minute and the last b minute of the film, and then intercepts M pictures at uniform time intervals;
after preprocessing M pictures, respectively inputting the M pictures into a convolutional neural network;
for each picture input, the convolutional neural network can infer an N-dimensional probability vector, and each dimension corresponds to the probability that the picture belongs to the corresponding age type;
after all reasoning of the M pictures is completed, the system for identifying the film age type inputs M probability vectors into a classifier;
the classifier analyzes the chronology type of the film according to the following algorithm: any playing time t of the video stream corresponds to a frameTaking the frame as the input of the model, an N-dimensional probability vector can be obtained>Each dimension of the vector corresponds to a chronology type respectively, and a scoring function is introduced>Wherein->Integrating the scoring functions of all playing moments to obtain total score +.>The total score is an N-dimensional vector whose dimensions are defined as +.>
And each dimension corresponds to one age type, and the age type with the largest total score is taken as the classification result of the video.
2. The film year type identification method as set forth in claim 1, comprising the steps of:
s1, constructing a training set and a verification set: preparing a large number of digital films belonging to N types of years, and uniformly intercepting pictures from each film time interval as training data and verification data of the type;
s2, building a convolutional neural network: using a deep convolutional network for image classification tasks as the main body of the network;
s3, training a convolutional neural network: freezing parameters of a convolution base, training a classifier, wherein a first layer of the classifier uses Dropout; after the classifier reaches a better accuracy through multiple rounds of training, the bottom layer of the convolution base is thawed, fine tuning training is carried out, and the accuracy of the verification network on the verification set is verified.
3. The film year type identification method according to claim 2, wherein: the preprocessing of the M pictures comprises the following steps: 1) Scaling to an input size required by the neural network; 2) Subtracting the average RGB value of the whole picture dataset from the pixels of each picture; 3) The pixel values of RGB are divided by 255 such that the RGB values are between 0 and 1.
CN202010580262.5A 2020-06-23 2020-06-23 System and method for identifying film age type and construction method of identification model Active CN111797912B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010580262.5A CN111797912B (en) 2020-06-23 2020-06-23 System and method for identifying film age type and construction method of identification model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010580262.5A CN111797912B (en) 2020-06-23 2020-06-23 System and method for identifying film age type and construction method of identification model

Publications (2)

Publication Number Publication Date
CN111797912A CN111797912A (en) 2020-10-20
CN111797912B true CN111797912B (en) 2023-09-22

Family

ID=72803677

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010580262.5A Active CN111797912B (en) 2020-06-23 2020-06-23 System and method for identifying film age type and construction method of identification model

Country Status (1)

Country Link
CN (1) CN111797912B (en)

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103229223A (en) * 2010-09-28 2013-07-31 国际商业机器公司 Providing answers to questions using multiple models to score candidate answers
CN105844251A (en) * 2016-03-31 2016-08-10 乐视控股(北京)有限公司 Cartoon video identification method and device
CN107067020A (en) * 2016-12-30 2017-08-18 腾讯科技(上海)有限公司 Image identification method and device
CN107194419A (en) * 2017-05-10 2017-09-22 百度在线网络技术(北京)有限公司 Video classification methods and device, computer equipment and computer-readable recording medium
CN108154134A (en) * 2018-01-11 2018-06-12 天格科技(杭州)有限公司 Internet live streaming pornographic image detection method based on depth convolutional neural networks
CN109753984A (en) * 2017-11-07 2019-05-14 北京京东尚科信息技术有限公司 Video classification methods, device and computer readable storage medium
CN109919031A (en) * 2019-01-31 2019-06-21 厦门大学 A kind of Human bodys' response method based on deep neural network
CN110147700A (en) * 2018-05-18 2019-08-20 腾讯科技(深圳)有限公司 Video classification methods, device, storage medium and equipment
CN110766096A (en) * 2019-10-31 2020-02-07 北京金山云网络技术有限公司 Video classification method and device and electronic equipment
CN110866593A (en) * 2019-11-05 2020-03-06 西南交通大学 Highway severe weather identification method based on artificial intelligence
CN110991246A (en) * 2019-10-31 2020-04-10 天津市国瑞数码安全系统股份有限公司 Video detection method and system

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170053024A1 (en) * 2014-04-28 2017-02-23 Hewlett Packard Enterprise Development Lp Term chain clustering
US10402697B2 (en) * 2016-08-01 2019-09-03 Nvidia Corporation Fusing multilayer and multimodal deep neural networks for video classification

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103229223A (en) * 2010-09-28 2013-07-31 国际商业机器公司 Providing answers to questions using multiple models to score candidate answers
CN105844251A (en) * 2016-03-31 2016-08-10 乐视控股(北京)有限公司 Cartoon video identification method and device
CN107067020A (en) * 2016-12-30 2017-08-18 腾讯科技(上海)有限公司 Image identification method and device
CN107194419A (en) * 2017-05-10 2017-09-22 百度在线网络技术(北京)有限公司 Video classification methods and device, computer equipment and computer-readable recording medium
CN109753984A (en) * 2017-11-07 2019-05-14 北京京东尚科信息技术有限公司 Video classification methods, device and computer readable storage medium
CN108154134A (en) * 2018-01-11 2018-06-12 天格科技(杭州)有限公司 Internet live streaming pornographic image detection method based on depth convolutional neural networks
CN110147700A (en) * 2018-05-18 2019-08-20 腾讯科技(深圳)有限公司 Video classification methods, device, storage medium and equipment
CN109919031A (en) * 2019-01-31 2019-06-21 厦门大学 A kind of Human bodys' response method based on deep neural network
CN110766096A (en) * 2019-10-31 2020-02-07 北京金山云网络技术有限公司 Video classification method and device and electronic equipment
CN110991246A (en) * 2019-10-31 2020-04-10 天津市国瑞数码安全系统股份有限公司 Video detection method and system
CN110866593A (en) * 2019-11-05 2020-03-06 西南交通大学 Highway severe weather identification method based on artificial intelligence

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
Video classification and recommendation based on affective analysis of viewers;Sicheng Zhao等;《Neurocomputing》;第119卷;第101–110页 *
基于多分类器集成的模式识别研究;潘翔;《中国优秀硕士学位论文全文数据库 信息科技辑》(第(2002)02期);I138-526 *
基于智能机顶盒视频点播系统的设计与实现;朱宁;《中国优秀硕士学位论文全文数据库 信息科技辑》(第(2015)07期);I136-439 *
基于视频流行度及视频分类标签的Top-N推荐;周瑞环;《中国优秀硕士学位论文全文数据库 信息科技辑》(第(2018)10期);I138-1031 *

Also Published As

Publication number Publication date
CN111797912A (en) 2020-10-20

Similar Documents

Publication Publication Date Title
CN109145784B (en) Method and apparatus for processing video
US10304458B1 (en) Systems and methods for transcribing videos using speaker identification
US9436876B1 (en) Video segmentation techniques
US10939165B2 (en) Facilitating television based interaction with social networking tools
US20090290791A1 (en) Automatic tracking of people and bodies in video
CN112954450B (en) Video processing method and device, electronic equipment and storage medium
Ul Haq et al. Personalized Movie Summarization Using Deep CNN‐Assisted Facial Expression Recognition
US20180005077A1 (en) Automated selection of subjectively best image frames from burst captured image sequences
WO2019007020A1 (en) Method and device for generating video summary
US20230353828A1 (en) Model-based data processing method and apparatus
CN102236796A (en) Method and system for sorting defective contents of digital video
CN109408672B (en) Article generation method, article generation device, server and storage medium
US20210225005A1 (en) Selection of Video Frames Using a Machine Learning Predictor
US10897658B1 (en) Techniques for annotating media content
CN109348287A (en) Video abstraction generating method, device, storage medium and electronic equipment
CN116977774A (en) Image generation method, device, equipment and medium
CN111797912B (en) System and method for identifying film age type and construction method of identification model
CN114064968A (en) News subtitle abstract generating method and system
CN113468925B (en) Occlusion face recognition method, intelligent terminal and storage medium
US20210224571A1 (en) Automated Cropping of Images Using a Machine Learning Predictor
US20140307968A1 (en) Method and apparatus for automatic genre identification and classification
US20230066331A1 (en) Method and system for automatically capturing and processing an image of a user
CN113705666B (en) Split network training method, use method, device, equipment and storage medium
CN114743170A (en) Automatic driving scene labeling method based on AI algorithm
US20210327071A1 (en) Automated Cropping of Images Using a Machine Learning Predictor

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 250100 West District, North First Floor, S06 Building, 1036 Langchao Road, Jinan High-tech Zone, Shandong Province

Applicant after: Shandong Inspur Ultra HD Video Industry Co.,Ltd.

Address before: 250101 west area, North 1st floor, S06 building, 1036 Chaochao Road, high tech Zone, Jinan City, Shandong Province

Applicant before: SHANDONG YUNMAN INTELLIGENT TECHNOLOGY Co.,Ltd.

CB02 Change of applicant information
GR01 Patent grant
GR01 Patent grant