CN113268631B - Video screening method and device based on big data - Google Patents

Video screening method and device based on big data Download PDF

Info

Publication number
CN113268631B
CN113268631B CN202110428964.6A CN202110428964A CN113268631B CN 113268631 B CN113268631 B CN 113268631B CN 202110428964 A CN202110428964 A CN 202110428964A CN 113268631 B CN113268631 B CN 113268631B
Authority
CN
China
Prior art keywords
video
initial
user
feature vector
screening
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110428964.6A
Other languages
Chinese (zh)
Other versions
CN113268631A (en
Inventor
干少明
张怀
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Dianzhong Kuaikan Technology Co ltd
Original Assignee
Beijing Dianzhong Kuaikan Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Dianzhong Kuaikan Technology Co ltd filed Critical Beijing Dianzhong Kuaikan Technology Co ltd
Priority to CN202110428964.6A priority Critical patent/CN113268631B/en
Publication of CN113268631A publication Critical patent/CN113268631A/en
Application granted granted Critical
Publication of CN113268631B publication Critical patent/CN113268631B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/73Querying
    • G06F16/732Query formulation
    • G06F16/7328Query by example, e.g. a complete video frame or video sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/75Clustering; Classification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods

Abstract

The invention relates to a video screening method and a device based on big data, which acquire an initial video set related to user information according to user information, acquire a first feature vector of the initial video according to a screening model for any one of the initial videos, acquire a first loss prediction result of the first feature vector according to a loss prediction model, screen a first intermediate video from the initial video set according to the first loss prediction result corresponding to each initial video, screen a second intermediate video from the first intermediate video by combining the first intermediate video and the user feature data, so that the accuracy of the initial video set is greatly improved compared with the existing video screening method by performing two-layer screening on the initial video set according to different screening rules.

Description

Video screening method and device based on big data
Technical Field
The invention relates to a video screening method and device based on big data.
Background
With the gradual maturity of big data application, when playing video through video APP, the background server can screen a large amount of videos stored in the background server according to relevant data of users, and the videos possibly interested by the users are screened out and recommended to the users. However, the current video filtering method usually filters according to the video field of interest filled in by the user during registration or according to the history record played by the user, which has low accuracy, and may filter out some irrelevant videos or leak closely related videos.
Disclosure of Invention
In order to solve the problem of low accuracy of the existing video screening method, the invention provides a video screening method and device based on big data.
A big data based video screening method comprising:
Acquiring an initial video set related to user information according to the user information, wherein the initial video set comprises at least two initial videos;
For any one initial video in the initial video set, acquiring a first feature vector of the initial video according to a preset screening model;
acquiring a first loss prediction result of the first feature vector according to a preset loss prediction model;
screening a first intermediate video from the initial video set according to a first loss prediction result corresponding to each initial video;
Acquiring user characteristic data of the user according to the user information;
And screening a second intermediate video from the first intermediate video by combining the first intermediate video with the user characteristic data.
Further, the user information includes a historical video play record of the user;
the method comprises the steps of obtaining an initial video set related to user information according to the user information, wherein the initial video set is specifically:
and acquiring videos in the same field and related fields as each video recorded in the historical video play record according to the historical video play record, wherein the acquired videos form the initial video set.
Further, the screening model includes at least two convolution layers, each capable of outputting a first feature vector;
The method comprises the steps of obtaining a first feature vector of the initial video according to a preset screening model, wherein the first feature vector is specifically:
obtaining a first feature vector of the initial video according to a convolution layer in the screening model;
The loss prediction model comprises at least two loss prediction sub-models and a classifier, wherein each loss prediction sub-model corresponds to each convolution layer one by one, and the input of each loss prediction sub-model is a first feature vector output by the corresponding convolution layer;
the method comprises the steps of obtaining a first loss prediction result of the first feature vector according to a preset loss prediction model, wherein the first loss prediction result is specifically:
For any one first feature vector, inputting the first feature vector into a loss predictor model corresponding to the first feature vector, and obtaining a first vector output by the loss predictor model aiming at the first feature vector;
integrating the obtained first vectors to obtain second vectors;
And obtaining the first loss prediction result according to the second vector and the classifier.
Further, the first loss prediction result corresponding to each initial video comprises a prediction loss value corresponding to each initial video;
The first intermediate video is screened from the initial video set according to the first loss prediction result corresponding to each initial video, specifically:
And comparing the predicted loss value corresponding to each initial video with a preset loss threshold value, and acquiring the initial video corresponding to the predicted loss value which is larger than or equal to the preset loss threshold value according to the comparison result to obtain the first intermediate video.
Further, the acquiring the user characteristic data of the user according to the user information specifically includes:
acquiring possible interesting video fields of the user according to the user information, wherein the possible interesting video fields comprise video fields of the user and associated video fields associated with the video fields of the user;
The step of screening a second intermediate video from the first intermediate video by combining the first intermediate video with the user characteristic data specifically includes:
And acquiring the video in the possible interesting video field in the first intermediate video according to the possible interesting video field, and obtaining the second intermediate video.
Further, the acquiring process of the associated video field specifically includes:
acquiring the association degree of the video domain concerned by the user and other video domains in the video domain knowledge graph according to a preset video domain knowledge graph;
And comparing each association degree with a preset association degree threshold value, acquiring a target association degree which is larger than or equal to the preset association degree threshold value according to a comparison result, and acquiring a video field corresponding to the target association degree to acquire the associated video field.
A big data based video screening device comprising a memory and a processor, and a computer program stored on the memory and running on the processor, the processor implementing the big data based video screening method as described above when executing the computer program.
The beneficial effects of the invention are as follows: according to user information, an initial video set related to the user information is obtained, then, according to a preset screening model and a loss prediction model, feature vectors are obtained, loss prediction results are obtained, according to first loss prediction results corresponding to all initial videos in the initial video set, first intermediate videos are screened out from the initial video set, then, user feature data of a user are obtained according to the user information, and finally, the first intermediate videos and the user feature data are combined, and second intermediate videos are screened out from the first intermediate videos. Therefore, the video screening method based on big data provided by the invention firstly acquires the initial video set, then combines the preset screening model and the loss prediction model, screens the first intermediate video from the initial video set, screens the second intermediate video from the first intermediate video according to the user characteristic data, and sequentially carries out two layers of screening on the initial video set according to different screening rules.
Drawings
Fig. 1 is a flow chart of a method of video screening based on big data.
Detailed Description
The embodiment provides a video screening method based on big data, which can be used for a computer or intelligent terminal equipment.
As shown in fig. 1, the video filtering method includes the following steps:
Step S1: according to user information, acquiring an initial video set related to the user information, wherein the initial video set comprises at least two initial videos:
and acquiring an initial video set related to the user information according to the user information, wherein the initial video set comprises at least two initial videos. It should be appreciated that the initial video set typically includes a large number of initial videos to be screened. The initial video set may be obtained from a corresponding background server.
The user information is specifically set according to actual needs, and in this embodiment, the user information includes a history video playing record of the user, and may also include personal information of the user, such as a focus video field filled in by the user during registration. It should be understood that, after the user logs in to the corresponding account, each time a video is played, the background will record the play record of the video, thus forming a historical video play record. Moreover, the length of the time period corresponding to the historical video playing record is set according to actual needs, for example: half a year or one year.
Then, according to the historical video play record, the videos in the same field and the related fields as the videos recorded in the historical video play record are acquired, and the acquired videos form an initial video set. Specifically: according to the historical video play records, the fields of all videos recorded in the historical video play records are acquired, then the fields related to all the video fields are acquired, and the related fields can be acquired according to a preset field relation database in a background server, wherein the field relation database comprises the relation among all the fields in all the currently known video fields. Then, as long as the fields related to the respective video fields recorded in the history video play record are the related fields.
Then, videos in the same field and related fields as each video recorded in the history video play record are acquired from the background server, and the acquired videos form an initial video set.
As other implementations, the initial video set may directly include all videos stored in the background server.
Step S2: for any one initial video in the initial video set, acquiring a first feature vector of the initial video according to a preset screening model:
A screening model is preset and is used for obtaining a corresponding first feature vector according to the initial video, so that the screening model is a network model for extracting the feature vector of the video, can be constructed according to actual needs, and can also be directly used for an existing network model with a feature vector extraction function. The specific type of screening model is set by the actual need, such as a convolutional neural network model.
The screening model may include only one convolution layer, or may set at least two convolution layers according to actual needs, in this embodiment, in order to improve accuracy of feature vectors and subsequent data processing, the screening model includes at least two convolution layers, and each convolution layer can output a first feature vector. In the present embodiment, each of the first feature vectors obtained from each of the convolution layers may be regarded as a feature vector in which the feature extraction depth gradually increases.
Since the data processing procedure of any one of the initial videos in the initial video set is the same, the following description will take any one of the initial videos in the initial video set as an example. And respectively obtaining the first eigenvectors of the initial video according to each convolution layer in the screening model.
Step S3: according to a preset loss prediction model, a first loss prediction result of the first feature vector is obtained:
The loss prediction model is preset and is used for obtaining a first loss prediction result according to a first feature vector obtained by the screening model, so that the loss prediction model is a network model for obtaining the loss prediction result according to the feature vector, can be constructed according to actual needs, and can also be a network model with a loss prediction function directly used. The specific type of the loss prediction model is set according to actual needs, such as a deep learning neural network, a convolution neural network and the like, and can comprise a pooling layer, a full connection layer and a nonlinear layer, wherein the number and the specific structure of each layer are not limited.
Because the screening model comprises at least two convolution layers, each convolution layer can output a first feature vector, and correspondingly, the loss prediction model comprises at least two loss prediction sub-models and a classifier, the loss prediction sub-models are in one-to-one correspondence with the convolution layers, and the input of each loss prediction sub-model is the first feature vector output by the corresponding convolution layer.
And as the screening model obtains the first feature vectors with the same number as the convolution layer, for any one first feature vector, inputting the first feature vector into a loss predictor model corresponding to the first feature vector, and obtaining a first vector output by the loss predictor model aiming at the first feature vector.
Then, the obtained first vectors are integrated to obtain a second vector, for example: and splicing the first vectors to obtain a second vector, or carrying out average calculation on elements at the same position in the first vectors to obtain the second vector.
And finally, obtaining a first loss prediction result of the initial video according to the obtained second vector and the classifier.
Through the above process, the first loss prediction result of other initial videos is obtained.
In this embodiment, the first feature vectors of the features with different depths are combined to obtain the first loss prediction result, so that the one-sided problem caused by a single feature vector can be avoided, and further the accuracy of loss prediction is improved.
Step S4: according to a first loss prediction result corresponding to each initial video, screening a first intermediate video from the initial video set:
and screening the first intermediate video from the initial video set according to the first loss prediction result corresponding to each initial video. In this embodiment, the first loss prediction result corresponding to each initial video includes a predicted loss value corresponding to each initial video, which may be also understood as: the first loss prediction result corresponding to each initial video is the prediction loss value corresponding to each initial video.
A loss threshold value is preset, and the preset loss threshold value is specifically set according to actual needs. And comparing the predicted loss value corresponding to each initial video with a preset loss threshold value, and acquiring the initial video corresponding to the predicted loss value which is greater than or equal to the preset loss threshold value according to the comparison result, wherein the acquired initial video is the first intermediate video.
Step S5: according to the user information, user characteristic data of the user are obtained:
And acquiring user characteristic data of the user according to the user information. The user characteristic data is characteristic data only related to a user, is data information specific to the user, and is taken as a specific implementation manner, the user characteristic data is a video field possibly interested by the user, and the video field possibly interested comprises two parts, namely: a video domain in which the user has focused, and an associated video domain associated with the video domain in which the user has focused. The video fields of interest of the user are the video fields of interest filled in by the user when registering. The associated video domain associated with the video domain in which the user has focused is a video domain having a certain association with the video domain in which the user has focused. As a specific embodiment, a specific acquisition procedure of the associated video domain is given below: the background server is preset with a video domain knowledge graph, and the video domain knowledge graph comprises the correlation degree between the known video domains, wherein the higher the correlation degree is, the closer the relationship between the two video domains is represented. Then, according to the preset video domain knowledge graph, acquiring the association degree of the video domain concerned by the user and other video domains in the video domain knowledge graph; and comparing each relevance with a preset relevance threshold value, wherein the preset relevance threshold value is set by actual requirements, acquiring a target relevance which is larger than or equal to the preset relevance threshold value according to a comparison result, namely acquiring a relevance which is larger than the relevance of the video domain concerned by the user, and finally acquiring the video domain corresponding to the target relevance to acquire the associated video domain.
Step S6: and screening a second intermediate video from the first intermediate video by combining the first intermediate video with the user characteristic data:
And after the first intermediate video and the user characteristic data are obtained, screening a second intermediate video from the first intermediate video. Since the fields of the respective videos in the first intermediate video are divided into two parts: and one part is in the video field of possible interest, and the other part is not in the video field of possible interest, then acquiring videos in the video field of possible interest in the first intermediate video, wherein the videos are second intermediate videos, namely the finally required videos. These videos may then be pushed to the user.
The embodiment also provides a big data based video screening device, which comprises a memory, a processor and a computer program stored on the memory and running on the processor, wherein the processor realizes the big data based video screening method when executing the computer program. Since the detailed description of the method for screening video based on big data is given above, the detailed description is omitted.

Claims (6)

1. A big data based video screening method, comprising:
Acquiring an initial video set related to user information according to the user information, wherein the initial video set comprises at least two initial videos;
For any one initial video in the initial video set, acquiring a first feature vector of the initial video according to a preset screening model;
acquiring a first loss prediction result of the first feature vector according to a preset loss prediction model;
screening a first intermediate video from the initial video set according to a first loss prediction result corresponding to each initial video;
Acquiring user characteristic data of the user according to the user information;
screening a second intermediate video from the first intermediate video by combining the first intermediate video with the user characteristic data;
the screening model comprises at least two convolution layers, and each convolution layer can output a first feature vector;
The method comprises the steps of obtaining a first feature vector of the initial video according to a preset screening model, wherein the first feature vector is specifically:
obtaining a first feature vector of the initial video according to a convolution layer in the screening model;
The loss prediction model comprises at least two loss prediction sub-models and a classifier, wherein each loss prediction sub-model corresponds to each convolution layer one by one, and the input of each loss prediction sub-model is a first feature vector output by the corresponding convolution layer;
the method comprises the steps of obtaining a first loss prediction result of the first feature vector according to a preset loss prediction model, wherein the first loss prediction result is specifically:
For any one first feature vector, inputting the first feature vector into a loss predictor model corresponding to the first feature vector, and obtaining a first vector output by the loss predictor model aiming at the first feature vector;
integrating the obtained first vectors to obtain second vectors;
And obtaining the first loss prediction result according to the second vector and the classifier.
2. The big data based video filtering method of claim 1, wherein the user information includes a historical video play record of the user;
the method comprises the steps of obtaining an initial video set related to user information according to the user information, wherein the initial video set is specifically:
and acquiring videos in the same field and related fields as each video recorded in the historical video play record according to the historical video play record, wherein the acquired videos form the initial video set.
3. The big data based video filtering method according to claim 1, wherein the first loss prediction result corresponding to each initial video includes a predicted loss value corresponding to each initial video;
The first intermediate video is screened from the initial video set according to the first loss prediction result corresponding to each initial video, specifically:
And comparing the predicted loss value corresponding to each initial video with a preset loss threshold value, and acquiring the initial video corresponding to the predicted loss value which is larger than or equal to the preset loss threshold value according to the comparison result to obtain the first intermediate video.
4. The big data based video filtering method according to claim 1, wherein the obtaining the user characteristic data of the user according to the user information specifically includes:
acquiring possible interesting video fields of the user according to the user information, wherein the possible interesting video fields comprise video fields of the user and associated video fields associated with the video fields of the user;
The step of screening a second intermediate video from the first intermediate video by combining the first intermediate video with the user characteristic data specifically includes:
And acquiring the video in the possible interesting video field in the first intermediate video according to the possible interesting video field, and obtaining the second intermediate video.
5. The big data-based video screening method according to claim 4, wherein the acquiring process of the associated video domain specifically includes:
acquiring the association degree of the video domain concerned by the user and other video domains in the video domain knowledge graph according to a preset video domain knowledge graph;
And comparing each association degree with a preset association degree threshold value, acquiring a target association degree which is larger than or equal to the preset association degree threshold value according to a comparison result, and acquiring a video field corresponding to the target association degree to acquire the associated video field.
6. A big data based video screening device comprising a memory and a processor, and a computer program stored on the memory and running on the processor, wherein the processor implements the big data based video screening method according to any of claims 1-5 when executing the computer program.
CN202110428964.6A 2021-04-21 2021-04-21 Video screening method and device based on big data Active CN113268631B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110428964.6A CN113268631B (en) 2021-04-21 2021-04-21 Video screening method and device based on big data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110428964.6A CN113268631B (en) 2021-04-21 2021-04-21 Video screening method and device based on big data

Publications (2)

Publication Number Publication Date
CN113268631A CN113268631A (en) 2021-08-17
CN113268631B true CN113268631B (en) 2024-04-19

Family

ID=77229202

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110428964.6A Active CN113268631B (en) 2021-04-21 2021-04-21 Video screening method and device based on big data

Country Status (1)

Country Link
CN (1) CN113268631B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110009013A (en) * 2019-03-21 2019-07-12 腾讯科技(深圳)有限公司 Encoder training and characterization information extracting method and device
CN110070067A (en) * 2019-04-29 2019-07-30 北京金山云网络技术有限公司 The training method of video classification methods and its model, device and electronic equipment
CN111737520A (en) * 2020-06-22 2020-10-02 Oppo广东移动通信有限公司 Video classification method, video classification device, electronic equipment and storage medium
CN112149604A (en) * 2020-09-30 2020-12-29 网易传媒科技(北京)有限公司 Training method of video feature extraction model, video recommendation method and device
CN112417207A (en) * 2020-11-24 2021-02-26 未来电视有限公司 Video recommendation method, device, equipment and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110009013A (en) * 2019-03-21 2019-07-12 腾讯科技(深圳)有限公司 Encoder training and characterization information extracting method and device
CN110070067A (en) * 2019-04-29 2019-07-30 北京金山云网络技术有限公司 The training method of video classification methods and its model, device and electronic equipment
CN111737520A (en) * 2020-06-22 2020-10-02 Oppo广东移动通信有限公司 Video classification method, video classification device, electronic equipment and storage medium
CN112149604A (en) * 2020-09-30 2020-12-29 网易传媒科技(北京)有限公司 Training method of video feature extraction model, video recommendation method and device
CN112417207A (en) * 2020-11-24 2021-02-26 未来电视有限公司 Video recommendation method, device, equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
一种基于用户播放行为序列的个性化视频推荐策略;王娜;何晓明;刘志强;王文君;李霞;;计算机学报(第01期) *

Also Published As

Publication number Publication date
CN113268631A (en) 2021-08-17

Similar Documents

Publication Publication Date Title
JP6843937B2 (en) Video data processing methods, equipment and readable storage media
CN110704674B (en) Video playing integrity prediction method and device
CN111741330B (en) Video content evaluation method and device, storage medium and computer equipment
CN110839173A (en) Music matching method, device, terminal and storage medium
CN105744292A (en) Video data processing method and device
CN104216956B (en) The searching method and device of a kind of pictorial information
CN108805151B (en) Image classification method based on depth similarity network
KR102042168B1 (en) Methods and apparatuses for generating text to video based on time series adversarial neural network
CN111401100A (en) Video quality evaluation method, device, equipment and storage medium
CN114637911B (en) Method for recommending next interest point of attention fusion perception network
CN108595526B (en) Resource recommendation method and device
CN111816170A (en) Training of audio classification model and junk audio recognition method and device
CN111242176A (en) Computer vision task processing method and device and electronic system
CN106604057A (en) Video processing method and apparatus thereof
CN113268631B (en) Video screening method and device based on big data
CN109740621B (en) Video classification method, device and equipment
CN114842382A (en) Method, device, equipment and medium for generating semantic vector of video
CN114500879A (en) Video data processing method, device, equipment and storage medium
CN111161741B (en) Personalized information identification method and device, electronic equipment and storage medium
CN116264606A (en) Method, apparatus and computer program product for processing video
CN113472834A (en) Object pushing method and device
CN112131995A (en) Action classification method and device, computing equipment and storage medium
CN117540062B (en) Retrieval model recommendation method and device based on knowledge graph
CN114819000B (en) Feedback information estimation model training method and device and electronic equipment
CN115331673B (en) Voiceprint recognition household appliance control method and device in complex sound scene

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20240327

Address after: Room 318, 3rd Floor, West Annex Building, Taixing Building, No. 11 Huayuan East Road, Haidian District, Beijing, 100191 (Dongsheng District)

Applicant after: Beijing Dianzhong Kuaikan Technology Co.,Ltd.

Country or region after: China

Address before: No. 903, unit 2, building 8, Zhoushan new town, east of Gaoxin Longping road and south of sunshi highway, Luoyang area, China (Henan) pilot Free Trade Zone, Luoyang City, Henan Province, 471000

Applicant before: Luoyang moxiao Network Technology Co.,Ltd.

Country or region before: China

GR01 Patent grant