CN110825912B

CN110825912B - Video generation method and device, electronic equipment and storage medium

Info

Publication number: CN110825912B
Application number: CN201911045259.7A
Authority: CN
Inventors: 王倩; 帕哈尔丁·帕力万; 毛珊珊; 孙启玉
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2019-10-30
Filing date: 2019-10-30
Publication date: 2022-04-22
Anticipated expiration: 2039-10-30
Also published as: CN110825912A

Abstract

The present disclosure relates to a video generation method, a video generation device, an electronic device, and a storage medium, where the method is applied to a client and includes: when a video production instruction is received, displaying a plurality of preset video templates; acquiring a first target template determined by a user from video templates; displaying the multimedia materials recommended for the user; receiving a selection instruction input by a user, and determining a first target multimedia material for a first target template; the first target multimedia material is: the user selects multimedia materials from the recommended multimedia materials for the user or from pre-stored material files; and filling the first target multimedia material into the first target template to generate a video. By applying the embodiment of the invention, the multimedia material matched with the first target template can be recommended for the user, so that the user can select the multimedia material from the pre-stored material file and can directly use the multimedia material recommended for the user, and the video making process is simplified.

Description

Video generation method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of video processing technologies, and in particular, to a video generation method and apparatus, an electronic device, and a storage medium.

Background

In recent years, with the popularization of mobile terminals, more and more users are willing to share their video works on various short video platforms. In order to enable a user to make a video more quickly, the video can be synthesized in an intelligent form by using multimedia materials imported by the user when the video is generated.

Specifically, when a video is made, a user can select one of a plurality of preset video templates, and then, according to the quantity of materials required by the selected template, a picture or a video is imported from a local album; and finally, filling the picture or the video imported by the user into the selected video template, and finishing the video production.

In the related technology, when a user makes a video for the first time or switches a selected template to make a video, the user needs to reselect and import a picture or a video from an album, and the operation is complicated.

Disclosure of Invention

The present disclosure provides a video generation method, an apparatus, an electronic device, and a storage medium, so as to at least solve the problem of complicated operation in video production in the related art. The technical scheme of the disclosure is as follows:

according to a first aspect of the embodiments of the present disclosure, there is provided a video generation method, applied to a client, including:

when a video production instruction is received, displaying a plurality of preset video templates;

acquiring a first target template determined by a user from the video template;

displaying the multimedia materials recommended for the user according to the first target template;

receiving a selection instruction input by a user, and determining a first target multimedia material for the first target template; the first target multimedia material is: the user selects multimedia materials from the multimedia materials recommended for the user or from pre-stored material files;

and filling the first target multimedia material into the first target template to generate a video.

Optionally, the step of displaying the multimedia material recommended for the user according to the first target template includes:

obtaining a first target template type of a first target template;

screening out a first target number of multimedia materials consistent with the first target template type from the prestored material files according to the first target template type of the first target template and the first target number of the required multimedia materials, and using the first target number of the multimedia materials as the multimedia materials recommended for the user;

generating a recommendation window;

and displaying the multimedia material recommended for the user in the recommendation window.

Optionally, the step of screening, according to the first target template type of the first target template and the first target number of the required multimedia materials, a first target number of multimedia materials consistent with the first target template type from the pre-stored material files as multimedia materials recommended to the user includes:

selecting a preset number of multimedia materials to be identified from the prestored material files according to the generation time of the multimedia materials;

performing type identification on the multimedia material to be identified, and determining the multimedia material with the type consistent with that of the first target template as a first type of candidate material;

and acquiring the newly generated first target number of multimedia materials as the multimedia materials recommended for the user according to the generation time of each multimedia material in the first type of candidate materials.

Optionally, the step of receiving a selection instruction input by a user and determining a first target multimedia material for the first target template includes:

detecting whether an operation instruction input by a user for using a control for the material in the recommendation window or adding a control for the material in the interface is received;

when an operation instruction input by a user for the material use control is detected, determining the multimedia material recommended for the user as the first target multimedia material;

and when an operation instruction for the material adding control, which is input by a user, is detected, receiving a first target number of multimedia materials selected by the user from pre-stored material files, and determining the multimedia materials as the first target multimedia materials.

Optionally, the method further includes:

detecting whether a switching instruction for switching the first target template to a second target template in the plurality of video templates is received, wherein the switching instruction is input by a user;

if so, second target multimedia material for the second target template is determined based on the determined first target multimedia material.

Optionally, the step of determining a second target multimedia material for the second target template based on the determined first target multimedia material includes:

judging whether the material formats supported by the second target template and the first target template are the same or not; the material format comprises the following steps: picture format and video format;

if the first target multimedia material and the second target template are different, deleting the target multimedia material in the material format which is not supported by the second target template in the determined first target multimedia material to obtain the residual multimedia material;

selecting a second target multimedia material for the second target template from the remaining multimedia materials;

and if so, selecting a second target multimedia material for the second target template from the determined first target multimedia materials.

comparing the number of the first target multimedia materials with a second target number of the multimedia materials required by the second target template;

if the number of the first target multimedia materials is less than or equal to the second target number of the multimedia materials required by the second target template, generating a recommendation window, and displaying the first target multimedia materials in the recommendation window as second target multimedia materials for the second target template;

and if the number of the first target multimedia materials is larger than the second target number of the multimedia materials required by the second target template, screening the first target multimedia materials to obtain second target multimedia materials for the second target template.

Optionally, if the number of the first target multimedia materials is greater than the second target number of the multimedia materials required by the second target template, the step of obtaining the second target multimedia materials for the second target template after screening the first target multimedia materials includes:

calculating a first similarity between the first target template and the second target template according to a first target template type of the first target template and a second target template type of the second target template;

if the first similarity is larger than a first preset threshold value, selecting a second target number of multimedia materials from the first target multimedia materials as second target multimedia materials for the second target template;

and if the first similarity is smaller than or equal to a first preset threshold value, selecting a second target number of multimedia materials from pre-stored material files as second target multimedia materials for the second target template.

Optionally, if the first similarity is smaller than or equal to a first preset threshold, before the step of selecting a second target number of multimedia materials from the pre-saved material files as second target multimedia materials for the second target template, the method further includes:

calculating a second similarity between the first target multimedia material and the second target template;

selecting a second target number of multimedia materials with the second similarity larger than a second preset threshold and the highest second similarity from the first target multimedia materials as second target multimedia materials for the second target template;

and if the first target multimedia materials have no materials with the second similarity larger than a second preset threshold, executing the step of selecting a second target number of multimedia materials from the pre-stored material files as second target multimedia materials for the second target template.

According to a second aspect of the embodiments of the present disclosure, there is provided a video generating apparatus applied to a client, including:

a template display unit configured to display a preset plurality of video templates when a video production instruction is received;

a first target template acquisition unit configured to acquire a first target template determined by a user from the video templates;

a material display unit configured to display a multimedia material recommended for a user according to the first target template;

a first target multimedia material determining unit configured to receive a selection instruction input by a user, and determine a first target multimedia material for the first target template; the first target multimedia material is: the user selects multimedia materials from the multimedia materials recommended for the user or from pre-stored material files;

and the video generation unit is configured to fill the first target multimedia material into the first target template to generate a video.

Optionally, the material display unit includes:

a first target template type obtaining module configured to obtain a first target template type of a first target template;

the material screening module is configured to screen out a first target number of multimedia materials which are consistent with the type of the first target template from the pre-stored material files according to the type of the first target template and the first target number of the required multimedia materials, and the first target number of the multimedia materials is used as the multimedia materials recommended to the user;

a window generation module configured to generate a recommendation window;

and the material display module is configured to display the multimedia material recommended for the user in the recommendation window.

Optionally, the material screening module is specifically configured to:

Optionally, the first target multimedia material determining unit is specifically configured to:

Optionally, the apparatus further comprises:

a switching instruction detection unit configured to detect whether a switching instruction for switching the first target template to a second target template of the plurality of video templates input by a user is received;

a second target multimedia material determination unit configured to determine, if so, a second target multimedia material for the second target template based on the determined first target multimedia material.

Optionally, the second target multimedia material determining unit is specifically configured to:

Optionally, the second target multimedia material determining unit includes:

a quantity comparison module configured to compare the quantity of the first target multimedia material with a second target quantity of multimedia material required by the second target template;

the second target multimedia material display module is configured to generate a recommendation window if the number of the first target multimedia materials is less than or equal to a second target number of the multimedia materials required by the second target template, and display the first target multimedia materials in the recommendation window as second target multimedia materials for the second target template;

and the second target multimedia material obtaining module is configured to obtain second target multimedia materials for the second target template after screening the first target multimedia materials if the number of the first target multimedia materials is larger than the second target number of the multimedia materials required by the second target template.

Optionally, the second target multimedia material obtaining module includes:

a first similarity operator module configured to calculate a first similarity between the first target template and the second target template according to a first target template type of the first target template and a second target template type of the second target template;

a first material selection sub-module configured to select a second target number of multimedia materials from the first target multimedia materials as second target multimedia materials for the second target template if the first similarity is greater than a first preset threshold;

a second material selection sub-module configured to select a second target number of multimedia materials from the pre-saved material files as second target multimedia materials for the second target template if the first similarity is less than or equal to a first preset threshold.

Optionally, the second target multimedia material obtaining module further includes:

a second similarity calculation sub-module configured to calculate a second similarity between the first target multimedia material and the second target template before the second material selection sub-module performs the step of selecting a second target number of multimedia materials from the pre-saved material files as second target multimedia materials for the second target template if the first similarity is less than or equal to a first preset threshold;

the selecting submodule is configured to select a second target number of multimedia materials, with the second similarity larger than a second preset threshold and the highest second similarity, from the first target multimedia materials as second target multimedia materials for the second target template; and if the first target multimedia materials have no materials with the second similarity larger than a second preset threshold, executing the step of selecting a second target number of multimedia materials from the pre-stored material files as second target multimedia materials for the second target template.

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

a memory for storing a computer program;

and a processor for implementing any of the method steps of the video generation described above when executing the program stored in the memory.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium having stored therein a computer program, which when executed by a processor, implements the steps of any of the above-described methods of video generation.

According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the methods of video generation described above.

The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects: displaying a plurality of preset video templates when a video production instruction is received; acquiring a first target template determined by a user from the video template; displaying the multimedia materials recommended for the user according to the first target template; receiving a selection instruction input by a user, and determining a first target multimedia material for the first target template; and filling the first target multimedia material into the first target template to generate a video. After the user determines the first target template, the multimedia material matched with the first target template can be recommended for the user, so that the user can select the multimedia material from the pre-stored material files by himself or can directly use the multimedia material recommended for the user, and the video making process is simplified.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.

Fig. 1 is a flow diagram illustrating a video generation method according to an example embodiment.

Fig. 2 is another flow diagram illustrating a video generation method according to an example embodiment.

FIG. 3 is a diagram of an example of a page in the embodiment shown in FIG. 2.

Fig. 4 is a detailed flowchart of switching to a second target template during video generation, and determining a second target multimedia material for the second target template.

Fig. 5 is a block diagram illustrating a video generation apparatus according to an example embodiment.

Fig. 6 is a block diagram illustrating an apparatus for video generation in accordance with an example embodiment.

Fig. 7 is a block diagram illustrating an apparatus for video generation in accordance with an example embodiment.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

In order to solve the problem that in the related art, when a user makes a video for the first time or switches a selected template to make a video, the user needs to reselect and import a picture or a video from an album, and the operation is complex, the embodiments of the present disclosure provide a generation method, an apparatus, an electronic device, and a storage medium. The method for generating the video provided by the embodiment of the disclosure can be applied to any client needing to generate the video, wherein the client is an electronic device, such as: a computer or a mobile terminal, etc., which are not limited herein.

Fig. 1 is a flow diagram illustrating a video generation method according to an example embodiment. As shown in fig. 1, the video generation method is applied to a client, and includes the following steps:

in step S101, when a video production instruction is received, a plurality of preset video templates are displayed.

In one possible embodiment, each video template displayed is pre-marked with a template type. Specifically, the template types may be "baby", "beauty", "travel", "gourmet", and "graduation season", etc.

In step S102, a first target template determined by the user from the video templates is obtained.

In step S103, the multimedia material recommended for the user is displayed according to the first target template.

In this step, the multimedia material recommended for the user may be a photo material and/or a video material.

When the multimedia material recommended for the user is determined, it can be selected from the material files saved in advance according to the template type of the first target template. For example, when the template type of the first target template determined by the user is "food", the photos and/or videos of the food taken in the local album may be used as the recommended multimedia material under the first target template.

In one particular embodiment, a first target template type of a first target template may be obtained; then according to the first target template type of the first target template and the first target number of the required multimedia materials, screening out the multimedia materials with the first target number consistent with the first target template type from the pre-stored material files, and using the multimedia materials as the multimedia materials recommended for the user; generating a recommendation window; and displaying the multimedia material recommended for the user in the recommendation window.

In step S104, receiving a selection instruction input by a user, and determining a first target multimedia material for the first target template; the first target multimedia material is: and the user selects the multimedia materials from the multimedia materials recommended for the user or from the pre-stored material files.

It can be understood that after the recommended multimedia material is displayed for the user, the user can directly use the recommended multimedia material as a first target multimedia material; or selecting the multimedia material from the prestored material files according to the preference of the user, and taking the selected multimedia material as the first target multimedia material.

In step S105, the first target multimedia material is filled into the first target template, and a video is generated.

Therefore, according to the embodiment of the disclosure, after the user determines the first target template, the multimedia material matched with the first target template can be recommended for the user, so that the user can select the multimedia material from the pre-stored material file by himself or can directly use the multimedia material recommended for the user, the video production process is simplified, and the operation is simple.

Fig. 2 is another flow diagram illustrating a video generation method according to an example embodiment. As shown in fig. 2, the method is applied to a client, and includes the following steps:

in step S201, when a video production instruction is received, a plurality of preset video templates are displayed.

As in step S101 in the embodiment shown in fig. 1, each of the video templates displayed is previously marked with a template type.

In step S202, a first target template determined by the user from the video templates is obtained.

In steps S201 to S202, a plurality of preset video templates may be displayed at the bottom of the current page, and the user browses each video template in a left-right sliding manner and selects the first target template through a click operation.

In step S203, a first target template type of the first target template is obtained.

In step S204, according to the generation time of the multimedia material, a preset number of multimedia materials to be identified are selected from the prestored material files.

Specifically, the pre-saved material file may be a pre-shot photograph or video. Since the user selects the multimedia material recently shot with a high probability when generating the video, in one possible embodiment, a preset number of multimedia materials to be identified are selected according to the generation time of each multimedia material in the prestored material file.

The preset number of the multimedia materials to be identified can be determined according to the number of the pre-stored material files. For example, the preset number of multimedia materials to be identified is set to 20% of the number of material files saved in advance.

In step S205, type identification is performed on the multimedia material to be identified, and the multimedia material with the same type as the first target template is determined as a first type candidate material.

In the embodiment, the pre-trained type recognition model is used for recognizing the multimedia material to be recognized. The type recognition model can be a pre-trained convolutional neural network model. Specifically, the multimedia material to be identified is input into a type identification model, the material type of each multimedia material is identified, and the multimedia material with the identified material type being the same as the type of the first target template is determined as a first type of candidate material.

In step S206, a first target number of multimedia materials that are generated latest are obtained as the multimedia materials recommended for the user according to the generation time of each multimedia material in the first type of candidate materials.

In this embodiment, the amount of multimedia material required for each video template may be different. If the number of the multimedia materials required by the first target template determined by the user is 3, that is, the first target number is 3, it indicates that the newly generated 3 multimedia materials should be selected from the first class of candidate materials as the multimedia materials recommended for the user.

Optionally, before selecting the multimedia material recommended for the user according to the generation time, whether the first target template needs the multimedia material containing the face or not may be judged, the first type of candidate material is screened according to the judgment result, and the repeated multimedia material is deleted; and finally, acquiring the newly generated first target number of multimedia materials according to the generation time of each screened first type of candidate material.

It is to be understood that when the first target template type of the first target template determined by the user is "beauty" or "baby", the target multimedia material for filling should be face-containing. Therefore, through the step of judging whether the first target template needs the multimedia material containing the human face, the recommended multimedia material can be matched with the target template, and the user can use the target template conveniently and directly.

In step S207, a recommendation window is generated.

In step S208, in the recommendation window, the multimedia material recommended for the user is displayed.

In step S209, it is detected whether an operation instruction input by a user to use a control for material use in the recommendation window or a control for material addition in the interface is received.

It is practicable to detect whether the user clicks a material adding button in the interface or a material using button in the recommendation window.

If none is detected, step S209 may be continued; when an operation instruction for the material use control input by a user is detected, executing step S210; when an operation instruction for the material addition control input by the user is detected, step S211 is executed.

In step S210, when an operation instruction of the material usage control input by the user is detected, the multimedia material recommended for the user is determined as a first target multimedia material.

In step S211, when an operation instruction for the material addition control input by the user is detected, a first target number of multimedia materials selected by the user from the pre-saved material files is received and determined as first target multimedia materials.

In step S212, the first target template is filled with the first target multimedia material, and a video is generated.

In step S213, it is detected whether a switching instruction for switching the first target template to a second target template of the plurality of video templates is received, which is input by a user.

If the result of the determination is yes, that is, a switching instruction for switching the first target template to a second target template in the plurality of video templates is received, the step S214 is executed; and if the judgment result is negative, namely a switching instruction for switching the first target template to a second target template in the plurality of video templates, which is input by a user, is not received, ending the operation.

In step S214, second target multimedia material for the second target template is determined based on the determined first target multimedia material.

In a specific embodiment, it may be determined whether the material formats supported by the second target template and the first target template are the same; the material format comprises the following steps: picture format and video format; if the first target multimedia material and the second target template are different, deleting the target multimedia material in the material format which is not supported by the second target template in the determined first target multimedia material to obtain the residual multimedia material; selecting a second target multimedia material for the second target template from the remaining multimedia materials; and if so, selecting a second target multimedia material for the second target template from the determined first target multimedia materials.

For example, if a video is produced by using the first target template determined last time, the required multimedia material may be multimedia material in picture format and video format, and if the first target multimedia material determined last time contains a picture and a video, but the switched second target template only supports the multimedia material in picture format, the multimedia material in video format in the first target multimedia material determined last time should be deleted completely.

In another specific embodiment, the number of the first target multimedia material may be compared with a second target number of the multimedia material required by the second target template; if the number of the first target multimedia materials is less than or equal to the second target number of the multimedia materials required by the second target template, generating a recommendation window, and displaying the first target multimedia materials in the recommendation window as second target multimedia materials for the second target template; and if the number of the first target multimedia materials is larger than the second target number of the multimedia materials required by the second target template, screening the first target multimedia materials to obtain second target multimedia materials for the second target template.

Obviously, in the embodiment shown in fig. 2, compared with the case that all the material files stored in advance are identified one by one, only the preset number of multimedia materials generated in the near term are selected for identification, so that the material identification time can be greatly reduced, the timeliness of the algorithm is improved, and the user experience is improved; meanwhile, the user can determine the first target multimedia material matched with the first target template in multiple modes, and the convenient and fast manufacturing requirement of the user for manufacturing the video according to the preference of the user is further met.

Moreover, if the user switches to the second target template in the process of generating the video, the second target multimedia material for the second target template can be determined based on the first target multimedia material, so that the time is saved.

FIG. 3 is a diagram of an example of a page in the embodiment shown in FIG. 2. Referring to fig. 3, the first target template type of the first target template selected by the user is "vitality", and the first target template corresponds to a first target number of 5, that is, the number of required multimedia materials is 5. The recommended multimedia material for the user may be displayed in the lower left recommendation window, i.e. the bubble above the "batch add" word in the figure. When the user clicks the "use" button in the recommendation window, the multimedia material recommended for the user is determined as the first target multimedia material. If the user clicks the material adding button, the local album is opened after the right of using the album is obtained, so that the user can select the multimedia material from the local album; specifically, a user can click a batch adding button, and 5 multimedia materials are selected from material files in the local album to serve as first target multimedia materials; or clicking a 'material adding' button positioned in the middle of the page to add the multimedia materials one by one, wherein the style of the material added to the first target template can be visually observed from a preview area at the bottom of the page when each multimedia material is added.

Referring to fig. 4, a specific process of determining a second target multimedia material for a second target template for a user to switch from a first target template to the second target template in a video generation process is shown in fig. 4, and includes:

in step S401, it is detected whether a switching instruction for switching the first target template to a second target template of the plurality of video templates is received, the switching instruction being input by a user.

If the judgment result is negative, the operation is ended; if the judgment result is yes, namely a switching instruction for switching the first target template to a second target template in the plurality of video templates is received, the processing is carried out in two modes.

The first mode is as follows: step S402 is directly performed.

The second mode is as follows: firstly, judging whether the material formats supported by a second target template and a first target template are the same, if so, deleting the target multimedia materials in the material formats which are not supported by the second target template in the determined first target multimedia materials to obtain the residual multimedia materials, using the residual multimedia materials as the first target multimedia materials, and then executing the step S402; if the second target template is the same as the material format supported by the first target template, step S402 is directly performed.

In step S402, the amount of the first target multimedia material is compared with a second target amount of multimedia material required by the second target template.

In step S403, if the number of the first target multimedia materials is less than or equal to the second target number of the multimedia materials required by the second target template, a recommendation window is generated, and the first target multimedia materials are displayed in the recommendation window as second target multimedia materials for the second target template.

In step S404, if the number of the first target multimedia materials is greater than the second target number of the multimedia materials required by the second target template, a first similarity between the first target template and the second target template is calculated according to the first target template type of the first target template and the second target template type of the second target template.

It is practicable to calculate the first similarity according to the following formula:

first similarity is template type similarity × 0.8+ music tempo similarity × 0.1+ picture style similarity × 0.1

(first similarity: genre, rhythm and style; second: audio similarity, image similarity, and, if any, work similarity)

Specifically, the characters of the template types of the first target template and the second target template may be converted into character vectors, respectively, and the euclidean distance of the character vectors is calculated as the template type similarity; extracting rhythm characteristics from the audio data of the first target template and the second target template respectively, and calculating music rhythm similarity between the audio characteristics of the first target template and the second target template; finally, extracting image features from the images of the first target template and the second target template respectively, and calculating the image similarity between the image features of the first target template and the second target template as the picture style similarity; and weighting and summing the template type similarity, the music rhythm similarity and the picture style similarity according to the formula to obtain a first similarity.

In step S405, it is determined whether the first similarity is greater than a first preset threshold.

If the first similarity is greater than the first preset threshold, executing step S406; if the first similarity is less than or equal to the first preset threshold, step S407 is executed.

In step S406, a second target number of multimedia materials is selected from the first target multimedia materials as second target multimedia materials for the second target template.

In step S407, a second similarity between the first target multimedia material and the second target template is calculated.

The method of calculating the second similarity is similar to the method of calculating the first similarity: firstly, extracting rhythm characteristics from audio data of a first target multimedia material and a second target template respectively, and calculating music rhythm similarity between the audio characteristics of the first target multimedia material and the second target template; then, respectively extracting image features from the images of the first target multimedia material and the second target template, and calculating the image similarity between the image features of the first target multimedia material and the second target template as the picture style similarity; furthermore, if the video history data generated under the second target template is locally saved, the image feature and the audio feature are respectively extracted from the video history data, and the similarity is calculated and averaged with the image feature and the audio feature extracted from the video history data generated based on the first target template as the work similarity between the first target multimedia material and the second target template. And finally, weighting and summing the music rhythm similarity, the picture style similarity and the work similarity between the first target multimedia material and the second target template according to the following formula to obtain a second similarity:

second similarity is work similarity × 0.2+ music tempo similarity × 0.4+ picture style similarity × 0.4

In addition, if the video data generated under the second target template is not locally saved, the work similarity may be set to 0.

In this embodiment, the manner of extracting the image feature and the audio feature is not limited, as long as the image feature or the audio feature can be extracted. For example: feature extraction can be performed by using a pre-trained neural network model respectively.

In step S408, a second target number of multimedia materials, of which the second similarity is greater than a second preset threshold and is the highest, are selected from the first target multimedia materials as second target multimedia materials for the second target template. And finishing the operation.

In step S409, if there is no material with the second similarity greater than the second preset threshold in the first target multimedia material, a second target number of multimedia materials are selected from the pre-saved material files as second target multimedia materials for the second target template.

For example: and if the second target number is 10 and the number of the materials with the second similarity larger than the second preset threshold value in the first target multimedia materials is 0, selecting 10 multimedia materials from the pre-stored material files as second target multimedia materials for the second target template.

It may be implemented, if the second target number is 10; in the first target multimedia material, the number of the materials with the second similarity larger than the second preset threshold is 2, and two implementation modes are available. In the first embodiment, the two materials with the second similarity greater than the second preset threshold are used as second target multimedia materials for the second target template, and then 8 multimedia materials are selected from the prestored material files and used as second target multimedia materials for the second target template; in the second embodiment, since the number of the materials with the second similarity greater than the second preset threshold is 2, and the number of the materials with the second similarity greater than the second preset threshold is not enough for the second target number, the two materials with the second similarity greater than the second preset threshold are not used as the second target multimedia materials for the second target template, and 10 multimedia materials are directly selected from the pre-saved material files and used as the second target multimedia materials for the second target template.

Therefore, if the user switches to the second target template in the process of generating the video, the second target multimedia material for the second target template can be determined based on the first target multimedia material, so that the time is saved.

Moreover, after the templates are switched, based on the first similarity, the first preset threshold, the second similarity and the second preset threshold, the multimedia material with higher similarity is further screened out from the first target multimedia material to serve as the second target multimedia material, the video making process is simplified, meanwhile, the recommended multimedia material can be matched with the second target template, and the user can use the recommended multimedia material conveniently and directly.

Fig. 5 is a block diagram illustrating a video generation apparatus applied to a client according to an exemplary embodiment. Referring to fig. 5, the apparatus includes a template display unit 510, a first target template acquisition unit 520, a material display unit 530, a first target multimedia material determination unit 540, and a video generation unit 550.

A template display unit 510 configured to display a preset plurality of video templates when a video production instruction is received;

a first target template obtaining unit 520 configured to obtain a first target template determined by a user from the video templates;

a material display unit 530 configured to display a multimedia material recommended for a user according to the first target template;

a first target multimedia material determining unit 540 configured to receive a selection instruction input by a user, and determine a first target multimedia material for the first target template; the first target multimedia material is: the user selects multimedia materials from the multimedia materials recommended for the user or from pre-stored material files;

a video generating unit 550 configured to fill the first target multimedia material into the first target template, generating a video.

Optionally, the material display unit includes:

a window generation module configured to generate a recommendation window;

Optionally, the material screening module is specifically configured to:

Optionally, the apparatus further comprises:

Optionally, the second target multimedia material determining unit includes:

Optionally, the second target multimedia material obtaining module includes:

Therefore, according to the embodiment of the disclosure, after the user determines the first target template, the multimedia material matched with the first target template can be recommended to the user, so that the user can select the multimedia material from the pre-stored material file by himself or can directly use the multimedia material recommended to the user, and the video production process is simplified.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Fig. 6 is a block diagram illustrating an apparatus 600 for video generation according to an example embodiment. For example, the apparatus 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.

Referring to fig. 6, apparatus 600 may include one or more of the following components: a processing component 602, a memory 604, a power component 606, a multimedia component 608, an audio component 610, an input/output (I/O) interface 612, a sensor component 614, and a communication component 616.

The processing component 602 generally controls overall operation of the device 600, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 602 can include one or more modules that facilitate interaction between the processing component 602 and other components. For example, the processing component 602 can include a multimedia module to facilitate interaction between the multimedia component 608 and the processing component 602.

The memory 604 is configured to store various types of data to support operations at the apparatus 600. Examples of such data include instructions for any application or method operating on device 600, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 604 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

Power supply component 606 provides power to the various components of device 600. The power components 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the apparatus 600.

The multimedia component 608 includes a screen that provides an output interface between the device 600 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the device 600 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 610 is configured to output and/or input audio signals. For example, audio component 610 includes a Microphone (MIC) configured to receive external audio signals when apparatus 600 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in the memory 604 or transmitted via the communication component 616. In some embodiments, audio component 610 further includes a speaker for outputting audio signals.

The I/O interface 612 provides an interface between the processing component 602 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor component 614 includes one or more sensors for providing status assessment of various aspects of the apparatus 600. For example, the sensor component 614 may detect an open/closed state of the device 600, the relative positioning of components, such as a display and keypad of the apparatus 600, the sensor component 614 may also detect a change in position of the apparatus 600 or a component of the apparatus 600, the presence or absence of user contact with the apparatus 600, orientation or acceleration/deceleration of the apparatus 600, and a change in temperature of the apparatus 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 616 is configured to facilitate communications between the apparatus 600 and other devices in a wired or wireless manner. The apparatus 600 may access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G, or 5G), or a combination thereof. In an exemplary embodiment, the communication component 616 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 600 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a storage medium comprising instructions, such as the memory 804 comprising instructions, executable by the processor 620 of the apparatus 600 to perform the method described above is also provided. Alternatively, the storage medium may be a non-transitory computer readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

Fig. 7 is a block diagram illustrating an apparatus 700 for video generation according to an example embodiment. For example, the apparatus 700 may be provided as a server. Referring to fig. 7, apparatus 700 includes a processing component 722 that further includes one or more processors and memory resources, represented by memory 732, for storing instructions, such as applications, that are executable by processing component 722. The application programs stored in memory 732 may include one or more modules that each correspond to a set of instructions. Further, the processing component 722 is configured to execute instructions to perform the methods of video generation described above.

The apparatus 700 may also include a power component 726 configured to perform power management of the apparatus 700, a wired or wireless network interface 750 configured to connect the apparatus 700 to a network, and an input output (I/O) interface 758. The apparatus 700 may operate based on an operating system, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, stored in memory 732.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video generation method is applied to a client and comprises the following steps:

acquiring a first target template determined by a user from the video template;

filling the first target multimedia material into the first target template to generate a video;

if yes, determining second target multimedia materials for the second target template based on the determined first target multimedia materials;

said step of determining second target multimedia material for said second target template based on said determined first target multimedia material comprises:

if the number of the first target multimedia materials is larger than the second target number of the multimedia materials required by the second target template, screening the first target multimedia materials to obtain second target multimedia materials for the second target template;

if the number of the first target multimedia materials is larger than the second target number of the multimedia materials required by the second target template, the step of obtaining the second target multimedia materials for the second target template after screening the first target multimedia materials comprises the following steps:

2. The video generation method according to claim 1, wherein the step of displaying the multimedia material recommended for the user according to the first target template includes:

obtaining a first target template type of a first target template;

generating a recommendation window;

3. The video generating method according to claim 2, wherein the step of filtering out a first target number of multimedia materials corresponding to the first target template type from the pre-saved material files as the multimedia materials recommended to the user, based on the first target template type of the first target template and the first target number of the required multimedia materials, comprises:

4. The video generation method according to claim 2, wherein the step of receiving a selection instruction input by a user and determining a first target multimedia material for the first target template comprises:

5. The video generation method according to claim 1, wherein the step of determining a second target multimedia material for the second target template based on the determined first target multimedia material comprises:

6. The video generation method according to claim 1, wherein, if the first similarity is less than or equal to a first preset threshold, the step of selecting a second target number of multimedia materials from the pre-saved material files as second target multimedia materials for the second target template further comprises, before the step of:

7. A video generation apparatus, applied to a client, comprising:

the video generation unit is configured to fill the first target multimedia material into the first target template to generate a video;

a second target multimedia material determination unit configured to determine, if so, a second target multimedia material for the second target template based on the determined first target multimedia material;

the second target multimedia material determination unit includes:

a second target multimedia material obtaining module, configured to, if the number of the first target multimedia materials is greater than a second target number of the multimedia materials required by the second target template, obtain second target multimedia materials for the second target template after screening the first target multimedia materials;

the second target multimedia material obtaining module comprises:

8. The video generation apparatus according to claim 7, wherein the material display unit includes:

a window generation module configured to generate a recommendation window;

9. The video generation apparatus of claim 8, wherein the material filtering module is specifically configured to:

10. The video generation apparatus according to claim 8, wherein the first target multimedia material determination unit is specifically configured to:

11. The video generation apparatus according to claim 7, wherein the second target multimedia material determination unit is specifically configured to:

12. The video generating apparatus as claimed in claim 7, wherein the second target multimedia material obtaining module further comprises:

13. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;

a memory for storing a computer program;

a processor for implementing the method steps of any of claims 1-6 when executing a program stored in the memory.

14. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 6.