WO2026012008A1 - 媒体内容生成方法、装置、设备、存储介质及程序产品 - Google Patents

媒体内容生成方法、装置、设备、存储介质及程序产品

Info

Publication number
WO2026012008A1
WO2026012008A1 PCT/CN2025/098557 CN2025098557W WO2026012008A1 WO 2026012008 A1 WO2026012008 A1 WO 2026012008A1 CN 2025098557 W CN2025098557 W CN 2025098557W WO 2026012008 A1 WO2026012008 A1 WO 2026012008A1
Authority
WO
WIPO (PCT)
Prior art keywords
text
target
episode
multimedia content
dialogue
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2025/098557
Other languages
English (en)
French (fr)
Inventor
尹超
王齐禹
周莜
李书苑
阳丹
李锐
蒋嵘
朱若瑜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology (Shenzhen) Co Ltd
Original Assignee
Tencent Technology (Shenzhen) Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology (Shenzhen) Co Ltd filed Critical Tencent Technology (Shenzhen) Co Ltd
Publication of WO2026012008A1 publication Critical patent/WO2026012008A1/zh
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/44—Browsing; Visualisation therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/48—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/483—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content

Definitions

  • This application relates to the field of information processing technology, and in particular to a media content generation method, apparatus, device, storage medium, and program product.
  • users need to perform multiple steps to generate target multimedia content. For example, users need to manually select material information for creating the target multimedia content, and then edit the selected material information to generate the target multimedia content.
  • This application provides a method, apparatus, device, and storage medium for generating media content.
  • the technical solution is as follows:
  • a media content generation method is provided, the method being executed by a client, the method comprising:
  • a target text for generating the target multimedia content is determined
  • a media content generation method is provided, the method being executed by a server, the method comprising:
  • the client sends target text for generating target multimedia content, wherein the target text is selected from text media content published by the media account.
  • the target multimedia content is sent to the client.
  • a media content generation apparatus comprising:
  • the display module is used to display text media content published by a media account, wherein the text media content includes several texts;
  • a determination module is configured to determine the target text for generating the target multimedia content in response to a selection operation on at least one character among the plurality of texts.
  • the display module is also used to display the target multimedia content generated based on the target text.
  • a media content generation apparatus comprising:
  • the receiving module is used to receive target text sent by the client for generating target multimedia content, wherein the target text is selected from the text media content published by the media account.
  • the generation module is used to generate the target multimedia content based on the target text
  • the sending module is used to send the target multimedia content to the client.
  • a computer device comprising: a processor and a memory, the memory storing at least one program, the at least one program being loaded and executed by the processor to implement the media content generation method as described above.
  • a computer-readable storage medium which stores at least one program that is loaded and executed by a processor to implement the media content generation method as described above.
  • a computer program product comprising at least one program segment stored in a computer-readable storage medium; a processor of a computer device reads the at least one program segment from the computer-readable storage medium, and the processor executes the at least one program segment, causing the computer device to perform the media content generation method as described above.
  • This invention provides a method for automatically generating target multimedia content. Once the target text is determined, the generated multimedia content can be directly displayed. Compared to methods where users manually generate target multimedia content, users do not need to perform multiple steps and can generate target multimedia content matching the target text with a single click, thus improving the efficiency of target multimedia content generation.
  • Figure 1 is a schematic diagram of a computer system provided in an exemplary embodiment of this application.
  • FIG. 2 is a flowchart of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 3 is a schematic diagram of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 4 is a schematic diagram of a media content generation method provided in an exemplary embodiment of this application.
  • FIG. 5 is a flowchart of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 6 is a schematic diagram of a media content generation method provided in an exemplary embodiment of this application.
  • FIG. 7 is a flowchart of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 8 is a schematic diagram of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 9 is a schematic diagram of a media content generation method provided in an exemplary embodiment of this application.
  • FIG. 10 is a flowchart of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 11 is a schematic diagram of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 12 is a schematic diagram of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 13 is a flowchart of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 14 is a flowchart of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 15 is a flowchart of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 16 is a schematic diagram of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 17 is a flowchart of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 18 is a flowchart of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 19 is a flowchart of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 20 is a flowchart of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 21 is a flowchart of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 22 is a flowchart of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 23 is a schematic diagram of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 24 is a schematic diagram of a media content generation method provided in an exemplary embodiment of this application.
  • Figure 25 is a block diagram of a media content generation apparatus provided in an exemplary embodiment of this application.
  • Figure 26 is a block diagram of a media content generation apparatus provided in an exemplary embodiment of this application.
  • Figure 27 is a block diagram of a computer device provided in an exemplary embodiment of this application.
  • first, second, etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another.
  • a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter.
  • the word "if” as used herein may be interpreted as "when,” “when,” or "in response to determination.”
  • Multimedia content refers to information content that is processed through digital signals and presented in the form of text, images, audio, video, etc.
  • Text-based media content refers to media information carried by text.
  • text-based media content refers to text-based media information disseminated on the internet.
  • Screenshots refers to video frame information in movies, TV series, short videos, and other videos.
  • Dialogue in TV series refers to the dialogue information in videos such as movies, TV series, and short videos.
  • FIG. 1 shows a structural block diagram of a computer system provided in an exemplary embodiment of this application.
  • the computer system 100 includes a terminal device 120 and a server 140.
  • Terminal device 120 has a client installed and running that supports displaying text-based media content.
  • the client supporting text-based media content display can be any of the following: video application, reading application, social application, live streaming application, virtual reality (VR) application, or augmented reality (AR) application.
  • Terminal device 120 is the terminal used by the first user, who uses terminal device 120 to view or read text-based media content and to generate corresponding target multimedia content by triggering the text-based media content.
  • Terminal device 120 is connected to server 140 via wireless network or wired network.
  • Server 140 includes at least one of a single server, multiple servers, a cloud computing platform, and a virtualization center.
  • server 140 includes a processor 144 and a memory 142.
  • Memory 142 includes a display module 1421, a determination module 1422, and a generation module 1423.
  • the display module 1421 is used to display multimedia information, such as text media content; the determination module 1422 is used to determine the target text; and the generation module 1423 is used to generate target multimedia content including scene images and dialogue.
  • Server 140 provides background services for clients supporting multimedia information.
  • server 140 undertakes the primary computing work, and terminal device 120 undertakes secondary computing work; or, server 140 undertakes secondary computing work, and terminal device 120 undertakes primary computing work; or, server 140 and terminal device 120 collaborate on computing using a distributed computing architecture.
  • the device type of terminal device 120 includes at least one of the following: smartphone, tablet computer, e-book reader, laptop computer, and desktop computer.
  • a smartphone is used as an example to illustrate terminal device 120.
  • the number of terminals described above can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more. This application does not limit the number of terminals or the type of device.
  • this application proposes a media content generation method that can automatically generate target multimedia content, thereby reducing the user's operation process and improving the generation efficiency of target multimedia content. For example, as shown in FIG1, the user of terminal device 120 can directly select a portion of the text in the text content 12 of the text media content to achieve one-click triggering of the display of the target multimedia content 21 corresponding to the selected text.
  • FIG 2 is a schematic diagram of a media content generation method provided in an exemplary embodiment of this application.
  • the method is described using the terminal device 120 shown in Figure 1 as an example.
  • the method includes:
  • Step 220 Text media content displayed on the terminal device.
  • the text media content is published by a media account.
  • a media account refers to a user account logged in on a client that supports editing and publishing text media content.
  • a media account refers to a user account corresponding to an application on a client that supports editing and publishing text media content.
  • a user account logged in on a social media application For example, a user account logged in on a social media application.
  • the social media application needs to support editing and publishing text media content.
  • text-based media content includes a number of texts.
  • text-based media content also includes other forms of content such as images and videos.
  • text-based media content includes articles from public WeChat accounts, current affairs commentary articles, etc.
  • the text media content is pre-created by the creator corresponding to the media account.
  • text media content 11 is displayed.
  • the text media content 11 includes at least text content 12.
  • the text media content 11 may also include image content 13.
  • Step 240 In response to a selection operation on at least one character among several texts, the terminal device determines the target text for generating the target multimedia content.
  • the selection operation for at least one character among several texts includes at least one of the following: single click, double click, left/right swipe, up/down swipe, long press, hover, facial recognition, and voice recognition. It is worth noting that the selection operation for at least one character among several texts includes, but is not limited to, the aforementioned methods. Those skilled in the art should understand that any operation capable of achieving the above functions falls within the protection scope of the embodiments of this application.
  • the target media content includes at least one of animation, poster, audio, and video.
  • the terminal device in response to a selection operation of at least one character among a plurality of texts, sends a generation request to the server for generating target multimedia content.
  • the generation request for generating target multimedia content includes a request to determine target text based on the selected at least one character.
  • the generation request for generating target multimedia content further includes a request to generate target multimedia content based on the target text.
  • the target text includes key information from the selected at least one character.
  • the terminal device in response to a selection operation on at least one character among a plurality of texts, switches the display state of the selected at least one character from a first display state to a second display state.
  • the first display state is a state where at least one character is not selected
  • the second display state is a state where at least one character is selected.
  • Step 260 The terminal device displays the target multimedia content generated based on the target text.
  • the target multimedia content includes episode scenes and episode dialogue that match the target text.
  • the content of the scene corresponding to the target text matches the text content expressed by the target text.
  • the meaning of the content of the scene corresponding to the target text is consistent with the meaning of the text content expressed by the target text. For example, assuming the target text is "xxx is so beautiful", and the content of scene 1 is "xxx has gorgeous makeup”, then scene 1 is considered to match the target text.
  • the text content corresponding to the TV drama dialogue that matches the target text matches the text content expressed by the target text.
  • the meaning of the text content corresponding to the TV drama dialogue that matches the target text is consistent with the meaning of the text content expressed by the target text.
  • the matching degree between the first text content corresponding to the TV drama dialogue that matches the target text and the second text content expressed by the target text is greater than or equal to a matching degree threshold. For example, assuming the target text is "xxx is really beautiful" and the TV drama dialogue is "xxx is really good-looking", since "beautiful" and "good-looking” express the same meaning, the TV drama dialogue is considered to match the target text.
  • the target multimedia content is generated by the server based on a generation request sent by the terminal device. Based on the generation request, the server determines the episode scenes and dialogue that match the target text, and then generates the target multimedia content. Afterward, the server sends the generated target multimedia content to the terminal device, which receives and displays the target multimedia content.
  • the target multimedia content displayed on the terminal device is the one that matches the target text most closely. That is, the server may generate multiple target multimedia content pieces based on the target text, but the matching degree between these multiple pieces of target multimedia content and the target text is different. For example, assuming that target multimedia content 1 matches the target text with a 90% matching degree and target multimedia content 2 matches the target text with an 80% matching degree, the server will only send target multimedia content 1, which has the highest matching degree, to the terminal device.
  • the target multimedia content displayed on the terminal device is all target multimedia content that matches the target text.
  • the target multimedia content displayed on the terminal device is the target multimedia content whose matching degree with the target text is greater than a matching degree threshold. For example, assuming that the matching degree of target multimedia content 1 with the target text is 90%, the matching degree of target multimedia content 2 with the target text is 80%, and the matching degree of target multimedia content 3 with the target text is 60%, and the matching degree threshold is 75%, then the server will only send target multimedia content 1 and target multimedia content 2 to the terminal device.
  • the source material corresponding to the target multimedia content is a static image. In some embodiments, the source material corresponding to the target multimedia content is a dynamic video frame.
  • the method provided in this embodiment offers an automatic way to generate target multimedia content. After determining the target text, the target multimedia content generated based on the target text can be directly displayed. Compared to methods where users manually generate target multimedia content, users do not need to perform multiple steps; they can generate target multimedia content matching the target text with a single click, reducing user workload and improving the efficiency of generating target multimedia content.
  • the target multimedia content includes an upper half and a lower half arranged side by side.
  • the upper half displays screen footage
  • the lower half displays dialogue.
  • the upper half 19 and the lower half 20 are arranged side by side to form the target multimedia content.
  • the upper half 19 displays screen footage
  • the lower half 20 displays dialogue.
  • the target multimedia content includes a side-by-side upper and lower half-area, with the upper half-area displaying dialogue from the series and the lower half-area displaying visuals from the series.
  • the target multimedia content includes a left half-area and a right half-area side by side, with the left half-area displaying dialogue from the series and the right half-area displaying images from the series.
  • the target multimedia content includes a left half-area and a right half-area side by side, with the left half-area displaying the scene and the right half-area displaying the dialogue.
  • the target multimedia content includes a content area.
  • the content area displays screen images and dialogue overlaid on the screen images.
  • the dialogue may be displayed separately from the screen images. This application does not limit the display method of the screen images and dialogue in these embodiments.
  • the target multimedia content further includes an information area.
  • the information area is used to display information and/or target text related to the episode.
  • the target multimedia content is automatically generated by a terminal device or a server.
  • the target multimedia content is generated by a server, or when the target multimedia content is generated by a terminal different from the terminal device, the following steps are included after step 240 and before step 260:
  • Step 251 The terminal device sends the target text used to generate the target multimedia content
  • Step 252 The terminal device receives the target multimedia content generated based on the target text.
  • step 240 above can also be replaced by the following sub-steps:
  • Step 241 In response to the selection operation of at least one character among a plurality of texts, the terminal device displays the selected text and the selection function bar.
  • the terminal device in response to a selection operation on at least one character among a plurality of texts, switches the selected text from a first display state to a second display state.
  • the first display state is an unselected state
  • the second display state is a selected state.
  • the terminal device displays a selection function bar in response to the selection of at least one character among a plurality of texts. For example, as shown in FIG6, the terminal device displays the selection function bar 14 in response to the selection of at least one character.
  • the selected function bar includes a generate control.
  • the generate control is used to trigger the generation of target multimedia content.
  • the terminal device in response to the triggering operation of the generate control, sends a generation request to the server for generating target multimedia content.
  • the generation request for generating target multimedia content includes a request to determine target text based on at least one selected character.
  • the generation request for generating target multimedia content further includes a request to generate target multimedia content based on the target text.
  • the selected function bar may further include other function controls besides the generate control.
  • the selected function bar may further include at least one of a copy control, a share control, and a favorite control.
  • the copy control is used to copy at least one selected text.
  • the share control is used to share at least one selected text.
  • the favorite control is used to favorite at least one selected text.
  • Step 242 In response to the triggering operation of the generation control, the terminal device determines the selected text as the target text for generating the target multimedia content.
  • the triggering operation for the generated control includes at least one of the following: single click, double click, left/right swipe, up/down swipe, long press, hover, facial recognition, and voice recognition. It is worth noting that the triggering operation for the generated control includes, but is not limited to, several of the above-mentioned operations. Those skilled in the art should understand that any operation capable of achieving the above functions falls within the protection scope of the embodiments of this application.
  • the target text for generating the target multimedia content is determined directly by the terminal device based on the selected text. For example, the terminal device directly determines the selected text as the target text for generating the target multimedia content.
  • the target text for generating the target multimedia content is indirectly determined by the terminal device based on the selected text.
  • the terminal device uses a sentence segmentation algorithm to analyze the selected text, segments the selected text into sentences, removes incomplete sentences from the selected text, and determines the remaining complete sentences as the target text for generating the target multimedia content.
  • the method provided in this embodiment allows the terminal device to display a selected function bar, enabling the user to determine the target text for generating target multimedia content based on triggering the generation control in the selected function bar.
  • the generation control By setting the generation control, the user can choose whether to generate target multimedia content according to their needs.
  • the plurality of texts includes a first text that is highlighted, the highlighting state indicating that the first text supports multimedia content generation functionality.
  • step 241 above can also be replaced by the following sub-step:
  • Step 2411 In response to the selection operation of the first text, the terminal device displays the first text as the selected text and displays the selected function bar.
  • the selection operation for the first text includes at least one of the following: single click, double click, left/right swipe, up/down swipe, long press, hover, facial recognition, and voice recognition. It is worth noting that the selection operation for the first text includes, but is not limited to, several of the above-mentioned operations. Those skilled in the art should understand that any operation capable of achieving the above functions falls within the protection scope of the embodiments of this application.
  • the terminal device in response to a selection operation on the first text, switches the first text from a first display state to a second display state.
  • the first display state is an unselected state
  • the second display state is a selected state.
  • the terminal device displays a selection bar in response to a selection operation on the first text.
  • the selection bar includes a generate control.
  • creators of text-based media content need to enable the multimedia content generation function before publishing the text-based media content.
  • the multimedia content generation function enabled, other users can select at least one character from a plurality of texts within the text-based media content. If the creator of the text-based media content has not enabled the multimedia content generation function, it indicates that the creator does not wish their content to be used to generate relevant target multimedia content; in this case, it is impossible to generate target multimedia content by selecting at least one character from a plurality of texts within the text-based media content.
  • a function switch control 16 is displayed on the text media content editing interface 15.
  • This function switch control 16 includes two options: "On” and "Off".
  • the multimedia content generation function is turned on or off, i.e., the poster generation function is started or stopped.
  • the multimedia content generation function is turned on (i.e., the poster generation function is turned on).
  • the multimedia content generation function is turned off (i.e., the poster generation function is turned off).
  • the editing interface 15 also displays a list of related materials 17.
  • at least one related material in the list of related materials 17 is automatically recommended by the backend server based on the text media content created by the creator.
  • the list of related materials 17 includes at least one related material, which is displayed in order of its matching degree with the text media content.
  • the first text is determined by the backend server from the text media content based on the matching degree between the associated material and the text media content. For example, as shown in FIG9, at least one matching target text 18 is displayed on the text media content editing interface 15. The at least one target text 18 is text within the text media content. The at least one target text 18 includes the aforementioned first text.
  • target text in response to an editing operation on any target text in at least one target text, either target text is retained or any target text is deleted.
  • the method provided in this embodiment enables users to quickly select text that can generate target multimedia content by directly triggering the first text that supports multimedia content generation, which helps to further accelerate the generation of target multimedia content.
  • the first text is determined by the creator of the text-based media content, it enables users to adapt to the requirements of the text-based media creator when creating target multimedia content based on the text-based media content, thus preserving the creator's ideas to a certain extent and respecting the creator's choices.
  • the user can choose to share the target multimedia content with the target object.
  • the above method further includes the following steps:
  • Step 320 Display the sharing control
  • the share control is used to trigger the sharing of target multimedia content.
  • the sharing control and the target multimedia content are displayed simultaneously.
  • the sharing control 22 is displayed at the same time as the target multimedia content 21.
  • the sharing control is displayed after the target multimedia content has already been displayed. That is, the target multimedia content and the sharing control are displayed sequentially.
  • Step 340 In response to the triggering operation of the sharing control, share the target multimedia content to the specified network location.
  • the triggering operation for the sharing control includes at least one of the following: single click, double click, left/right swipe, up/down swipe, long press, hover, facial recognition, and voice recognition. It is worth noting that the triggering operation for the sharing control includes, but is not limited to, several of the above-mentioned operations. Those skilled in the art should understand that any operation capable of achieving the above functions falls within the protection scope of the embodiments of this application.
  • Sharing controls allow users to share target multimedia content, expanding its reach. Furthermore, sharing can facilitate the promotion of the corresponding TV series or shows.
  • the terminal device may also prompt the user with some information to indicate that the target multimedia content is being generated, or that the target multimedia content has been successfully generated, or that the target multimedia content has failed to be generated.
  • a first prompt message is displayed in response to the selection of at least one character among a plurality of texts. In some embodiments, the first prompt message is displayed in response to a triggering operation on a generation control. The first prompt message is used to indicate that target multimedia content is being generated. For example, as shown in FIG12, the first prompt message 23 is displayed.
  • a second prompt message is displayed to indicate that the generation of target multimedia content has failed.
  • the second prompt message instructs the user to reselect at least one character from a plurality of texts.
  • a third prompt message is displayed to indicate that the target multimedia content has been successfully generated.
  • first, second, and third prompts users can be promptly informed of the generation status of the target multimedia content. This helps users stay informed about the current progress and perform subsequent actions based on the corresponding prompts.
  • Figure 13 is a schematic diagram of a media content generation method provided in an exemplary embodiment of this application.
  • the generation process of the target multimedia content is described in detail.
  • the target multimedia content is generated by a server or a client.
  • the method is described using the server 140 shown in Figure 1 as an example. The method includes:
  • Step 420 Receive the target text sent by the client for generating the target multimedia content
  • At least one text selected by the user of the client is received. That is, the selected text sent by the client is received.
  • the target text is selected from text media content published by a media account. That is, the target text is the selected text.
  • the target text is determined by the server based on the selected text.
  • the selected text is analyzed using a sentence segmentation algorithm, which segments the selected text into sentences, removes incomplete sentences from the selected text, and determines the remaining complete sentences as the target text for generating the target multimedia content.
  • the target text is a first text highlighted in the text media content published by the media account, the highlighted state being used to indicate that the first text supports multimedia content generation functionality.
  • the first text is determined based on the degree of matching between the associated episode material and the text media content.
  • the associated TV series materials related to the text media content are determined based on the similarity between the semantic text of the text media content and the semantic text of the TV series dialogue subtitles in the associated TV series materials.
  • the semantic similarity between the semantic text of the text media content and the semantic text of the TV series dialogue subtitles in the associated TV series materials is calculated, and TV series materials with a semantic similarity greater than or equal to a semantic similarity threshold are considered as associated TV series materials related to the text media content.
  • the associated TV series materials can also be sorted according to the magnitude of semantic similarity.
  • the text media content is segmented into sentences.
  • the segmented sentences are then matched against subtitles from related TV series footage.
  • the similarity between the segmented sentences and the subtitles is calculated. If the text similarity between the segmented sentences and the subtitles is greater than or equal to a text similarity threshold, the segmented sentence is designated as the first text.
  • the first text can be sorted according to the magnitude of text similarity.
  • the information related to the first text is sent synchronously to the relevant users when the media account publishes text media content.
  • the display method of the first text on the client is different from the display method of other texts in the text media content. For example, the first text is highlighted, while other texts in the text media content are displayed normally.
  • the information related to the first text includes at least one of the following: display information related to the first text, links related to the first text, associated TV series material related to the first text, and associated target multimedia content related to the first text.
  • the display information related to the first text indicates how the first text is displayed on the client.
  • the links related to the first text indicate that the first text supports multimedia content generation.
  • the user in response to a triggering operation on the first text, the user can be redirected to a multimedia content generation interface.
  • the associated TV series material related to the first text indicates the associated TV series material corresponding to the first text, which is used to generate target multimedia content related to the first text.
  • the associated target multimedia content related to the first text refers to pre-generated target multimedia content. In this case, when the user triggers the first text, the associated target multimedia content can be directly obtained.
  • server or client that determines the first text described in the embodiments of this application may be the same as or different from the server or client that generates the multimedia content. This embodiment of the application does not impose any limitations on this.
  • the target media content includes at least one of animation, poster, audio, and video.
  • Step 440 Generate target multimedia content based on the target text
  • the target text is matched with corresponding episode scenes and dialogue within a drama series database.
  • This drama series database is pre-generated by the server.
  • the server pre-segments different drama series to obtain a database containing episode scenes and dialogue. It should be understood that the episode scenes and dialogue in this embodiment are correlated. Typically, multiple episode scenes may correspond to the same dialogue.
  • target multimedia content including episode images and episode dialogues is generated based on episode images and episode dialogues that match the target text.
  • Step 460 Send the target multimedia content to the client.
  • the method provided in this embodiment offers an automatic way to generate target multimedia content. Once the target text is determined, the target multimedia content generated based on the target text can be directly displayed. Compared to methods where users manually generate target multimedia content, users do not need to perform multiple steps and can generate target multimedia content matching the target text with a single click, thus improving the efficiency of target multimedia content generation.
  • the target media content includes episode scenes and episode dialogue that match the target text.
  • the target multimedia content may also include promotional content provided by the promoter.
  • the target media content may also include advertising information to be promoted provided by the advertiser.
  • step 440 can also be replaced by the following sub-steps:
  • Step 441 Based on the target text, query the TV series images and lines that match the target text in the preset correspondence.
  • the TV series images and lines are used to generate the target multimedia content.
  • the preset correspondence is determined in advance by the server. For example, taking a first episode as an example, the preset correspondence between the episode scenes and dialogue included in the first episode is determined by the following method. As shown in Figure 15, before step 441 above, the method further includes:
  • Step 520 Obtain the first episode video and episode dialogue subtitles
  • the video of the first episode and the subtitles of the episode's dialogue corresponding to the first episode are obtained.
  • the video of the first episode and the subtitles of the episode's dialogue are obtained from a video database.
  • Step 540 Divide the first episode video into at least one video segment according to the start and end times of the subtitles in the episode dialogue;
  • the first episode video is divided into at least one video segment according to the start and end times of the subtitles corresponding to the dialogue subtitles of the same episode. It should be understood that the first number of video segments is greater than or equal to the second number of episode dialogue subtitles.
  • the first episode video corresponds to two episode dialogue subtitles, including episode dialogue subtitle 1 and episode dialogue subtitle 2.
  • the video segments corresponding to episode dialogue subtitle 1 and episode dialogue subtitle 2 are not consecutive. That is, as shown in Figure 16, there is a video segment without subtitles between episode dialogue subtitle 1 and episode dialogue subtitle 2.
  • each video segment corresponds to one drama series subtitle.
  • the first drama series video can be divided into two video segments, including video segment 1 and video segment 2. Among them, video segment 1 corresponds to drama series subtitle 1, and video segment 2 corresponds to drama series subtitle 2.
  • Step 560 Extract at least one episode frame from the video segments
  • the video is divided into segments, with each segment consisting of a video frame.
  • Each video segment is divided into at least one video frame. Scenes from each video frame are then extracted. It is important to understand that there is a one-to-one correspondence between video frames and scenes from each video frame.
  • At least one target video frame is extracted from the video slice, and the quality of each target video frame is greater than or equal to a quality threshold. For example, suppose video slice 1 includes video frame 1, video frame 2, and video frame 3. Where the quality of video frame 1 and video frame 2 is greater than the quality threshold, and the quality of video frame 3 is less than the quality threshold, then only video frame 1 and video frame 2 are extracted from video slice 1, while video frame 3 is ignored.
  • Step 580 Generate a preset correspondence based on at least one episode scene and episode dialogue subtitles.
  • a preset correspondence between episode scenes and episode dialogue is generated based on at least one extracted episode scene. Examples are shown in Table 1 below:
  • episode image 1 corresponds to episode dialogue 1.
  • episode image 2 and episode image 3 both correspond to episode dialogue 2.
  • the server can, after determining the target text, query the episode scenes and episode dialogue that match the target text within the pre-determined correspondence, which helps improve matching efficiency.
  • the TV series scenes and dialogue that match the target text are determined.
  • the TV series dialogue is used to connect the TV series dialogue and the target text.
  • step 440 can also be replaced by the following sub-steps:
  • Step 442 Calculate the first feature vector corresponding to the target text, and calculate the vector similarity between the first feature vector and at least one second feature vector; determine the TV scene and TV line corresponding to the second feature vector with the highest vector similarity as the TV scene and TV line that match the target text, and use the TV scene and TV line to generate target multimedia content.
  • the episode scenes and episode lines corresponding to the second feature vector with a vector similarity greater than or equal to the similarity threshold are all determined as episode scenes and episode lines that match the target text.
  • the second feature vector is predetermined by the server.
  • the second feature vector is determined by the following method. As shown in Figure 18, prior to step 442 above, the method further includes:
  • Step 610 Obtain the video of the second episode
  • the second episode video is obtained from a video database.
  • Step 620 Divide the second episode video into at least one video segment according to the division step size
  • the step size is random. In some embodiments, the step size is preset. For example, the step size is 3 seconds.
  • the second episode video is continuously divided according to a division step size. That is, at least one video segment in the embodiments of this application is continuous. It can be understood that combining at least one video segment in sequence can obtain a complete second episode video.
  • Step 630 Extract computer vision information for each video segment from at least one video segment
  • computer vision algorithms are used to extract computer vision information from each video segment.
  • the computer vision information includes subject information and/or text information in each video segment.
  • Subject information includes human body information, object information, landscape information, etc.
  • Step 640 Based on the computer vision information of each video segment, generate content description information for each video segment;
  • content description information for each video segment is generated based on the extracted computer vision information for each video segment.
  • content description information for each video segment in at least one video segment is obtained based on a multimodal visual recognition model. In this case, it is unnecessary to perform step 630 above to obtain the computer vision information for each video segment. That is, steps 630 and 640 can be implemented directly based on the multimodal visual recognition model.
  • Step 650 Generate a second feature vector for each video segment based on the content description information of each video segment.
  • a second feature vector for each video segment is generated based on the content description information of each video segment and a vector database.
  • the vector database includes the correspondence between the content description information and the feature vectors.
  • a second feature vector for each video segment is generated based on a multimodal embedding algorithm.
  • a multimodal embedding algorithm based on a multimodal embedding algorithm, at least two of the audio, image, and text information corresponding to each video segment are input to generate a second feature vector corresponding to each video segment.
  • steps 630 and 640 above are unnecessary. That is, steps 630 to 650 above can be implemented directly based on the multimodal embedding algorithm.
  • the server can determine the target text and then identify the matching episode scenes and dialogue based on the target text, which helps improve matching efficiency.
  • the method further includes:
  • Step 720 Extract at least one episode frame from the video segments
  • the video is divided into segments, with each segment consisting of a video frame.
  • Each video segment is divided into at least one video frame. Scenes from each video frame are then extracted. It is important to understand that there is a one-to-one correspondence between video frames and scenes from each video frame.
  • At least one target video frame is extracted from the video slice, and the quality of each target video frame is greater than or equal to a quality threshold. For example, suppose video slice 1 includes video frame 1, video frame 2, and video frame 3. Where the quality of video frame 1 and video frame 2 is greater than the quality threshold, and the quality of video frame 3 is less than the quality threshold, then only video frame 1 and video frame 2 are extracted from video slice 1, while video frame 3 is ignored.
  • Step 740 Store the correspondence between at least one episode frame and the second feature vector of the video segment.
  • the second feature vector corresponding to each video segment and at least one episode frame corresponding to each video segment are known, there is a correspondence between the at least one episode frame and the second feature vector. Examples are shown in Table 2 below:
  • episode frame 1 corresponds to second feature vector 1.
  • episode frame 1 and episode frame 2 both correspond to the second feature vector 2.
  • episode frame 1 corresponds to both second feature vector 1 and second feature vector 2.
  • the method provided in this embodiment by storing the correspondence between at least one episode frame and the second feature vector of a video segment, enables the server to quickly find the episode frame corresponding to the target text after obtaining the target text, based on the correspondence between at least one episode frame and the second feature vector, as well as the first feature vector corresponding to the target text.
  • step 560 or step 720 above can be replaced by the following sub-steps:
  • Step 820 Score each video frame in the video segment according to at least one scoring factor to obtain the quality score corresponding to each video frame;
  • each video segment includes multiple video frames, and each of the multiple video frames corresponds to a quality score.
  • At least one scoring factor includes at least one of brightness, sharpness, and confidence in the identification of the subject in the video frame.
  • Brightness refers to the light intensity in the video frame.
  • Sharpness refers to the clarity of the image in the video frame.
  • Confidence in the identification of the subject in the video frame refers to the probability that the subject in the video frame is identified.
  • Step 840 Based on the quality score, extract at least one episode frame from each video frame in the video segment.
  • At least one episode frame is extracted from the target video frame in the video segment based on the quality score of each video frame.
  • the quality score corresponding to the target video frame is greater than or equal to a quality threshold score.
  • the video frames in the video segment are sorted from high to low or from low to high according to their quality scores, and at least one video frame with a quality score greater than or equal to a quality threshold score is selected. At least one episode scene is extracted from the at least one video frame with a quality score greater than or equal to the quality threshold score.
  • the method provided in this embodiment scores each video frame in each video segment, enabling the server to extract episode scenes based on preferred video frames. This helps ensure that the brightness, clarity, or confidence level of the main subject in the extracted video scenes are all better than the threshold.
  • Step 443 If there are at least two candidate episode frames in the target text match, calculate the image quality score of at least two candidate episode frames;
  • each candidate episode frame is scored according to at least one scoring factor to obtain a picture quality score for each candidate episode frame.
  • the at least one scoring factor includes at least one of brightness, sharpness, and confidence in the identification of the subject in the picture.
  • At least one target episode image is selected from at least two candidate episode images that match the target text.
  • the image quality score corresponding to the target episode image is greater than or equal to an image quality score threshold.
  • Step 445 Generate target multimedia content based on the target episode images and the episode dialogue that matches the target text.
  • the target episode image includes at least one, and the episode dialogue matching the target text may also include at least one.
  • each episode image corresponds to its own episode dialogue; that is, there is a one-to-one correspondence between episode images and episode dialogue.
  • the target text matches the first episode image of a first series and the second episode image of a second series.
  • the first and second series are two different series.
  • the first episode image corresponds to episode dialogue 1
  • the second episode image corresponds to episode dialogue 2.
  • the candidate episode frame is determined as the target episode frame. That is, if there is one and only one matching candidate episode frame for the target text, that candidate episode frame is the target episode frame used to generate the target multimedia content.
  • the method provided in this embodiment calculates the picture quality score of each candidate TV series picture, enabling the server to generate target multimedia content based on the preferred candidate TV series picture, thereby helping to ensure that the quality of the TV series picture corresponding to the generated target multimedia content is relatively good.
  • the target multimedia content is generated based on cropped episode footage. Based on the embodiment shown in FIG13 above, as shown in FIG22, prior to step 460, the method further includes:
  • Step 920 Identify the bounding box of the main subject in the scene.
  • the main subjects in a scene are assigned a priority. Based on the priority of the main subjects in the scene, the main subjects in the scene are identified, and a bounding box is used to select the corresponding main subject.
  • the main subject of the image includes at least one of a face, a human body, an animal, a plant, furniture, clutter, and scenery.
  • the priority of a face is higher than that of a human body
  • the priority of a human body is higher than that of an animal
  • the priority of an animal is higher than that of a plant
  • the priority of a plant is higher than that of furniture
  • the priority of furniture is higher than that of clutter
  • the priority of clutter is higher than that of scenery.
  • the main subject 24 in the scene of the drama is identified, and the corresponding main subject 24 is selected by the bounding box 25.
  • the bounding boxes of the subjects of the same priority in the scene are identified simultaneously. For example, assuming that there are multiple faces in the same scene, multiple faces are identified simultaneously, and multiple faces are selected by bounding boxes.
  • the episode frame is cropped based on the bounding box of the identified main subject to obtain the cropped episode frame.
  • the cropped episode frame is smaller than or equal to the original episode frame.
  • Step 941 If at least two bounding boxes are identified based on the main subject of the image, merge the at least two bounding boxes to obtain a merged bounding box;
  • the identified at least two bounding boxes are merged to obtain a merged bounding box.
  • the area of the merged bounding box is determined based on the merged at least two bounding boxes.
  • At least one group of bounding boxes to be merged is selected based on the priorities corresponding to the at least two different subjects.
  • the number of bounding boxes in a group does not exceed a second threshold, and the priority of the bounding boxes in the group is higher than a third threshold.
  • the number of main subjects in the final episode frame cropped based on the bounding boxes is also limited to a certain threshold. This helps to ensure that the number of main subjects in the cropped episode frame is reasonable and avoids a cluttered overall picture.
  • the maximum top-left coordinate, maximum top-right coordinate, minimum bottom-left coordinate, and minimum bottom-right coordinate in the bounding box group are determined. Based on the maximum top-left coordinate, maximum top-right coordinate, minimum bottom-left coordinate, and minimum bottom-right coordinate, a merged bounding box for each bounding box group is generated.
  • each bounding box corresponds to a top-left coordinate, a top-right coordinate, a bottom-left coordinate, and a bottom-right coordinate.
  • each bounding box can be determined by a combination of (x start , y start ) and (x end , y end ).
  • (x start , y start ) represents the bottom-left coordinate
  • (x start , y end ) represents the top-left coordinate
  • (x end , y start ) represents the bottom-right coordinate
  • (x end , y end ) represents the top-right coordinate.
  • the merging of the bounding boxes can be determined by min(x1 start , x2 start , ..., xn start ), min(y1 start , y2 start , ..., yn start ), max(x1 end , x2 end , ..., xn end ), max(y1 end , y2 end , ..., yn end ).
  • Step 942 Crop the TV series image based on the merged bounding box to obtain the cropped TV series image.
  • the episode frame is cropped based on a defined fusion bounding box to obtain a cropped episode frame.
  • the cropped episode frame is smaller than or equal to the original episode frame.
  • the server can generate target multimedia content based on the cropped episode footage, thus ensuring that the episode footage corresponding to the generated target multimedia content is based on the optimized episode footage obtained from the cropping.
  • the main subject of the screen can be displayed reasonably, making the generated target multimedia content more aesthetically pleasing and increasing its attractiveness.
  • the episode frame may not be able to completely fill the entire content area.
  • the theme color of the episode frame can be extracted and used to fill the area outside the episode frame within the content area, which helps to make the generated target multimedia content more natural.
  • the display size of the dialogue in a TV series is dynamically adjusted based on the text content corresponding to the dialogue.
  • the display size of each character in the dialogue is dynamically adjusted based on the number of characters in the text content corresponding to the dialogue and the size of the text area used to display the dialogue.
  • the size of the text area occupied by each character in the dialogue is equal to the quotient of the size of the text area used to display the dialogue and the number of characters in the text content corresponding to the dialogue. For example, assuming the size of the text area used to display the dialogue is 10 square centimeters and the number of characters in the text content corresponding to the dialogue is 10, then the size of the text area occupied by each character in the dialogue is 1 square centimeter.
  • the display size of each character in the dialogue is directly proportional to the size of the text area occupied by each character.
  • the goal is to make the dialogue in the TV series adaptably displayed in the text area of the target multimedia content. This means avoiding both excessively large display sizes of the dialogue, which would make the overall image of the target multimedia content look unattractive, and excessively small display sizes, which would result in poor clarity of the dialogue.
  • the promotional content is pre-set. For example, before the function of generating target multimedia content with one click is enabled, the promoter pre-determines the promotional content to be promoted. Alternatively, after the function of generating target multimedia content with one click is enabled, the promotional content that the promoter needs to promote can be added in real time during the process of users using this function to generate target multimedia content. This facilitates the dynamic adjustment of the promotional content in the target multimedia content.
  • the promotional content is displayed overlaid with the aforementioned TV series footage.
  • the promotional content is displayed in a manner that does not obscure the main subject of the TV series footage.
  • promotional content is displayed as a display element within the aforementioned episode scene.
  • the promotional content is displayed in combination with at least one main subject within the episode scene.
  • the promotional content is a decorative element of at least one main subject.
  • the promotional content is a scene object within the scene depicted in the episode scene.
  • the promotional content is displayed in combination with the aforementioned TV drama dialogue.
  • the promotional content is an advertisement
  • the promotional content and the TV drama dialogue are merged to obtain merged text.
  • the merged text is displayed in the display area where the TV drama dialogue is displayed.
  • the promotional content is displayed at the start of the target multimedia content, or at the end of the target multimedia content.
  • a media content generation method proposed in this application can be used to generate electronic posters.
  • the method is described using the terminal device 120 shown in FIG1 as an example. The method includes:
  • Step 240-1 In response to the selection of at least one character among several texts, determine the target text for generating the target e-poster;
  • the selection operation for at least one character among several texts includes at least one of the following: single click, double click, left/right swipe, up/down swipe, long press, hover, facial recognition, and voice recognition. It is worth noting that the selection operation for at least one character among several texts includes, but is not limited to, the aforementioned methods. Those skilled in the art should understand that any operation capable of achieving the above functions falls within the protection scope of the embodiments of this application.
  • a generation request for generating a target e-poster in response to a selection operation of at least one character among a plurality of texts, is sent to the server.
  • the generation request for generating the target e-poster includes a request to determine target text based on the selected at least one character.
  • the generation request for generating the target e-poster further includes a request to generate the target e-poster based on the target text.
  • the target text is key information in the selected at least one character.
  • the display state of the selected at least one character in response to a selection operation on at least one character among a plurality of texts, is switched from a first display state to a second display state.
  • the first display state is a state where at least one character is not selected
  • the second display state is a state where at least one character is selected.
  • Step 260-1 Display the target e-poster generated based on the target text.
  • the target digital poster includes images and dialogue from the TV series that match the target text.
  • the content of the scene corresponding to the target text matches the text content expressed by the target text.
  • the meaning of the content of the scene corresponding to the target text is consistent with the meaning of the text content expressed by the target text. For example, assuming the target text is "xxx is so beautiful", and the content of scene 1 is "xxx has gorgeous makeup”, then scene 1 is considered to match the target text.
  • the text content corresponding to the TV drama dialogue that matches the target text matches the text content expressed by the target text.
  • the meaning of the text content corresponding to the TV drama dialogue that matches the target text is consistent with the meaning of the text content expressed by the target text.
  • the matching degree between the first text content corresponding to the TV drama dialogue that matches the target text and the second text content expressed by the target text is greater than or equal to a matching degree threshold. For example, assuming the target text is "xxx is really beautiful" and the TV drama dialogue is "xxx is really good-looking", since "beautiful" and "good-looking” express the same meaning, the TV drama dialogue is considered to match the target text.
  • the target e-poster is generated by the server based on a generation request sent by the terminal device. Based on the generation request, the server determines the episode scenes and dialogue that match the target text, and then generates the target e-poster. Afterward, the server sends the generated target e-poster to the terminal device, which receives and displays the target e-poster.
  • the target e-poster displayed on the terminal device is the one with the highest matching degree to the target text. That is, the server may generate multiple target e-posters based on the target text, but the matching degree between the multiple target e-posters and the target text is different. For example, assuming that the matching degree between target e-poster 1 and the target text is 90% and the matching degree between target e-poster 2 and the target text is 80%, the server will only send the target e-poster 1 with the highest matching degree to the terminal device.
  • the target e-posters displayed on the terminal device are all target e-posters that match the target text.
  • the target e-posters displayed on the terminal device are those whose matching degree with the target text is greater than a matching degree threshold. For example, assuming that target e-poster 1 has a matching degree of 90% with the target text, target e-poster 2 has a matching degree of 80% with the target text, and target e-poster 3 has a matching degree of 60% with the target text, and the matching degree threshold is 75%, then the server will only send target e-poster 1 and target e-poster 2 to the terminal device.
  • the material corresponding to the target electronic poster is a static image. In some embodiments, the material corresponding to the target electronic poster is a dynamic video frame.
  • the targeted e-poster may also include promotional content provided by the promoter.
  • the targeted media content may also include advertising information to be promoted provided by the advertiser.
  • the target electronic poster includes a side-by-side upper and lower half, with the upper half displaying images from the TV series and the lower half displaying dialogue from the TV series.
  • the target electronic poster includes a side-by-side upper and lower half, with the upper half displaying dialogue from the TV series and the lower half displaying visuals from the TV series.
  • the target electronic poster includes a left half and a right half side by side, with the left half displaying dialogue from the TV series and the right half displaying visuals from the TV series.
  • the target electronic poster includes a left half and a right half side by side, with the left half displaying images from the show and the right half displaying dialogue from the show.
  • the target electronic poster includes a content area.
  • the content area displays screen images of the TV series, as well as dialogue overlaid on the screen images.
  • the dialogue may be displayed separately from the screen images. This application does not limit the display method of the screen images and dialogue in this embodiment.
  • the target e-poster also includes an information area.
  • the information area is used to display information and/or target text related to the show.
  • the target electronic poster is automatically generated by a client or a server.
  • the target electronic poster is generated by a server, or by a terminal different from the terminal device, the following steps are included after step 240-1 and before step 260-1:
  • Step 251-1 Send the target text for generating the target electronic poster
  • Step 252-1 Receive the target electronic poster generated based on the target text.
  • Step 240-1 above can also be replaced by the following sub-steps:
  • Step 241-1 In response to the selection of at least one character among a plurality of texts, display the selected text and the selection function bar;
  • the selected text in response to a selection operation on at least one character among a plurality of texts, is switched from a first display state to a second display state.
  • the first display state is an unselected state
  • the second display state is a selected state.
  • a selection bar is displayed in response to the selection of at least one character among a plurality of texts.
  • the selected function bar includes a generate control.
  • the generate control is used to trigger the generation of a target e-poster.
  • a generation request for generating the target e-poster is sent to the server.
  • the generation request for generating the target e-poster includes a request to determine target text based on at least one selected character.
  • the generation request for generating the target e-poster further includes a request to generate the target e-poster based on the target text.
  • the selected function bar may further include other function controls besides the generate control.
  • the selected function bar may further include at least one of a copy control, a share control, and a favorite control.
  • the copy control is used to copy at least one selected text.
  • the share control is used to share at least one selected text.
  • the favorite control is used to favorite at least one selected text.
  • Step 242-1 In response to the triggering operation of the generation control, the selected text is determined as the target text for generating the target electronic poster.
  • the triggering operation for the generated control includes at least one of the following: single click, double click, left/right swipe, up/down swipe, long press, hover, facial recognition, and voice recognition. It is worth noting that the triggering operation for the generated control includes, but is not limited to, several of the above-mentioned operations. Those skilled in the art should understand that any operation capable of achieving the above functions falls within the protection scope of the embodiments of this application.
  • the target text for generating the target e-poster is determined directly based on the selected text.
  • the selected text is directly determined as the target text for generating the target e-poster.
  • the target text for generating the target e-poster is determined indirectly based on the selected text.
  • the selected text is analyzed using a sentence segmentation algorithm, which segments the selected text into sentences, removes incomplete sentences from the selected text, and determines the remaining complete sentences as the target text for generating the target e-poster.
  • the text includes a first text that is highlighted, the highlighting state indicating that the first text supports the electronic poster generation function.
  • Step 241-1 above can also be replaced by the following sub-steps:
  • Step 2411-1 In response to the selection operation of the first text, display the first text as the selected text and display the selection function bar.
  • the selection operation for the first text includes at least one of the following: single click, double click, left/right swipe, up/down swipe, long press, hover, facial recognition, and voice recognition. It is worth noting that the selection operation for the first text includes, but is not limited to, several of the above-mentioned operations. Those skilled in the art should understand that any operation capable of achieving the above functions falls within the protection scope of the embodiments of this application.
  • the first text in response to a selection operation on the first text, is switched from a first display state to a second display state.
  • the first display state is an unselected state
  • the second display state is a selected state.
  • a selection bar in response to a selection operation on the first text, a selection bar is displayed.
  • the selection bar includes a generation control.
  • creators of text-based media content need to enable the e-poster generation function before publishing the content.
  • the e-poster generation function enabled, other users can select at least one character from a set of text within the text-based media content. If the creator of the text-based media content has not enabled the e-poster generation function, it indicates that the creator does not wish their content to be used to generate a target e-poster; in this case, it is impossible to generate a target e-poster by selecting at least one character from a set of text within the text-based media content.
  • the user can choose to share the target e-poster with the target object.
  • the method further includes the following steps:
  • Step 320-1 Display the sharing control
  • the share control is used to trigger the sharing of the target e-poster.
  • the sharing control and the target e-poster are displayed simultaneously.
  • the sharing control is displayed after the target e-poster has already been displayed. That is, the target e-poster and the sharing control are displayed in a sequential order.
  • Step 340-1 In response to the triggering operation of the sharing control, share the target electronic poster to the specified network location.
  • the triggering operation for the sharing control includes at least one of the following: single click, double click, left/right swipe, up/down swipe, long press, hover, facial recognition, and voice recognition. It is worth noting that the triggering operation for the sharing control includes, but is not limited to, several of the above-mentioned operations. Those skilled in the art should understand that any operation capable of achieving the above functions falls within the protection scope of the embodiments of this application.
  • the terminal device will also prompt the user with some prompts indicating that the target electronic poster is being generated, the target electronic poster has been successfully generated, or the target electronic poster has failed to be generated.
  • a first prompt message is displayed in response to the selection of at least one character among a plurality of texts. In some embodiments, the first prompt message is displayed in response to a triggering operation on a generation control. The first prompt message indicates that a target electronic poster is being generated.
  • a second prompt message is displayed to indicate that the generation of the target e-poster has failed.
  • the second prompt message instructs the user to reselect at least one character from a plurality of texts.
  • a third prompt message is displayed to indicate that the target electronic poster has been successfully generated.
  • the target electronic poster is generated by a server or a client.
  • the method is illustrated by example, with the method executed by the server 140 shown in Figure 1.
  • the method includes:
  • Step 420-1 Receive the target text sent by the client for generating the target electronic poster
  • At least one text selected by the user of the client is received. That is, the selected text sent by the client is received.
  • the target text is selected from text media content published by a media account. That is, the target text is the selected text.
  • the target text is determined by the server based on the selected text.
  • the selected text is analyzed using a sentence segmentation algorithm, which segments the selected text into sentences, removes incomplete sentences from the selected text, and determines the remaining complete sentences as the target text for generating the target e-poster.
  • the target text is a first text highlighted in the text media content published by the media account, and the highlighted state is used to indicate that the first text supports the electronic poster generation function.
  • the first text is determined based on the degree of matching between the associated episode material and the text media content.
  • the associated TV series materials related to the text media content are determined based on the similarity between the semantic text of the text media content and the semantic text of the TV series dialogue subtitles in the associated TV series materials.
  • the semantic similarity between the semantic text of the text media content and the semantic text of the TV series dialogue subtitles in the associated TV series materials is calculated, and TV series materials with a semantic similarity greater than or equal to a semantic similarity threshold are considered as associated TV series materials related to the text media content.
  • the associated TV series materials can also be sorted according to the magnitude of semantic similarity.
  • the text media content is segmented into sentences.
  • the segmented sentences are then matched against subtitles from related TV series footage.
  • the similarity between the segmented sentences and the subtitles is calculated. If the text similarity between the segmented sentences and the subtitles is greater than or equal to a text similarity threshold, the segmented sentence is designated as the first text.
  • the first text can be sorted according to the magnitude of text similarity.
  • the information related to the first text is sent synchronously to the relevant users when the media account publishes text media content.
  • the display method of the first text on the client is different from the display method of other texts in the text media content. For example, the first text is highlighted, while other texts in the text media content are displayed normally.
  • the information related to the first text includes at least one of the following: display information related to the first text, links related to the first text, associated TV series material related to the first text, and associated target e-posters related to the first text.
  • the display information related to the first text indicates how the first text is displayed on the client.
  • the links related to the first text indicate that the first text supports e-poster generation.
  • the user in response to a triggering operation on the first text, the user can be redirected to a poster generation interface.
  • the associated TV series material related to the first text indicates the associated TV series material corresponding to the first text, which is used to generate the target e-poster related to the first text.
  • the associated target e-poster related to the first text refers to a pre-generated target e-poster. In this case, when the user triggers the first text, the associated target e-poster can be directly obtained.
  • server or client for determining the first text described in this application embodiment may be the same as or different from the server or client for generating the poster. This application embodiment does not impose any limitation on this.
  • the target media content includes at least one of animation, poster, audio, and video.
  • Step 440-1 Generate a target electronic poster based on the target text
  • the target text is matched with corresponding episode scenes and dialogue within a drama series database.
  • This drama series database is pre-generated by the server.
  • the server pre-segments different drama series to obtain a database containing episode scenes and dialogue. It should be understood that the episode scenes and dialogue in this embodiment are correlated. Typically, multiple episode scenes may correspond to the same dialogue.
  • a target e-poster including episode images and episode lines is generated based on episode images and episode lines that match the target text.
  • Step 460-1 Send the target electronic poster to the client.
  • the target media content includes episode scenes and episode dialogue that match the target text.
  • the targeted e-poster may also include promotional content provided by the promoter.
  • the targeted media content may also include advertising information to be promoted provided by the advertiser.
  • Step 440-1 above can also be replaced by the following sub-steps:
  • Step 441-1 Based on the target text, query the TV series images and lines that match the target text in the preset correspondence. The TV series images and lines are used to generate the target electronic poster.
  • the preset correspondence is determined in advance by the server. For example, taking a first TV series as an example, the preset correspondence between the episode scenes and dialogue included in the first TV series is determined by the following method. Before step 441-1 above, the method further includes:
  • Step 520-1 Obtain the first episode video and episode dialogue subtitles
  • the video of the first episode and the subtitles of the episode's dialogue corresponding to the first episode are obtained.
  • the video of the first episode and the subtitles of the episode's dialogue are obtained from a video database.
  • Step 540-1 Divide the first episode video into at least one video segment according to the start and end times of the subtitles in the episode dialogue;
  • the first episode video is divided into at least one video segment according to the start and end times of the subtitles corresponding to the dialogue subtitles of the same episode. It should be understood that the first number of video segments is greater than or equal to the second number of episode dialogue subtitles.
  • each video segment corresponds to one TV series dialogue subtitle.
  • Step 560-1 Extract at least one episode frame from the video segments
  • the video is divided into segments, with each segment consisting of a video frame.
  • Each video segment is divided into at least one video frame. Scenes from each video frame are then extracted. It is important to understand that there is a one-to-one correspondence between video frames and scenes from each video frame.
  • At least one target video frame is extracted from the video slice, and the quality of each target video frame is greater than or equal to a quality threshold.
  • Step 580 Generate a preset correspondence based on at least one episode scene and episode dialogue subtitles.
  • a preset correspondence between episode scenes and episode dialogue is generated based on at least one extracted episode scene.
  • the TV series scenes and dialogue that match the target text are determined.
  • the TV series dialogue is used to connect the TV series dialogue and the target text.
  • Step 440-1 above can also be replaced by the following sub-steps:
  • Step 442-1 Calculate the first feature vector corresponding to the target text, and calculate the vector similarity between the first feature vector and at least one second feature vector; determine the TV scene and TV line corresponding to the second feature vector with the highest vector similarity as the TV scene and TV line matching the target text, and use the TV scene and TV line to generate the target electronic poster.
  • the episode scenes and episode lines corresponding to the second feature vector with a vector similarity greater than or equal to the similarity threshold are all determined as episode scenes and episode lines that match the target text.
  • the second feature vector is predetermined by the server. For example, taking the second episode as an example, the second feature vector is determined by the following method. Prior to step 442-1 above, the method further includes:
  • Step 610-1 Obtain the video of the second episode
  • the second episode video is obtained from a video database.
  • Step 620-1 Divide the second episode video into at least one video segment according to the division step size
  • the step size is random. In some embodiments, the step size is preset. For example, the step size is 3 seconds.
  • the second episode video is continuously divided according to a division step size. That is, at least one video segment in the embodiments of this application is continuous. It can be understood that combining at least one video segment in sequence can obtain a complete second episode video.
  • Step 630-1 Extract computer vision information for each video segment from at least one video segment
  • computer vision algorithms are used to extract computer vision information from each video segment.
  • the computer vision information includes subject information and/or text information in each video segment.
  • Subject information includes human body information, object information, landscape information, etc.
  • content description information for each video segment is generated based on the extracted computer vision information for each video segment.
  • content description information for each video segment in at least one video segment is obtained based on a multimodal visual recognition model.
  • steps 630-1 and 640-1 can be implemented directly based on the multimodal visual recognition model.
  • Step 650-1 Generate a second feature vector for each video segment based on the content description information of each video segment.
  • a second feature vector for each video segment is generated based on the content description information of each video segment and a vector database.
  • the vector database includes the correspondence between the content description information and the feature vectors.
  • a second feature vector for each video segment is generated based on a multimodal embedding algorithm.
  • a multimodal embedding algorithm based on a multimodal embedding algorithm, at least two of the audio, image, and text information corresponding to each video segment are input to generate a second feature vector corresponding to each video segment.
  • steps 630-1 and 640-1 above do not need to be performed. That is, steps 630-1 to 650-1 above can be implemented directly based on the multimodal embedding algorithm.
  • the above method further includes:
  • Step 720-1 Extract at least one episode frame from the video segments
  • the video is divided into segments, with each segment consisting of a video frame.
  • Each video segment is divided into at least one video frame. Scenes from each video frame are then extracted. It is important to understand that there is a one-to-one correspondence between video frames and scenes from each video frame.
  • At least one target video frame is extracted from the video slice, and the quality of each target video frame is greater than or equal to a quality threshold.
  • Step 740-1 Store the correspondence between at least one episode frame and the second feature vector of the video segment.
  • the second feature vector corresponding to each video segment and at least one episode frame corresponding to each video segment are known, there is a correspondence between at least one episode frame and the second feature vector.
  • step 560-1 or step 720-1 above can be replaced by the following sub-steps:
  • Step 820-1 Score each video frame in the video segment according to at least one scoring factor to obtain the quality score corresponding to each video frame;
  • each video segment includes multiple video frames, and each of the multiple video frames corresponds to a quality score.
  • At least one scoring factor includes at least one of brightness, sharpness, and confidence in the identification of the subject in the video frame.
  • Brightness refers to the light intensity in the video frame.
  • Sharpness refers to the clarity of the image in the video frame.
  • Confidence in the identification of the subject in the video frame refers to the probability that the subject in the video frame is identified.
  • Step 840-1 Based on the quality score, extract at least one episode frame from each video frame in the video segment.
  • At least one episode frame is extracted from the target video frame in the video segment based on the quality score of each video frame.
  • the quality score corresponding to the target video frame is greater than or equal to a quality threshold score.
  • the video frames in the video segment are sorted from high to low or from low to high according to their quality scores, and at least one video frame with a quality score greater than or equal to a quality threshold score is selected. At least one episode scene is extracted from the at least one video frame with a quality score greater than or equal to the quality threshold score.
  • the target e-poster is generated based on the corresponding scenes and dialogue from the TV series that match the target text.
  • Step 440-1 above can also be replaced by the following sub-steps:
  • Step 443-1 If there are at least two candidate episode frames in the target text match, calculate the image quality score of at least two candidate episode frames;
  • each candidate episode frame is scored according to at least one scoring factor to obtain a picture quality score for each candidate episode frame.
  • the at least one scoring factor includes at least one of brightness, sharpness, and confidence in the identification of the subject in the picture.
  • Step 444-1 Select the target episode from at least two candidate episodes based on the picture quality score
  • At least one target episode image is selected from at least two candidate episode images that match the target text.
  • the image quality score corresponding to the target episode image is greater than or equal to an image quality score threshold.
  • Step 445-1 Generate a target e-poster based on the target TV series images and the TV series dialogue that matches the target text.
  • the target episode image includes at least one, and the episode dialogue matching the target text may also include at least one.
  • each episode image corresponds to its own episode dialogue; that is, there is a one-to-one correspondence between episode images and episode dialogue.
  • the target text matches the first episode image of a first series and the second episode image of a second series.
  • the first and second series are two different series.
  • the first episode image corresponds to episode dialogue 1
  • the second episode image corresponds to episode dialogue 2.
  • the candidate episode image is determined as the target episode image. That is, if there is one and only one matching candidate episode image for the target text, that candidate episode image is the target episode image used to generate the target e-poster.
  • the target digital poster is generated based on cropped episode footage.
  • the method further includes:
  • Step 920-1 Identify the bounding box of the main subject in the scene
  • the main subjects in a scene are assigned a priority. Based on the priority of the main subjects in the scene, the main subjects in the scene are identified, and a bounding box is used to select the corresponding main subject.
  • the main subject of the image includes at least one of a face, a human body, an animal, a plant, furniture, clutter, and scenery.
  • the priority of a face is higher than that of a human body
  • the priority of a human body is higher than that of an animal
  • the priority of an animal is higher than that of a plant
  • the priority of a plant is higher than that of furniture
  • the priority of furniture is higher than that of clutter
  • the priority of clutter is higher than that of scenery.
  • the bounding boxes of the subjects of the same priority in the scene are identified simultaneously. For example, assuming that there are multiple faces in the same scene, multiple faces are identified simultaneously, and multiple faces are selected by bounding boxes.
  • Step 940-1 Based on the bounding box of the main subject of the image, crop the screen of the series to obtain the cropped screen of the series.
  • the episode frame is cropped based on the bounding box of the identified main subject to obtain the cropped episode frame.
  • the cropped episode frame is smaller than or equal to the original episode frame.
  • step 940-1 above can be replaced by the following sub-steps:
  • Step 941-1 If at least two bounding boxes are identified based on the main subject of the image, merge the at least two bounding boxes to obtain a merged bounding box;
  • the identified at least two bounding boxes are merged to obtain a merged bounding box.
  • the area of the merged bounding box is determined based on the merged at least two bounding boxes.
  • the merged bounding box covers all or part of the bounding boxes in at least two bounding boxes.
  • At least one group of bounding boxes to be merged is selected based on the priorities corresponding to the at least two different subjects.
  • the number of bounding boxes in a group does not exceed a second threshold, and the priority of the bounding boxes in the group is higher than a third threshold.
  • the number of main subjects in the final episode frame cropped based on the bounding boxes is also limited to a certain threshold. This helps to ensure that the number of main subjects in the cropped episode frame is reasonable and avoids a cluttered overall picture.
  • the maximum top-left coordinate, maximum top-right coordinate, minimum bottom-left coordinate, and minimum bottom-right coordinate in the bounding box group are determined. Based on the maximum top-left coordinate, maximum top-right coordinate, minimum bottom-left coordinate, and minimum bottom-right coordinate, a merged bounding box for each bounding box group is generated.
  • each bounding box corresponds to a top-left coordinate, a top-right coordinate, a bottom-left coordinate, and a bottom-right coordinate.
  • each bounding box can be determined by a combination of (x start , y start ) and (x end , y end ).
  • (x start , y start ) represents the bottom-left coordinate
  • (x start , y end ) represents the top-left coordinate
  • (x end , y start ) represents the bottom-right coordinate
  • (x end , y end ) represents the top-right coordinate.
  • the merging of the bounding boxes can be determined by min(x1 start , x2 start , ..., xn start ), min(y1 start , y2 start , ..., yn start ), max(x1 end , x2 end , ..., xn end ), max(y1 end , y2 end , ..., yn end ).
  • Step 942-1 Crop the TV series image based on the merged bounding box to obtain the cropped TV series image.
  • the episode frame is cropped based on a defined fusion bounding box to obtain a cropped episode frame.
  • the cropped episode frame is smaller than or equal to the original episode frame.
  • the server can generate the target e-poster based on the cropped footage, thus ensuring that the TV series footage corresponding to the generated target e-poster is based on the optimized footage obtained from the cropping.
  • the method further includes: determining whether the layout of the target e-poster is a portrait or landscape layout based on the aspect ratio of the cropped TV series image.
  • the episode frame may not be able to completely fill the entire content area.
  • the theme color of the episode frame can be extracted and used to fill the area outside the episode frame within the content area, which helps to make the generated target e-poster look more natural.
  • the display size of the dialogue in a TV series is dynamically adjusted based on the text content corresponding to the dialogue.
  • the display size of each character in the dialogue is dynamically adjusted based on the number of characters in the text content corresponding to the dialogue and the size of the text area used to display the dialogue.
  • the size of the text area occupied by each character in the dialogue is equal to the quotient of the size of the text area used to display the dialogue and the number of characters in the text content corresponding to the dialogue. For example, assuming the size of the text area used to display the dialogue is 10 square centimeters and the number of characters in the text content corresponding to the dialogue is 10, then the size of the text area occupied by each character in the dialogue is 1 square centimeter.
  • the display size of each character in the dialogue is directly proportional to the size of the text area occupied by each character.
  • the goal is to make the dialogue from the TV series adaptably displayed in the text area of the target electronic poster. This means avoiding both excessively large display sizes that would make the overall image of the target electronic poster unattractive and excessively small display sizes that would result in poor clarity of the dialogue.
  • Figure 25 shows a block diagram of a media content generation apparatus provided in an exemplary embodiment of this application.
  • the apparatus includes:
  • Display module 2510 is used to display text media content published by media accounts.
  • a media account refers to a user account logged in on a client that supports editing and publishing text media content.
  • a media account refers to a user account corresponding to an application on a client that supports editing and publishing text media content.
  • a user account logged in on a social media application For example, a user account logged in on a social media application.
  • the social media application needs to support editing and publishing text media content.
  • text-based media content includes a number of texts.
  • text-based media content also includes other forms of content such as images and videos.
  • text-based media content includes articles from public WeChat accounts, current affairs commentary articles, etc.
  • the text media content is pre-created by the creator corresponding to the media account.
  • text media content may also include other forms of content such as images and videos.
  • the determination module 2520 is used to determine the target text for generating target multimedia content in response to a selection operation of at least one character among a plurality of texts.
  • the selection operation for at least one character among several texts includes at least one of the following: single click, double click, left/right swipe, up/down swipe, long press, hover, facial recognition, and voice recognition. It is worth noting that the selection operation for at least one character among several texts includes, but is not limited to, the aforementioned methods. Those skilled in the art should understand that any operation capable of achieving the above functions falls within the protection scope of the embodiments of this application.
  • a generation request for generating target multimedia content in response to a selection operation of at least one character among a plurality of texts, is sent to the server.
  • the generation request for generating target multimedia content includes a request to determine target text based on the selected at least one character.
  • the generation request for generating target multimedia content further includes a request to generate target multimedia content based on the target text.
  • the target text is key information in the selected at least one character.
  • the display state of the selected at least one character in response to a selection operation on at least one character among a plurality of texts, is switched from a first display state to a second display state.
  • the first display state is a state where at least one character is not selected
  • the second display state is a state where at least one character is selected.
  • Display module 2510 is also used to display target multimedia content generated based on target text.
  • the target multimedia content includes episode scenes and episode dialogue that match the target text.
  • the content of the scene corresponding to the episode that matches the target text matches the text content expressed by the target text.
  • the meaning of the content of the scene corresponding to the episode that matches the target text is consistent with the meaning of the text content expressed by the target text.
  • the text content corresponding to the TV drama dialogue that matches the target text matches the text content expressed by the target text.
  • the meaning of the text content corresponding to the TV drama dialogue that matches the target text is consistent with the meaning of the text content expressed by the target text.
  • the matching degree between the first text content corresponding to the TV drama dialogue that matches the target text and the second text content expressed by the target text is greater than or equal to a matching degree threshold.
  • the target multimedia content is generated by the server based on a generation request sent by the terminal device. Based on the generation request, the server determines the episode scenes and dialogue that match the target text, and then generates the target multimedia content. Afterward, the server sends the generated target multimedia content to the terminal device, which receives and displays the target multimedia content.
  • the source material corresponding to the target multimedia content is a static image. In some embodiments, the source material corresponding to the target multimedia content is a dynamic video frame.
  • the function of generating target multimedia content needs to be triggered by a generation control.
  • the target multimedia content may also include promotional content provided by the promoter.
  • the target media content may also include advertising information to be promoted provided by the advertiser.
  • the display module 2510 is also used to display the selected text and the selection function bar in response to the selection operation of at least one character among a plurality of texts.
  • the selected text in response to a selection operation on at least one character among a plurality of texts, is switched from a first display state to a second display state.
  • the first display state is an unselected state
  • the second display state is a selected state.
  • a selection bar is displayed in response to the selection of at least one character among a plurality of texts.
  • the selected function bar includes a generate control.
  • the generate control is used to trigger the generation of target multimedia content.
  • a generation request for generating target multimedia content is sent to the server.
  • the generation request for generating target multimedia content includes a request to determine target text based on at least one selected character.
  • the generation request for generating target multimedia content further includes a request to generate target multimedia content based on the target text.
  • the selected function bar may further include other function controls besides the generate control.
  • the selected function bar may further include at least one of a copy control, a share control, and a favorite control.
  • the copy control is used to copy at least one selected text.
  • the share control is used to share at least one selected text.
  • the favorite control is used to favorite at least one selected text.
  • the determination module 2520 is used to determine the selected text as the target text for generating target multimedia content in response to a trigger operation on the generation control.
  • the triggering operation for the generated control includes at least one of the following: single click, double click, left/right swipe, up/down swipe, long press, hover, facial recognition, and voice recognition. It is worth noting that the triggering operation for the generated control includes, but is not limited to, several of the above-mentioned operations. Those skilled in the art should understand that any operation capable of achieving the above functions falls within the protection scope of the embodiments of this application.
  • the target text for generating the target multimedia content is determined directly based on the selected text.
  • the selected text is directly determined as the target text for generating the target multimedia content.
  • the target text for generating the target multimedia content is determined indirectly based on the selected text.
  • the selected text is analyzed using a sentence segmentation algorithm, which segments the selected text into sentences, removes incomplete sentences from the selected text, and determines the remaining complete sentences as the target text for generating the target multimedia content.
  • the text includes a first text that is highlighted, the highlighting state indicating that the first text supports multimedia content generation functionality.
  • the display module 2510 is also used to respond to the selection operation of the first text, display the first text as the selected text, and display the selected function bar.
  • the selection operation for the first text includes at least one of the following: single click, double click, left/right swipe, up/down swipe, long press, hover, facial recognition, and voice recognition. It is worth noting that the selection operation for the first text includes, but is not limited to, several of the above-mentioned operations. Those skilled in the art should understand that any operation capable of achieving the above functions falls within the protection scope of the embodiments of this application.
  • the first text in response to a selection operation on the first text, is switched from a first display state to a second display state.
  • the first display state is an unselected state
  • the second display state is a selected state.
  • a selection bar in response to a selection operation on the first text, a selection bar is displayed.
  • the selection bar includes a generation control.
  • the first text is determined by the creator of the text media content before it is published.
  • creators of text-based media content need to enable the multimedia content generation function before publishing the content.
  • the multimedia content generation function enabled, other users can select at least one character from a plurality of texts within the text-based media content. If the creator of the text-based media content has not enabled the multimedia content generation function, it indicates that the creator does not wish their content to be used to generate relevant target multimedia content; in this case, it is impossible to generate target multimedia content by selecting at least one character from a plurality of texts within the text-based media content.
  • the first text is determined by the backend server from the text media content based on the matching degree between the associated material and the text media content.
  • the target text in response to an editing operation on any target text in at least one target text, either the target text is retained or the target text is deleted.
  • the user may choose to share the target multimedia content with the target object.
  • Display module 2510 is also used to display sharing controls.
  • the share control is used to trigger the sharing of target multimedia content.
  • the sharing control and the target multimedia content are displayed simultaneously.
  • the sharing control is displayed after the target multimedia content has already been displayed. That is, the target multimedia content and the sharing control are displayed sequentially.
  • the sending module 2530 is used to share target multimedia content to a specified network location in response to a trigger operation of the sharing control.
  • the triggering operation for the sharing control includes at least one of the following: single click, double click, left/right swipe, up/down swipe, long press, hover, facial recognition, and voice recognition. It is worth noting that the triggering operation for the sharing control includes, but is not limited to, several of the above-mentioned operations. Those skilled in the art should understand that any operation capable of achieving the above functions falls within the protection scope of the embodiments of this application.
  • the target multimedia content includes a side-by-side upper and lower half-area, with the upper half-area displaying screen footage and the lower half-area displaying dialogue from the series.
  • the target multimedia content includes a side-by-side upper and lower half-area, with the upper half-area displaying dialogue from the series and the lower half-area displaying visuals from the series.
  • the target multimedia content includes a left half-area and a right half-area side by side, with the left half-area displaying dialogue from the series and the right half-area displaying images from the series.
  • the target multimedia content includes a left half-area and a right half-area side by side, with the left half-area displaying the scene and the right half-area displaying the dialogue.
  • the target multimedia content includes a content area.
  • the content area displays screen images and dialogue overlaid on the screen images.
  • the dialogue may be displayed separately from the screen images. This application does not limit the display method of the screen images and dialogue in these embodiments.
  • the target multimedia content further includes an information area.
  • the information area is used to display information and/or target text related to the episode.
  • the terminal device may also prompt the user with some information to indicate that the target multimedia content is being generated, or that the target multimedia content has been successfully generated, or that the target multimedia content has failed to be generated.
  • a first prompt message is displayed in response to the selection of at least one character among a plurality of texts. In some embodiments, the first prompt message is displayed in response to a triggering operation on a generation control. The first prompt message indicates that target multimedia content is being generated.
  • a second prompt message is displayed to indicate that the generation of target multimedia content has failed.
  • the second prompt message instructs the user to reselect at least one character from a plurality of texts.
  • a third prompt message is displayed to indicate that the target multimedia content has been successfully generated.
  • the sending module 2530 is further configured to send a generation request for generating target multimedia content to the server.
  • the above-described apparatus further includes:
  • the receiving module 2540 is used to receive target multimedia content sent by the server.
  • Figure 26 shows a block diagram of a media content generation apparatus provided in an exemplary embodiment of this application.
  • the apparatus includes:
  • the receiving module 2610 is used to receive the target text sent by the client for generating target multimedia content.
  • At least one text selected by the user of the client is received. That is, the selected text sent by the client is received.
  • the target text is selected from text media content published by a media account. That is, the target text is the selected text.
  • the target text is determined by the server based on the selected text.
  • the selected text is analyzed using a sentence segmentation algorithm, which segments the selected text into sentences, removes incomplete sentences from the selected text, and determines the remaining complete sentences as the target text for generating the target multimedia content.
  • the target text is a first text highlighted in the text media content published by the media account, the highlighted state being used to indicate that the first text supports multimedia content generation functionality.
  • the first text is determined based on the degree of matching between the associated episode material and the text media content.
  • the associated TV series materials related to the text media content are determined based on the similarity between the semantic text of the text media content and the semantic text of the TV series dialogue subtitles in the associated TV series materials.
  • the semantic similarity between the semantic text of the text media content and the semantic text of the TV series dialogue subtitles in the associated TV series materials is calculated, and TV series materials with a semantic similarity greater than or equal to a semantic similarity threshold are considered as associated TV series materials related to the text media content.
  • the associated TV series materials can also be sorted according to the magnitude of semantic similarity.
  • the text media content is segmented into sentences.
  • the segmented sentences are then matched against subtitles from related TV series footage.
  • the similarity between the segmented sentences and the subtitles is calculated. If the text similarity between the segmented sentences and the subtitles is greater than or equal to a text similarity threshold, the segmented sentence is designated as the first text.
  • the first text can be sorted according to the magnitude of text similarity.
  • the target media content includes at least one of animation, poster, audio, and video.
  • the generation module 2620 is used to generate target multimedia content based on target text.
  • the target text is matched with corresponding episode scenes and dialogue within a drama series database.
  • This drama series database is pre-generated by the server.
  • the server pre-segments different drama series to obtain a database containing episode scenes and dialogue. It should be understood that the episode scenes and dialogue in this embodiment are correlated. Typically, multiple episode scenes may correspond to the same dialogue.
  • target multimedia content including episode images and episode dialogues is generated based on episode images and episode dialogues that match the target text.
  • the sending module 2630 is used to send target multimedia content to the client.
  • the target media content includes episode scenes and episode dialogue that match the target text.
  • the target multimedia content may also include promotional content provided by the promoter.
  • the target media content may also include advertising information to be promoted provided by the advertiser.
  • the generation module 2620 is also used to query TV series images and TV series lines that match the target text in a preset correspondence based on the target text.
  • the TV series images and TV series lines are used to generate target multimedia content.
  • the preset mapping relationship is predetermined by the server.
  • the generation module 2620 is also used to obtain the first episode video and episode dialogue subtitles.
  • the video of the first episode and the subtitles of the episode's dialogue corresponding to the first episode are obtained.
  • the video of the first episode and the subtitles of the episode's dialogue are obtained from a video database.
  • the generation module 2620 is also used to divide the first episode video into at least one video segment according to the start and end times of the subtitles in the episode dialogue.
  • the first episode video is divided into at least one video segment according to the start and end times of the subtitles corresponding to the dialogue subtitles of the same episode. It should be understood that the first number of video segments is greater than or equal to the second number of episode dialogue subtitles.
  • the generation module 2620 is also used to extract at least one episode frame from the video segments.
  • the video is divided into segments, with each segment consisting of a video frame.
  • Each video segment is divided into at least one video frame. Scenes from each video frame are then extracted. It is important to understand that there is a one-to-one correspondence between video frames and scenes from each video frame.
  • At least one target video frame is extracted from the video slice, and the quality of each target video frame is greater than or equal to a quality threshold.
  • the generation module 2620 is also used to generate a preset correspondence based on at least one episode screen and episode dialogue subtitles.
  • a preset correspondence between episode scenes and episode dialogue is generated based on at least one extracted episode scene.
  • episode images there is a many-to-one correspondence between episode images and episode dialogue.
  • the generation module 2620 is also used to calculate the first feature vector corresponding to the target text, and to calculate the vector similarity between the first feature vector and at least one second feature vector; the TV scene and TV line corresponding to the second feature vector with the highest vector similarity are determined as the TV scene and TV line matching the target text, and the TV scene and TV line are used to generate target multimedia content.
  • the episode scenes and episode lines corresponding to the second feature vector with a vector similarity greater than or equal to the similarity threshold are all determined as episode scenes and episode lines that match the target text.
  • the second feature vector is predetermined by the server.
  • the generation module 2620 is also used to obtain the video of the second episode.
  • the second episode video is obtained from a video database.
  • the generation module 2620 is also used to divide the second episode video into at least one video segment according to the division step size.
  • the partitioning step size is random. In some embodiments, the partitioning step size is preset.
  • the second episode video is continuously divided according to a division step size. That is, at least one video segment in the embodiments of this application is continuous. It can be understood that combining at least one video segment in sequence can obtain a complete second episode video.
  • the generation module 2620 is also used to extract computer vision information from each video segment in at least one video segment.
  • content description information for each video segment is generated based on the extracted computer vision information for each video segment.
  • content description information of each video segment in at least one video segment is obtained.
  • the generation module 2620 is also used to generate a second feature vector for each video segment based on the content description information of each video segment.
  • a second feature vector for each video segment is generated based on the content description information of each video segment and a vector database.
  • the vector database includes the correspondence between the content description information and the feature vectors.
  • the generation module 2620 is also used to extract at least one episode frame from the video segments.
  • the video is divided into segments, with each segment consisting of a video frame.
  • Each video segment is divided into at least one video frame. Scenes from each video frame are then extracted. It is important to understand that there is a one-to-one correspondence between video frames and scenes from each video frame.
  • the generation module 2620 is also used to store the correspondence between at least one episode frame of the video segment and the second feature vector.
  • the second feature vector corresponding to each video segment and at least one episode frame corresponding to each video segment are known, there is a correspondence between at least one episode frame and the second feature vector.
  • the correspondence between episode scenes and the second feature vector is many-to-one.
  • the relationship between episode scenes and the second feature vector is one-to-many.
  • the quality is determined based on the quality of the video frames in each video segment.
  • the generation module 2620 is also used to score each video frame in the video segment according to at least one scoring factor to obtain a quality score corresponding to each video frame.
  • each video segment includes multiple video frames, and each of the multiple video frames corresponds to a quality score.
  • At least one scoring factor includes at least one of brightness, sharpness, and confidence in the identification of the subject in the video frame.
  • Brightness refers to the light intensity in the video frame.
  • Sharpness refers to the clarity of the image in the video frame.
  • Confidence in the identification of the subject in the video frame refers to the probability that the subject in the video frame is identified.
  • the generation module 2620 is also used to extract at least one episode frame from each video frame in the video segment based on quality scoring.
  • At least one episode frame is extracted from the target video frame in the video segment based on the quality score of each video frame.
  • the quality score corresponding to the target video frame is greater than or equal to a quality threshold score.
  • the video frames in the video segment are sorted from high to low or from low to high according to their quality scores, and at least one video frame with a quality score greater than or equal to a quality threshold score is selected. At least one episode frame is extracted from the at least one video frame with a quality score greater than or equal to the quality threshold score.
  • the target multimedia content is generated based on the corresponding episode scenes and episode lines that match the target text.
  • the generation module 2620 is also used to calculate the picture quality score of at least two candidate episode scenes when there are at least two candidate episode scenes in the target text matching.
  • each candidate episode frame is scored according to at least one scoring factor to obtain a picture quality score for each candidate episode frame.
  • the at least one scoring factor includes at least one of brightness, sharpness, and confidence in the identification of the subject in the picture.
  • the generation module 2620 is also used to select the target episode from at least two candidate episodes based on the episode quality score.
  • At least one target episode image is selected from at least two candidate episode images that match the target text.
  • the image quality score corresponding to the target episode image is greater than or equal to an image quality score threshold.
  • the generation module 2620 is also used to generate target multimedia content based on the target TV series images and TV series dialogue that matches the target text.
  • the target episode image includes at least one, and the episode dialogue matching the target text may also include at least one.
  • each episode image corresponds to its own episode dialogue; that is, there is a one-to-one correspondence between episode images and episode dialogue.
  • the target text matches the first episode image of a first series and the second episode image of a second series.
  • the first and second series are two different series.
  • the first episode image corresponds to episode dialogue 1
  • the second episode image corresponds to episode dialogue 2.
  • the candidate episode frame is determined as the target episode frame. That is, if there is one and only one matching candidate episode frame for the target text, that candidate episode frame is the target episode frame used to generate the target multimedia content.
  • the target multimedia content is generated based on cropped episode footage.
  • the main subjects in a scene are assigned a priority. Based on the priority of the main subjects in the scene, the main subjects in the scene are identified, and a bounding box is used to select the corresponding main subject.
  • the main subject of the image includes at least one of a face, a human body, an animal, a plant, furniture, clutter, and scenery.
  • the priority of a face is higher than that of a human body
  • the priority of a human body is higher than that of an animal
  • the priority of an animal is higher than that of a plant
  • the priority of a plant is higher than that of furniture
  • the priority of furniture is higher than that of clutter
  • the priority of clutter is higher than that of scenery.
  • the bounding boxes of the subjects of the same priority in the scene are identified simultaneously. For example, assuming that there are multiple faces in the same scene, multiple faces are identified simultaneously, and multiple faces are selected by bounding boxes.
  • the generation module 2620 is also used to crop the TV series screen based on the bounding box of the main subject of the screen to obtain the cropped TV series screen.
  • the episode frame is cropped based on the bounding box of the identified main subject to obtain the cropped episode frame.
  • the cropped episode frame is smaller than or equal to the original episode frame.
  • the generation module 2620 is also used to merge at least two bounding boxes to obtain a merged bounding box when at least two bounding boxes are identified based on the main body of the image.
  • the identified at least two bounding boxes are merged to obtain a merged bounding box.
  • the area of the merged bounding box is determined based on the merged at least two bounding boxes.
  • the merged bounding box covers all or part of the bounding boxes in at least two bounding boxes.
  • At least one group of bounding boxes to be merged is selected based on the priorities corresponding to the at least two different subjects.
  • the number of bounding boxes in a group does not exceed a second threshold, and the priority of the bounding boxes in the group is higher than a third threshold.
  • the number of main subjects in the final episode frame cropped based on the bounding boxes is also limited to a certain threshold. This helps to ensure that the number of main subjects in the cropped episode frame is reasonable and avoids a cluttered overall picture.
  • the maximum top-left coordinate, maximum top-right coordinate, minimum bottom-left coordinate, and minimum bottom-right coordinate in the bounding box group are determined. Based on the maximum top-left coordinate, maximum top-right coordinate, minimum bottom-left coordinate, and minimum bottom-right coordinate, a merged bounding box for each bounding box group is generated.
  • each bounding box corresponds to a top-left coordinate, a top-right coordinate, a bottom-left coordinate, and a bottom-right coordinate.
  • each bounding box can be determined by a combination of (x start , y start ) and (x end , y end ).
  • (x start , y start ) represents the bottom-left coordinate
  • (x start , y end ) represents the top-left coordinate
  • (x end , y start ) represents the bottom-right coordinate
  • (x end , y end ) represents the top-right coordinate.
  • the merging of the bounding boxes can be determined by min(x1 start , x2 start , ..., xn start ), min(y1 start , y2 start , ..., yn start ), max(x1 end , x2 end , ..., xn end ), max(y1 end , y2 end , ..., yn end ).
  • the episode frame is cropped based on a defined fusion bounding box to obtain a cropped episode frame.
  • the cropped episode frame is smaller than or equal to the original episode frame.
  • Figure 27 illustrates a schematic diagram of a computer device provided in an exemplary embodiment of this application.
  • the computer device 1100 includes a Central Processing Unit (CPU) 1101, a system memory 1104 including Random Access Memory (RAM) 1102 and Read-Only Memory (ROM) 1103, and a system bus 1105 connecting the system memory 1104 and the CPU 1101.
  • the computer device 1100 also includes a basic input/output system 1106 that facilitates information transfer between various devices within the computer, and a mass storage device 1107 for storing an operating system 1113, application programs 1114, and other program modules 1115.
  • the basic input/output system 1106 includes a display 1108 for displaying information and an input device 1109 for user input, such as a mouse or keyboard. Both the display 1108 and the input device 1109 are connected to the central processing unit 1101 via an input/output controller 1110 connected to the system bus 1105.
  • the basic input/output system 1106 may also include the input/output controller 1110 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input/output controller 1110 also provides output to a display screen, printer, or other types of output devices.
  • Mass storage device 1107 is connected to central processing unit 1101 via a mass storage controller (not shown) connected to system bus 1105. Mass storage device 1107 and its associated computer-readable media provide non-volatile storage for computer device 1100. That is, mass storage device 1107 may include computer-readable media (not shown) such as hard disk or compact disc read-only memory (CD-ROM) drive.
  • mass storage device 1107 may include computer-readable media (not shown) such as hard disk or compact disc read-only memory (CD-ROM) drive.
  • Computer-readable media can include computer storage media and communication media.
  • Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data.
  • Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • flash memory or other solid-state storage technologies
  • CD-ROM compact disc
  • DVD digital versatile disc
  • the computer device 1100 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 1100 can be connected to the network 1112 via the network interface unit 1111 connected to the system bus 1105, or the network interface unit 1111 can be used to connect to other types of networks or remote computer systems (not shown).
  • An exemplary embodiment of this application also provides a computer-readable storage medium storing at least one program, which is loaded and executed by a processor to implement the media content generation method provided in the above-described method embodiments.
  • An exemplary embodiment of this application also provides a computer program product, which includes at least one program segment stored in a readable storage medium; the processor of the communication device reads signaling from the readable storage medium, and the processor executes the signaling to cause the communication device to perform the media content generation method provided in the above-described method embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Library & Information Science (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

本申请公开了一种媒体内容生成方法、装置、设备、存储介质及程序产品,涉及信息处理技术领域。该方法包括:显示媒体账号发布的文字媒体内容,所述文字媒体内容包括若干文本;响应于对所述若干文本中的至少一个文字的选中操作,确定用于生成目标多媒体内容的目标文本;显示基于所述目标文本生成的所述目标多媒体内容。该方法能够在确定目标文本之后,自动生成目标多媒体内容。用户无需执行多个操作步骤,就能获取到与目标文本匹配的目标多媒体内容,有利于提高生成目标多媒体内容的效率。

Description

媒体内容生成方法、装置、设备、存储介质及程序产品 技术领域
本申请要求于2024年07月12日提交中国专利局、申请号为202410942622X、发明名称为“媒体内容生成方法、装置、设备及存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及信息处理技术领域,特别涉及一种媒体内容生成方法、装置、设备、存储介质及程序产品。
背景技术
随着通信技术的发展,人们的社交行为和需求不断发生着变化。用户在社交过程中,可以通过生成动画、海报、音视频等多媒体内容的方式进行主动分享。
相关技术中,用户需要执行多个操作步骤,以生成目标多媒体内容。比如,用户需要手动选择用于制作目标多媒体内容的素材信息,然后再对选择的多种素材信息进行编辑,从而生成目标多媒体内容。
发明内容
本申请提供了一种媒体内容生成方法、装置、设备及存储介质。所述技术方案如下:
根据本申请的一方面,提供了一种媒体内容生成方法,所述方法由客户端执行,所述方法包括:
显示媒体账号发布的文字媒体内容,所述文字媒体内容包括若干文本;
响应于对所述若干文本中的至少一个文字的选中操作,确定用于生成目标多媒体内容的目标文本;
显示基于所述目标文本生成的所述目标多媒体内容。
根据本申请的另一方面,提供了一种媒体内容生成方法,所述方法由服务器执行,所述方法包括:
接收客户端发送的用于生成目标多媒体内容的目标文本,所述目标文本是从媒体账号发布的文字媒体内容中选择得到的;
基于所述目标文本生成所述目标多媒体内容;
向所述客户端发送所述目标多媒体内容。
根据本申请的另一方面,提供了一种媒体内容生成装置,所述装置包括:
显示模块,用于显示媒体账号发布的文字媒体内容,所述文字媒体内容包括若干文本;
确定模块,用于响应于对所述若干文本中的至少一个文字的选中操作,确定用于生成目标多媒体内容的目标文本;
所述显示模块,还用于显示基于所述目标文本生成的所述目标多媒体内容。
根据本申请的另一方面,提供了一种媒体内容生成装置,所述装置包括:
接收模块,用于接收客户端发送的用于生成目标多媒体内容的目标文本,所述目标文本是从媒体账号发布的文字媒体内容中选择得到的;
生成模块,用于基于所述目标文本生成所述目标多媒体内容;
发送模块,用于向所述客户端发送所述目标多媒体内容。
根据本申请的另一方面,提供了一种计算机设备,所述计算机设备包括:处理器和存储器,所述存储器中存储有至少一段程序,所述至少一段程序由所述处理器加载并执行,以实现如上方面所述的媒体内容生成方法。
根据本申请的另一方面,提供了一种计算机可读存储介质,该计算机可读存储介质中存储有至少一段程序,所述至少一段程序由处理器加载并执行以实现如上方面所述的媒体内容生成方法。
根据本申请的另一方面,提供了一种计算机程序产品,该计算机程序产品包括至少一段程序,所述至少一段程序存储在计算机可读存储介质中;计算机设备的处理器从所述计算机可读存储介质中读取所述至少一段程序,所述处理器执行所述至少一段程序,使得所述计算机设备执行如上方面所述的媒体内容生成方法。
本申请提供的技术方案带来的有益效果至少包括:
提供了一种自动生成目标多媒体内容的方式,在确定目标文本之后,即能直接显示基于目标文本生成的目标多媒体内容。相比于用户手动生成目标多媒体内容的方法,用户无需执行多个操作步骤,就可以一键式生成与目标文本匹配的目标多媒体内容,有利于提高生成目标多媒体内容的效率。
附图说明
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
图1是本申请的一个示例性实施例提供的计算机系统的示意图;
图2是本申请的一个示例性实施例提供的媒体内容生成方法的流程图;
图3是本申请的一个示例性实施例提供的媒体内容生成方法的示意图;
图4是本申请的一个示例性实施例提供的媒体内容生成方法的示意图;
图5是本申请的一个示例性实施例提供的媒体内容生成方法的流程图;
图6是本申请的一个示例性实施例提供的媒体内容生成方法的示意图;
图7是本申请的一个示例性实施例提供的媒体内容生成方法的流程图;
图8是本申请的一个示例性实施例提供的媒体内容生成方法的示意图;
图9是本申请的一个示例性实施例提供的媒体内容生成方法的示意图;
图10是本申请的一个示例性实施例提供的媒体内容生成方法的流程图;
图11是本申请的一个示例性实施例提供的媒体内容生成方法的示意图;
图12是本申请的一个示例性实施例提供的媒体内容生成方法的示意图;
图13是本申请的一个示例性实施例提供的媒体内容生成方法的流程图;
图14是本申请的一个示例性实施例提供的媒体内容生成方法的流程图;
图15是本申请的一个示例性实施例提供的媒体内容生成方法的流程图;
图16是本申请的一个示例性实施例提供的媒体内容生成方法的示意图;
图17是本申请的一个示例性实施例提供的媒体内容生成方法的流程图;
图18是本申请的一个示例性实施例提供的媒体内容生成方法的流程图;
图19是本申请的一个示例性实施例提供的媒体内容生成方法的流程图;
图20是本申请的一个示例性实施例提供的媒体内容生成方法的流程图;
图21是本申请的一个示例性实施例提供的媒体内容生成方法的流程图;
图22是本申请的一个示例性实施例提供的媒体内容生成方法的流程图;
图23是本申请的一个示例性实施例提供的媒体内容生成方法的示意图;
图24是本申请的一个示例性实施例提供的媒体内容生成方法的示意图;
图25是本申请的一个示例性实施例提供的媒体内容生成装置的框图;
图26是本申请的一个示例性实施例提供的媒体内容生成装置的框图;
图27是本申请的一个示例性实施例提供的计算机设备的框图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚,下面将结合附图对本申请实施方式作进一步地详细描述。
这里将详细地对示例性实施例进行说明,其示例表示在附图中。下面的描述涉及附图时,除非另有表示,不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反,它们仅是与如所附权利要求书中所详述的、本申请的一些方面相一致的装置和方法的例子。
在本申请使用的术语是仅仅出于描述特定实施例的目的,而非旨在限制本申请。在本申请和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式,除非上下文清楚地表示其他含义。还应当理解,本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解,尽管在本申请可能采用术语第一、第二等来描述各种信息,但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如,在不脱离本申请范围的情况下,第一参数也可以被称为第二参数,类似地,第二参数也可以被称为第一参数。取决于语境,如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
首先,对本申请实施例中涉及的名词进行简单介绍:
多媒体内容:是指通过数字信号进行处理,以文字、图像、音频、视频等表现形式进行展示的信息内容。
文字媒体内容:是指以文字为载体的媒体信息。本申请实施例中,该文字媒体内容指在网络媒介中传播的文字媒体信息。
剧集画面:是指电影、电视剧、短视频等视频中的视频帧画面信息。
剧集台词:是指电影、电视剧、短视频等视频中的台词信息。
图1示出了本申请一个示例性实施例提供的计算机系统的结构框图。该计算机系统100包括:终端设备120和服务器140。
终端设备120安装和运行有支持展示文字媒体内容的客户端。支持展示文字媒体内容的客户端可以是视频程序、阅读程序、社交程序、直播程序、虚拟现实(Virtual Reality,VR)应用程序、增强现实(Augmented Reality,AR)程序中的任意一种。终端设备120是第一用户使用的终端,第一用户使用终端设备120观看或阅读文字媒体内容,以及通过触发文字媒体内容以生成对应的目标多媒体内容。
终端设备120通过无线网络或有线网络与服务器140相连。
服务器140包括一台服务器、多台服务器、云计算平台和虚拟化中心中的至少一种。示例性的,服务器140包括处理器144和存储器142,存储器142包括显示模块1421、确定模块1422和生成模块1423,显示模块1421用于显示多媒体信息,比如文字媒体内容;确定模块1422用于确定目标文本;生成模块1423用于生成包括剧集画面和剧集台词的目标多媒体内容。服务器140用于为支持多媒体信息的客户端提供后台服务。可选地,服务器140承担主要计算工作,终端设备120承担次要计算工作;或者,服务器140承担次要计算工作,终端设备120承担主要计算工作;或者,服务器140和终端设备120之间采用分布式计算架构进行协同计算。
可选的,终端设备120的设备类型包括:智能手机、平板电脑、电子书阅读器、膝上型便携计算机和台式计算机中的至少一种。本申请实施例中,以终端设备120为智能手机来举例说明。
本领域技术人员可以知晓,上述终端的数量可以更多或更少。比如上述终端可以仅为一个,或者上述终端为几十个或几百个,或者更多数量。本申请实施例对终端的数量和设备类型不加以限定。
相关技术中,用户需要执行多个操作步骤,以生成目标多媒体内容。比如,用户需要手动选择用于制作目标多媒体内容的素材信息,然后再对选择的多种素材信息进行编辑,从而生成目标目标多媒体内容。该生成目标多媒体内容的方法对用户而言较为繁琐。基于此,本申请实施例提出了一种媒体内容生成方法,能够自动生成目标多媒体内容,进而减轻用户的操作流程,提高目标多媒体内容的生成效率。示例性的,如图1所示,终端设备120的用户可以直接通过选中文字媒体内容中的文本内容12中的部分文本,以实现一键触发显示与选中文本对应的目标多媒体内容21。
图2是本申请一个示例性实施例提供的媒体内容生成方法的示意图。本申请实施例中,以该方法由图1所示的终端设备120执行为例进行说明。该方法包括:
步骤220:终端设备显示的文字媒体内容。
在一些实施例中,该文字媒体内容为媒体账号发布的。
在一些实施例中,媒体账号是指支持编辑以及发布文字媒体内容的客户端中所登录的用户账号。可选的,媒体账号是指客户端中支持编辑以及发布文字媒体内容的应用程序所对应的用户账号。比如,社交程序中所登录的用户账号。可选的,该社交程序需要支持编辑以及发布文字媒体内容。
在一些实施例中,文字媒体内容包括若干文本。可选的,文字媒体内容还包括图片、视频等其他形式的内容。示例性的,文字媒体内容包括公众号文章、时事评论文章等。
在一些实施例中,文字媒体内容是基于媒体账号对应的创作者预先创作得到的。
示例性的,如图3所示,显示文字媒体内容11。其中,文字媒体内容11中至少包括文本内容12。可选的,文字媒体内容11中还包括图片内容13。
步骤240:终端设备响应于对若干文本中的至少一个文字的选中操作,确定用于生成目标多媒体内容的目标文本。
在一些实施例中,针对若干文本中的至少一个文字的选中操作包括单击、双击、左右滑动、上下滑动、长按、悬停、面部识别、语音识别中的至少一种。值得说明的是,该针对若干文本中的至少一个文字的选中操作包括但不限于上述提及几种,本领域内的技术人员应该能够知晓,任意能够实现上述功能的操作均属于本申请实施例的保护范围。
在一些实施例中,目标媒体内容包括动画、海报、音频、视频中的至少之一。
在一些实施例中,终端设备响应于对若干文本中的至少一个文字的选中操作,向服务器发送用于生成目标多媒体内容的生成请求。可选的,用于生成目标多媒体内容的生成请求中包括基于被选中的至少一个文字确定目标文本的请求。可选的,用于生成目标多媒体内容的生成请求中还包括基于目标文本生成目标多媒体内容的请求。可选的,目标文本包括被选中的至少一个文字中的关键信息。
在一些实施例中,终端设备响应于对若干文本中的至少一个文字的选中操作,将被选中的至少一个文字的显示态从第一显示态切换为第二显示态。其中,第一显示态是至少一个文字未被选中的状态,第二显示态是至少一个文字被选中的状态。
步骤260:终端设备显示基于目标文本生成的目标多媒体内容。
在一些实施例中,目标多媒体内容包括与目标文本匹配的剧集画面和剧集台词。
在一些实施例中,与目标文本匹配的剧集画面所对应的画面内容与目标文本所表达的文本内容相互匹配。可选的,与目标文本匹配的剧集画面所对应的画面内容的含义,与目标文本所表达的文本内容的含义是一致的。比如,假设目标文本为“xxx真美丽”,剧集画面1所对应的画面内容为“xxx”妆容艳丽,则认为剧集画面1与目标文本匹配。
在一些实施例中,与目标文本匹配的剧集台词所对应的文本内容与目标文本所表达的文本内容相互匹配。可选的,与目标文本匹配的剧集台词所对应的文本内容的含义,与目标文本所表达的文本内容的含义是一致的。可选的,目标文本匹配的剧集台词所对应的第一文本内容,与目标文本所表达的第二文本内容的匹配度大于或等于匹配度阈值。示例性的,假设目标文本为“xxx真美丽”,剧集台词为“xxx真好看”,“美丽”和“好看”所表达的含义一致,则认为剧集台词与目标文本匹配。
在一些实施例中,目标多媒体内容是服务器基于终端设备发送的用于生成目标多媒体内容的生成请求生成的。服务器基于用于生成目标多媒体内容的生成请求,确定与目标文本匹配的剧集画面和剧集台词,然后生成目标多媒体内容。之后,服务器向终端设备发送生成好的目标多媒体内容,终端设备接收以及显示该目标多媒体内容。
在一些实施例中,在终端设备上显示的目标多媒体内容是与目标文本的匹配度最高的一张目标多媒体内容。也即,服务器根据目标文本可能生成了多张目标多媒体内容,但多张目标多媒体内容与目标文本的匹配度是不同的。比如,假设目标多媒体内容1与目标文本的匹配度为90%,目标多媒体内容2与目标文本的匹配度为80%,则服务器仅会向终端设备发送匹配度最高的目标多媒体内容1。
在一些实施例中,在终端设备上显示的目标多媒体内容是与目标文本匹配的全部目标多媒体内容。可选的,在终端设备上显示的目标多媒体内容是与目标文本的匹配度大于匹配度阈值的目标多媒体内容。比如,假设目标多媒体内容1与目标文本的匹配度为90%,目标多媒体内容2与目标文本的匹配度为80%,目标多媒体内容3与目标文本的匹配度为60%,匹配度阈值为75%,则服务器仅会向终端设备发送目标多媒体内容1和目标多媒体内容2。
在一些实施例中,目标多媒体内容所对应的素材是静态的图片。在一些实施例中,目标多媒体内容所对应的素材是动态的视频帧。
综上所述,本实施例提供的方法,提供了一种自动生成目标多媒体内容的方式,在确定目标文本之后,即能直接显示基于目标文本生成的目标多媒体内容。相比于用户手动生成目标多媒体内容的方法,用户无需执行多个操作步骤,就可以一键式生成与目标文本匹配的目标多媒体内容,减少用户的工作量,有利于提高生成目标多媒体内容的效率。
在一些实施例中,目标多媒体内容包括并列的上半区域和下半区域,上半区域显示有剧集画面,下半区域显示有剧集台词。示例性的,如图4所示,上半区域19和下半区域20并列组成了目标多媒体内容。其中,上半区域19显示有剧集画面,下半区域20显示有剧集台词。
在一些实施例中,目标多媒体内容包括并列的上半区域和下半区域,上半区域显示有剧集台词,下半区域显示有剧集画面。
在一些实施例中,目标多媒体内容包括并列的左半区域和右半区域,左半区域显示有剧集台词,右半区域显示有剧集画面。
在一些实施例中,目标多媒体内容包括并列的左半区域和右半区域,左半区域显示有剧集画面,右半区域显示有剧集台词。
在一些实施例中,目标多媒体内容包括内容区域。内容区域显示有剧集画面,以及叠加在剧集画面上的剧集台词。可选的,剧集台词还可以是与剧集画面分开显示的。本申请实施例中对剧集画面和剧集台词的显示方式不做限定。
在一些实施例中,目标多媒体内容还包括信息区域。信息区域用于显示与剧集相关的信息和/或目标文本。
在一些实施例中,上述目标多媒体内容由终端设备或服务器自动生成。在上述目标多媒体内容是由服务器生成的情况下,或上述目标多媒体内容是由与该终端设备不同的其他终端生成的情况下,上述步骤240之后,步骤260之前,还包括如下步骤:
步骤251:终端设备发送用于生成目标多媒体内容的目标文本;
步骤252:终端设备接收基于目标文本生成的目标多媒体内容。
在一些实施例中,生成目标多媒体内容的功能需要由生成控件触发。在上述图2所示的实施例的基础上,如图5所示,上述步骤240还可以替换为如下子步骤:
步骤241:终端设备响应于对若干文本中的至少一个文字的选中操作,显示被选中的文本以及选中功能栏。
在一些实施例中,终端设备响应于对若干文本中的至少一个文字的选中操作,将被选中的文本从第一显示态切换为第二显示态。其中,第一显示态是未被选中的状态,第二显示态是被选中的状态。
在一些实施例中,终端设备响应于对若干文本中的至少一个文字的选中操作,显示选中功能栏。示例性的,如图6所示,终端设备响应于对至少一个文字的选中操作,显示选中功能栏14。
在一些实施例中,选中功能栏包括生成控件。其中,生成控件用于触发生成目标多媒体内容。可选的,终端设备响应于对生成控件的触发操作,向服务器发送用于生成目标多媒体内容的生成请求。可选的,用于生成目标多媒体内容的生成请求中包括基于选中的至少一个文字确定目标文本的请求。可选的,用于生成目标多媒体内容的生成请求中还包括基于目标文本生成目标多媒体内容的请求。
在一些实施例中,选中功能栏还包括除生成控件以外的其他功能控件。可选的,选中功能栏还包括复制控件、分享控件、收藏控件中的至少一种。其中,复制控件用于复制选中的至少一个文字。分享控件用于分享选中的至少一个文字。收藏控件用于收藏选中的至少一个文字。
步骤242:终端设备响应于对生成控件的触发操作,将被选中的文本确定为用于生成目标多媒体内容的目标文本。
在一些实施例中,针对生成控件的触发操作包括单击、双击、左右滑动、上下滑动、长按、悬停、面部识别、语音识别中的至少一种。值得说明的是,该针对生成控件的触发操作包括但不限于上述提及几种,本领域内的技术人员应该能够知晓,任意能够实现上述功能的操作均属于本申请实施例的保护范围。
在一些实施例中,用于生成目标多媒体内容的目标文本是终端设备直接根据被选中的文本确定的。比如,终端设备将被选中的文本直接确定为用于生成目标多媒体内容的目标文本。
在一些实施例中,用于生成目标多媒体内容的目标文本是终端设备间接根据被选中的文本确定的。比如,终端设备使用语句分割算法对被选中的文本进行分析,将被选中的文本按句子进行分割,剔除被选中的文本中的不完整语句,将剩余完整语句确定为用于生成目标多媒体内容的目标文本。
综上所述,本实施例提供的方法,终端设备通过显示选中功能栏,使用户能够基于触发选中功能栏中的生成控件,确定用于生成目标多媒体内容的目标文本。通过设置生成控件,使用户能够根据需求选择是否生成目标多媒体内容。
在一些实施例中,若干文本包括呈突出显示状态的第一文本,突出显示状态用于指示第一文本支持多媒体内容生成功能。在上述图5所示的实施例的基础上,如图7所示,上述步骤241还可以替换为如下子步骤:
步骤2411:终端设备响应于对第一文本的选中操作,将第一文本显示为被选中的文本以及显示选中功能栏。
在一些实施例中,针对第一文本的选中操作包括单击、双击、左右滑动、上下滑动、长按、悬停、面部识别、语音识别中的至少一种。值得说明的是,该针对第一文本的选中操作包括但不限于上述提及几种,本领域内的技术人员应该能够知晓,任意能够实现上述功能的操作均属于本申请实施例的保护范围。
在一些实施例中,终端设备响应于对第一文本的选中操作,将第一文本从第一显示态切换为第二显示态。其中,第一显示态是未被选中的状态,第二显示态是被选中的状态。
在一些实施例中,终端设备响应于对第一文本的选中操作,显示选中功能栏。其中,选中功能栏包括生成控件。
在一些实施例中,文字媒体内容的创作者需要在发布文字媒体内容之前开启多媒体内容生成功能。在开启了多媒体内容生成功能的情况下,其他用户才能对文字媒体内容中的若干文本中的至少一个文字进行选择。在文字媒体内容的创作者未开启多媒体内容生成功能的情况下,即表示文字媒体内容的创作者不希望自己的创作内容用于生成相关目标多媒体内容,此时则无法通过对文字媒体内容中的若干文本中的至少一个文字进行选择,以生成目标多媒体内容。
示例性的,以生成海报这一多媒体为例,如图8所示,在文字媒体内容的编辑界面15上显示有功能开关控件16。该功能开关控件16包括开和关这两个选项。响应于针对功能开关控件16的触发操作,开启或关闭多媒体内容生成功能,即开始或关闭海报生成功能。例如,响应于针对功能开关控件16中开这一选项触发操作,开启多媒体内容生成功能(即开启生成海报功能)。再例如,响应于针对功能开关控件16中关这一选项触发操作,关闭多媒体内容生成功能(即关闭生成海报功能)。
在一些实施例中,编辑界面15上还显示也有关联素材列表17。可选的,关联素材列表17中的至少一个关联素材是后端服务器自动根据创作者创作的文字媒体内容推荐的。在一些实施例中,关联素材列表17中包括至少一个关联素材,至少一个关联素材根据与文字媒体内容的匹配度进行排序显示。
在一些实施例中,第一文本是后端服务器根据关联素材与文字媒体内容的匹配度,从文字媒体内容中确定的。示例性的,如图9所示,在文字媒体内容的编辑界面15上显示有匹配的至少一个目标文本18。至少一个目标文本18是文字媒体内容中的文本。至少一个目标文本18中包括上述第一文本。
在一些实施例中,响应于针对至少一个目标文本中的任一目标文本的编辑操作,保留任一目标文本或删除任一目标文本。
综上所述,本实施例提供的方法,通过直接触发支持多媒体内容生成功能的第一文本,使用户能够快速选择能够生成目标多媒体内容的文本,有利于进一步加快生成目标多媒体内容。
进一步的,在第一文本是由文字媒体内容的创作者确定的情况下,使得用户在基于文字媒体内容创作目标多媒体内容时能够适应文字媒体创作者的要求,在一定程度上能够保留文字媒体内容创作者的想法,尊重文字媒体创作者的选择。
在一些实施例中,在目标多媒体内容生成之后,用户可以选择将目标多媒体内容分享给目标对象。如图10所示,上述方法还包括如下步骤:
步骤320:显示分享控件;
在一些实施例中,分享控件用于触发对目标多媒体内容进行分享。
在一些实施例中,分享控件和目标多媒体内容是同时显示的。示例性的,如图11所示,在显示目标多媒体内容21的同时显示分享控件22。
在一些实施例中,在目标多媒体内容已显示的情况下,显示分享控件。也即,目标多媒体内容和分享控件是按照先后顺序进行显示的。
步骤340:响应于对分享控件的触发操作,将目标多媒体内容分享至指定网络位置。
在一些实施例中,针对分享控件的触发操作包括单击、双击、左右滑动、上下滑动、长按、悬停、面部识别、语音识别中的至少一种。值得说明的是,该针对分享控件的触发操作包括但不限于上述提及几种,本领域内的技术人员应该能够知晓,任意能够实现上述功能的操作均属于本申请实施例的保护范围。
通过分享控件可以实现用户对目标多媒体内容进行分享,有利于扩展目标多媒体内容的传播范围。通过对目标多媒体内容的分享,还能进一步实现对目标多媒体内容所对应的剧集进行宣传。
在一些实施例中,终端设备在生成目标多媒体内容的过程中,还会通过一些提示信息提示用户正在生成目标多媒体内容或目标多媒体内容生成成功或目标多媒体内容生成失败。
在一些实施例中,响应于对若干文本中的至少一个文字的选中操作,显示第一提示信息。在一些实施例中,响应于针对生成控件的触发操作,显示第一提示信息。第一提示信息用于指示正在生成目标多媒体内容。示例性的,如图12所示,显示第一提示信息23。
在一些实施例中,在目标多媒体内容生成失败的情况下,显示第二提示信息,第二提示信息用于指示目标多媒体内容生成失败。在一些实施例中,第二提示信息用于指示重新选择若干文本中的至少一个文字。
在一些实施例中,在目标多媒体内容生成成功的情况下,显示第三提示信息,第三提示信息用于指示目标多媒体内容生成成功。
通过第一提示信息/第二提示信息/第三提示信息,使用户能够及时获知目标多媒体内容的生成情况,有利于使用户及时了解到当前进度,以及及时根据对应的提示信息执行后续操作。
接下来,图13是本申请一个示例性实施例提供的媒体内容生成方法的示意图。具体介绍目标多媒体内容的生成过程。可选的,目标多媒体内容由服务器或客户端生成。本申请实施例中,以该方法由图1所示的服务器140执行为例进行说明。该方法包括:
步骤420:接收客户端发送的用于生成目标多媒体内容的目标文本;
在一些实施例中,接收客户端的用户选中的至少一个文字。也即,接收客户端发送的被选中的文本。
在一些实施例中,目标文本是从媒体账号发布的文字媒体内容中选择得到的。也即,目标文本就是被选中的文本。
在一些实施例中,目标文本是由服务器基于被选中的文本确定的。比如,使用语句分割算法对被选中的文本进行分析,将被选中的文本按句子进行分割,剔除被选中的文本中的不完整语句,将剩余完整语句确定为用于生成目标多媒体内容的目标文本。
在一些实施例中,目标文本是媒体账号发布的文字媒体内容中的呈突出显示状态的第一文本,突出显示状态用于指示第一文本支持多媒体内容生成功能。
在一些实施例中,第一文本是基于文字媒体内容的关联剧集素材与文字媒体内容的匹配度确定的。
可选的,与文字媒体内容关联的关联剧集素材是根据文字媒体内容所对应的文本语义和关联剧集素材中的剧集台词字幕的文本语义之间的相似度确定的。计算文字媒体内容所对应的文本语义和关联剧集素材中的剧集台词字幕的文本语义之间的语义相似度,将语义相似度大于或等于语义相似度阈值的剧集素材作为与文字媒体内容相关联的关联剧集素材。进一步地,还可以按照语义相似度的大小,对关联剧集素材进行排序。
可选的,以文字媒体内容中的句子为单位,拆分文字媒体内容。将拆分后的句子与关联剧集素材中的剧集台词字幕进行匹配。计算拆分后的句子与关联剧集素材中的剧集台词字幕的相似度。在拆分后的句子与关联剧集素材中的剧集台词字幕的文本相似度大于或等于文本相似度阈值的情况下,将拆分后的句子确定为第一文本。进一步地,还可以按照文本相似度的大小,对第一文本进行排序。
在一些实施例中,第一文本的相关信息是在媒体账号发布文字媒体内容时同步发送给相关用户的。第一文本在客户端的显示方式,与文字媒体内容中除第一文本以外的文本的显示方式是不同的。示例性的,第一文本呈突出显示状态,文字媒体内容中除第一文本以外的文本呈普通显示状态。
在一些实施例中,第一文本的相关信息中包括与第一文本相关的显示信息、与第一文本相关的链接、与第一文本相关的关联剧集素材、与第一文本相关的关联目标多媒体内容中的至少一种。其中,与第一文本相关的显示信息用于指示第一文本在客户端中的显示方式。与第一文本相关的链接用于指示第一文本支持多媒体内容生成功能。比如,响应于对第一文本的触发操作,可以跳转至多媒体内容生成界面。与第一文本相关的关联剧集素材用于指示第一文本对应的关联剧集素材,该关联剧集素材用于生成与第一文本相关的目标多媒体内容。与第一文本相关的关联目标多媒体内容是指预生成的目标多媒体内容。此时,在用户触发第一文本的情况下,可以直接获取该关联目标多媒体内容。
需要说明的是,本申请实施例中所述的确定第一文本的服务器或客户端,与生成多媒体内容的服务器或客户端,可以是相同的或是不同的。本申请实施例中对此不做限定。
在一些实施例中,目标媒体内容包括动画、海报、音频、视频中的至少之一。
步骤440:基于目标文本生成目标多媒体内容;
在一些实施例中,基于目标文本在剧集数据库内匹配对应的剧集画面和剧集台词。其中,剧集数据库是服务器预先生成的。服务器预先对不同剧集进行拆分,得到包括剧集画面和剧集台词的剧集数据库。需要理解的是,本申请实施例中的剧集画面和剧集台词是相关对应的。通常情况下,多帧剧集画面可能对应同一段剧集台词。
在一些实施例中,基于与目标文本匹配的剧集画面和剧集台词,生成包括剧集画面和剧集台词的目标多媒体内容。
步骤460:向客户端发送目标多媒体内容。
综上所述,本实施例提供的方法,提供了一种自动生成目标多媒体内容的方式,在确定目标文本之后,即能直接显示基于目标文本生成的目标多媒体内容。相比于用户手动生成目标多媒体内容的方法,用户无需执行多个操作步骤,就可以一键式生成与目标文本匹配的目标多媒体内容,有利于提高生成目标多媒体内容的效率。
在一些实施例中,目标媒体内容包括与目标文本匹配的剧集画面和剧集台词。
在一些实施例中,目标多媒体内容还包括推广方提供的推广内容。比如,目标媒体内容还包括广告主提供的待推广的广告信息。
在一些实施例中,剧集画面和剧集台词之间存在预设对应关系。在上述图13所示的实施例的基础上,如图14所示,上述步骤440还可以替换为如下子步骤:
步骤441:基于目标文本在预设对应关系中,查询与目标文本匹配的剧集画面和剧集台词,剧集画面和剧集台词用于生成目标多媒体内容。
在一些实施例中,预设对应关系是服务器预先确定的。示例性的,以第一剧集为例,第一剧集所包括的剧集画面和剧集台词的预设对应关系由如下方法确定。如图15所示,在上述步骤441之前,上述方法还包括:
步骤520:获取第一剧集视频和剧集台词字幕;
在一些实施例中,获取第一剧集对应的第一剧集视频和剧集台词字幕。可选的,第一剧集视频和剧集台词字幕是从视频数据库中获取的。
步骤540:按照剧集台词字幕的字幕起止时间,将第一剧集视频划分为至少一个视频分片;
在一些实施例中,按照同一个剧集台词字幕所对应的字幕起始时间和字幕终止时间,将第一剧集视频划分为至少一个视频分片。需要理解的是,视频分片的第一数量大于或等于剧集台词字幕的第二数量。
示例性的,如图16所示,假设第一剧集视频对应两条剧集台词字幕,包括剧集台词字幕1和剧集台词字幕2。其中,剧集台词字幕1和剧集台词字幕2所对应的视频分片是不连续。也即,如图16所示,在剧集台词字幕1和剧集台词字幕2之间包括无字幕的视频片段。
本申请实施例中,以视频分片与剧集台词字幕一一对应为例进行举例说明。也即,每一个视频分片均对应有一个剧集台词字幕。示例性的,如图16所示,第一剧集视频可以划分为两个视频分片,包括视频分片1和视频分片2。其中,视频分片1对应剧集台词字幕1,视频分片2对应剧集台词字幕2。
步骤560:在视频分片中提取至少一个剧集画面;
在一些实施例中,以视频帧为单位,对视频分片进行划分。将视频分片划分为至少一个视频帧。从每个视频帧中提取剧集画面。需要理解的是,视频帧与剧集画面是一一对应的。
在一些实施例中,从视频分片中提取至少一个目标视频帧,每个目标视频帧的质量大于或等于质量阈值。示例性的,假设视频分片1中包括视频帧1、视频帧2和视频帧3。其中,视频帧1和视频帧2的质量大于质量阈值,视频帧3的质量小于质量阈值,则仅从视频分片1中提取视频帧1和视频帧2,而忽略视频帧3。
步骤580:基于至少一个剧集画面和剧集台词字幕生成预设对应关系。
在一些实施例中,根据提取到的至少一个剧集画面,生成剧集画面与剧集台词的预设对应关系。示例性的,如下述表1所示:
表1
在一些实施例中,剧集画面与剧集台词是一对一的对应关系。比如,剧集画面1对应剧集台词1。
在一些实施例中,剧集画面与剧集台词是多对一的对应关系。比如,剧集画面2和剧集画面3均对应剧集台词2。
通常情况下,剧集画面与剧集台词不会出现一对多的对应关系。但在一些特殊情况下,也可能存在剧集画面与剧集台词是一对多的对应关系,本申请实施例中对此不做限定。
通过预先确定剧集画面与剧集台词之间的预设对应关系,使服务器确定目标文本之后,可以根据目标文本,在预设对应关系中,查询与目标文本匹配的剧集画面和剧集台词,有利于提高匹配效率。
可选的,根据目标文本与剧集台词之间匹配度,以及预设对应关系中的剧集画面与剧集台词的对应关系,确定与目标文本匹配的剧集画面和剧集台词。其中,剧集台词用于连接剧集台词和目标文本。
在一些实施例中,剧集画面和剧集台词之间存在预设对应关系。在上述图13所示的实施例的基础上,如图17所示,上述步骤440还可以替换为如下子步骤:
步骤442:计算目标文本对应的第一特征向量,以及计算第一特征向量与至少一个第二特征向量的向量相似度;将向量相似度最高的第二特征向量对应的剧集画面和剧集台词,确定为与目标文本匹配的剧集画面和剧集台词,剧集画面和剧集台词用于生成目标多媒体内容。
在一些实施例中,将向量相似度大于或等于相似度阈值的第二特征向量对应的剧集画面和剧集台词,均确定为与目标文本匹配的剧集画面和剧集台词。
在一些实施例中,第二特征向量是服务器预先确定的。示例性的,以第二剧集为例,第二特征向量由如下方法确定。如图18所示,在上述步骤442之前,上述方法还包括:
步骤610:获取第二剧集视频;
在一些实施例中,第二剧集视频是从视频数据库中获取的。
步骤620:按照划分步长将第二剧集视频划分为至少一个视频分片;
在一些实施例中,划分步长是随机的。在一些实施例中,划分步长是预设置的。比如,划分步长为3秒。
在一些实施例中,按照划分步长对第二剧集视频进行连续划分。也即,本申请实施例中的至少一个视频分片是连续的。可以理解的是,将至少一个视频分片按照顺序组合即能得到完整的第二剧集视频。
步骤630:提取至少一个视频分片中每个视频分片的计算机视觉信息;
在一些实施例中,基于计算机视觉算法对每个视频分片进行提取,获得每个视频分片的计算机视觉信息。可选的,计算机视觉信息包括每个视频分片中的画面主体信息和/或文本信息。其中,画面主体信息包括人体信息、物体信息、风景信息等。
步骤640:基于每个视频分片的计算机视觉信息,生成每个视频分片的内容描述信息;
在一些实施例中,根据提取到的每个视频分片的计算机视觉信息,生成每个视频分片的内容描述信息。
在一些实施例中,基于多模态视觉识别模型,获取至少一个视频分片中每个视频分片的内容描述信息。此时,则无需执行上述步骤630,获取每个视频分片的计算机视觉信息。也即,可以直接基于多模态视觉识别模型,实现上述步骤630和步骤640。
步骤650:基于每个视频分片的内容描述信息,生成每个视频分片的第二特征向量。
在一些实施例中,基于每个视频分片的内容描述信息,以及向量数据库,生成每个视频分片的第二特征向量。其中,向量数据库中包括内容描述信息和特征向量之间的对应关系。
在一些实施例中,基于多模态嵌入算法,生成每个视频分片的第二特征向量。可选的,基于多模态嵌入算法,输入每个视频分片所对应的声音信息、图像信息、文本信息中的至少之二,生成每个视频分片所对应的第二特征向量。此时,则无需执行上述步骤630和步骤640。也即,可以直接基于多模态嵌入算法,实现上述步骤630至步骤650。
通过匹配第一特征向量和第二特征向量,使服务器确定目标文本之后,可以根据目标文本确定与目标文本匹配的剧集画面和剧集台词,有利于提高匹配效率。
在一些实施例中,在上述图18所示的实施例的基础上,如图19所示,上述方法还包括:
步骤720:在视频分片中提取至少一个剧集画面;
在一些实施例中,以视频帧为单位,对视频分片进行划分。将视频分片划分为至少一个视频帧。从每个视频帧中提取剧集画面。需要理解的是,视频帧与剧集画面是一一对应的。
在一些实施例中,从视频分片中提取至少一个目标视频帧,每个目标视频帧的质量大于或等于质量阈值。示例性的,假设视频分片1中包括视频帧1、视频帧2和视频帧3。其中,视频帧1和视频帧2的质量大于质量阈值,视频帧3的质量小于质量阈值,则仅从视频分片1中提取视频帧1和视频帧2,而忽略视频帧3。
步骤740:存储视频分片的至少一个剧集画面和第二特征向量之间的对应关系。
在一些实施例中,在已知每个视频分片对应的第二特征向量,以及每个视频分片对应的至少一个剧集画面的情况下,至少一个剧集画面与第二特征向量之间存在对应关系。示例性的,如下表2所示:
表2
在一些实施例中,剧集画面与第二特征向量是一对一的对应关系。比如,剧集画面1对应第二特征向量1。
在一些实施例中,剧集画面与第二特征向量是多对一的对应关系。比如,剧集画面1和剧集画面2均对应第二特征向量2。
在一些实施例中,剧集画面与第二特征向量是一对多的对应关系。比如,剧集画面1既对应第二特征向量1,又对应第二特征向量2。
综上所述,本实施例提供的方法,通过存储视频分片的至少一个剧集画面和第二特征向量之间的对应关系,使得服务器在获得目标文本之后,可以尽快根据至少一个剧集画面和第二特征向量之间的对应关系,以及目标文本对应的第一特征向量,及时查找到目标文本对应的剧集画面。
在一些实施例中,在提取视频分片所对应的至少一个剧集画面时,根据每个视频分片中的视频帧的质量确定。可选的,如图20所示,上述步骤560或步骤720,可替换为如下子步骤:
步骤820:按照至少一个打分因素对视频分片中的每个视频帧打分,得到每个视频帧对应的质量打分;
在一些实施例中,每个视频分片包括多个视频帧,多个视频帧中的每个视频帧均对应有质量分。
在一些实施例中,至少一个打分因素包括亮度、清晰度、画面主体的识别置信度中的至少一个。其中,亮度是指视频帧中的光亮度。清晰度是指视频帧中的画面清晰程度。画面主体的识别置信度是指视频帧中的画面主体被识别到的概率。
步骤840:基于质量打分,从视频分片中的每个视频帧中提取至少一个剧集画面。
在一些实施例中,基于每个视频帧的质量打分,从视频分片中的目标视频帧中提取至少一个剧集画面。其中,目标视频帧所对应的质量打分大于或等于质量阈值分。
在一些实施例中,按照视频分片中的每个视频帧的质量打分从高到低或从低到高进行排序,选择质量打分大于或等于质量阈值分的至少一个视频帧,从质量打分大于或等于质量阈值分的至少一个视频帧中提取至少一个剧集画面。
综上所述,本实施例提供的方法,通过对每个视频分段中的每个视频帧进行打分,使服务器能够基于优选的视频帧提取剧集画面,从而有利于保证提取到的视频画面所对应的亮度或清晰度或画面主体的识别置信度均优于阈值。
在一些实施例中,目标多媒体内容是基于与目标文本匹配的剧集画面和剧集台词对应生成的。在上述图13所示的实施例的基础上,如图21所示,上述步骤440还可替换为如下子步骤:
步骤443:在目标文本匹配有至少两个候选剧集画面的情况下,计算至少两个候选剧集画面的画面质量评分;
在一些实施例中,按照至少一个打分因素对每个候选剧集画面打分,得到每个候选剧集画面的画面质量评分。可选的,至少一个打分因素包括亮度、清晰度、画面主体的识别置信度中的至少一个。
步骤444:基于画面质量评分在至少两个候选剧集画面中,选择出目标剧集画面;
在一些实施例中,基于每个候选剧集画面的画面质量评分,从目标文本匹配的至少两个候选剧集画面中,选择出至少一个目标剧集画面。其中,目标剧集画面所对应的画面质量评分大于或等于画面质量评分阈值。
步骤445:基于目标剧集画面和与目标文本匹配的剧集台词,生成目标多媒体内容。
在一些实施例中,目标剧集画面包括至少一个,此时与目标文本匹配的剧集台词也可能包括至少一个。但是,需要理解的是,每个剧集画面对应有自己的剧集台词,也即剧集画面和剧集台词是一一对应的。比如,假设目标文本与第一剧集的第一剧集画面和第二剧集的第二剧集画面匹配。其中,第一剧集和第二剧集是不同的两个剧集。第一剧集画面对应剧集台词1,第二剧集画面对应剧集台词2。
上述剧集画面和剧集台词一一对应的情形仅适用于剧集画面存在对应的剧集台词的情况。在未查询到与目标文本匹配的剧集台词的情况下,则将目标文本或目标文本的一部分,生成为与目标文本匹配的剧集台词。
在一些实施例中,在目标文本匹配有一个候选剧集画面的情况下,将候选剧集画面确定为目标剧集画面。也即,在目标文本有且只有一个匹配的候选剧集画面的情况下,该候选剧集画面即为用于生成目标多媒体内容的目标剧集画面。
综上所述,本实施例提供的方法,通过计算每个候选剧集画面的画面质量评分,使服务器能够基于优选的候选剧集画面生成目标多媒体内容,从而有利于保证生成的目标多媒体内容所对应的剧集画面的质量较好。
在一些实施例中,目标多媒体内容是基于裁剪后的剧集画面生成的。在上述图13所示的实施例的基础上,如图22所示,在上述步骤460之前,上述方法还包括:
步骤920:识别剧集画面中的画面主体的包围框;
在一些实施例中,画面主体对应有优先级。按照剧集画面中的画面主体的优先级,对剧集画面中的画面主体进行识别,并且以包围框的方式框选对应的画面主体。
在一些实施例中,画面主体包括人脸、人体、动物、植物、家具、杂物、风景中的至少一种。可选的,人脸的优先级高于人体的优先级,人体的优先级高于动物的优先级,动物的优先级高于植物的优先级,植物的优先级高于家具的优先级,家具的优先级高于杂物的优先级,杂物的优先级高于风景的优先级。
示例性的,如图23所示,识别剧集画面中的画面主体24,并通过包围框25框选对应的画面主体24。
在一些实施例中,在同一优先级的画面主体包括多个的情况下,同时识别剧集画面中的同一优先级的画面主体的包围框。比如,假设同一剧集画面中包括多个人脸,则同时识别多个人脸,并以包围框的方式框选多个人脸。
步骤940:基于画面主体的包围框,对剧集画面裁剪,得到裁剪后的剧集画面。
在一些实施例中,基于已识别的画面主体的包围框,对剧集画面进行裁剪,得到裁剪后的剧集画面。通常情况下,裁剪后的剧集画面小于或等于裁剪前的剧集画面。
在一些实施例中,在基于剧集画面中的画面主体识别到至少两个包围框的情况下,上述步骤940还可替换为如下子步骤:
步骤941:在基于画面主体识别出至少两个包围框的情况下,融合至少两个包围框得到融合包围框;
在一些实施例中,在基于剧集画面中的画面主体识别到至少两个包围框的情况下,对识别到的至少两个包围框进行融合,得到融合包围框。可选的,融合包围框的面积是基于融合的至少两个包围框确定的。
在一些实施例中,融合包围框覆盖至少两个包围框中的全部或部分包围框。
在一些实施例中,在识别出至少两个不同画面主体的包围框且包围框的总数量大于第一阈值的情况下,基于至少两个不同画面主体分别对应的优先级,选择出至少一个待融合的包围框组。其中,包围框组中的包围框数量不超过第二阈值且包围框组中的包围框对应的优先级高于第三阈值。
通过限制包围框组中的包围框的数量,使最终基于包围框裁剪得到的剧集画面中的画面主体的数量也被限制在一定阈值内,有利于保证裁剪后的剧集画面中所包含的画面主体的数量较合理,避免整体画面杂乱。
在一些实施例中,对于至少一个待融合的包围框组中的每个包围框组,确定包围框组中的最大左上坐标、最大右上坐标、最小左下坐标、最小右下坐标。基于最大左上坐标、最大右上坐标、最小左下坐标、最小右下坐标,生成每个包围框组的融合包围框。
在一些实施例中,每个包围框均对应有左上坐标、右上坐标、左下坐标、右下坐标。示例性的,每个包围框可以通过(xstart,ystart)和(xend,yend)组合确定。其中,(xstart,ystart)表示左下坐标,(xstart,yend)表示左上坐标,(xend,ystart)表示右下坐标,(xend,yend)表示右上坐标。
在一些实施例中,在包围框组中包括多个包围框的情况下,融合包围框的可以通过min(x1start,x2start,….,xnstart),min(y1start,y2start,….,ynstart),max(x1end,x2end,….,xnend),max(y1end,y2end,….,ynend)确定。
步骤942:基于融合包围框对剧集画面裁剪,得到裁剪后的剧集画面。
在一些实施例中,基于确定的融合包围框,对剧集画面进行裁剪,得到裁剪后的剧集画面。通常情况下,裁剪后的剧集画面小于或等于裁剪前的剧集画面。
通过对剧集画面进行裁剪,使服务器在生成目标多媒体内容时能够基于裁剪后的剧集画面进行,从而有利于保证生成的目标多媒体内容所对应的剧集画面是基于裁剪得到的优选的剧集画面。
在一些实施例中,在得到裁剪后的剧集画面之后,上述方法还包括:基于裁剪后的剧集画面的宽高比,确定目标多媒体内容的布局为竖版布局或横版布局。示例性的,如图24所示,在裁剪后的剧集画面的宽高比小于或等于第一比值的情况下,确定目标多媒体内容的布局为竖版布局。比如,图24(a)所示,其中,第一区域26为放置与剧集相关的信息和/或目标文本的区域,第二区域27为剧集画面的边框区域,第三区域28为放置剧集画面的区域。在裁剪后的剧集画面的宽高比大于第二比值的情况下,确定目标多媒体内容的布局为横版布局。比如,图24(b)所示。
由此,通过对剧集画面进行裁剪能够实现将剧集画面中的画面主体合理地进行展示,使生成的目标多媒体内容更美观,有利于增加目标多媒体内容的吸引力。
在一些实施例中,基于裁剪后的剧集画面的宽高比确定目标多媒体内容的布局之后,剧集画面可能会出现无法完整填充整个内容区域的情况下。此时,可以通过提取剧集画面的主题颜色,将主题颜色填充至内容区域内除剧集画面以外的区域,有利于使得生成的目标多媒体内容更自然。
在一些实施例中,根据剧集台词所对应的文字内容,动态调整剧集台词的显示尺寸。
在一些实施例中,根据剧集台词所对应的文字内容的字数和用于显示剧集台词的文本区域的区域大小,动态调整剧集台词中的每个文字的显示尺寸。可选的,剧集台词中的每个文字所占据的文字区域大小,等于,用于显示剧集台词的文本区域的区域大小与剧集台词所对应的文字内容的字数的商。比如,假设用于显示剧集台词的文本区域的区域大小为10平方厘米,剧集台词所对应的文字内容的字数为10,则剧集台词中的每个文字所占据的文字区域的大小为1平方厘米。剧集台词中的每个文字的显示尺寸与每个文字所占据的文字区域的大小成正比。
在一些实施例中,剧集台词中的每个文字所占据的文字区域大小小于或等于文字区域最大值。示例性的,假设文字区域最大值为3平方厘米。用于显示剧集台词的文本区域的区域大小为10平方厘米,剧集台词所对应的文字内容的字数为2,由于10/2=5平方厘米,5大于3,因此剧集台词中的每个文字所占据的文字区域的大小为3平方厘米。
在一些实施例中,剧集台词中的每个文字所占据的文字区域大小大于或等于文字区域最小值。示例性的,假设文字区域最小值为1平方厘米。用于显示剧集台词的文本区域的区域大小为10平方厘米,剧集台词所对应的文字内容的字数为20,由于10/20=0.5平方厘米,0.5小于1,因此剧集台词中的每个文字所占据的文字区域的大小为1平方厘米。但在剧集台词中的文字数量大于文字区域的数量的情况下,按照剧集台词中的文字顺序动态显示剧集台词。
以使剧集台词适应性显示在目标多媒体内容的文本区域为目标,既要避免剧集台词在文本区域的显示尺寸过大,使目标多媒体内容的整体画面不美观;同时又要避免剧集台词在文本区域的显示尺寸过小,导致剧集台词的清晰度不佳。
在一些实施例中,目标多媒体内容还包括推广方提供的推广内容。比如,目标媒体内容还包括广告主提供的待推广的广告信息。也即,推广内容包括用于宣传或推广的传播内容。在一些实施例中,推广内容为动态的显示内容,或,推广内容为静态的显示内容。
在一些实施例中,推广内容是预设置的内容。比如,在开放一键生成目标多媒体内容的功能之前,推广方预先确定了需要推广的推广内容。或者,在一键生成目标多媒体内容的功能已开放之后,在用户使用该功能生成目标多媒体内容的过程中,可实时补充需要推广方需要推广的推广内容。有利于实现动态调整目标多媒体内容中的推广内容。
在一些实施例中,推广内容与上述剧集画面叠加显示。可选的,在推广内容与剧集画面叠加显示的情况下,推广内容以不遮挡剧集画面中的画面主体为原则进行显示。
在一些实施例中,推广内容作为上述剧集画面中的一种显示元素进行显示。示例性的,推广内容与剧集画面中的至少一个画面主体组合显示。可选的,推广内容是至少一个画面主体的装饰元素。示例性的,推广内容是剧集画面所示的场景中的一个场景物体。
在一些实施例中,推广内容与上述剧集台词组合显示。可选的,在推广内容为广告词的情况下,融合推广内容与剧集台词,得到融合文本。在显示剧集台词的显示区域,显示融合文本。
在一些实施例中,在目标多媒体内容是动态的显示内容的情况下,推广内容在目标多媒体内容的起始时刻显示,或,推广内容在目标多媒体内容的终止时刻显示。
在一些实施例中,本申请实施例提出的一种媒体内容生成方法能够用于生成电子海报。本申请实施例中,以该方法由图1所示的终端设备120执行为例进行说明。该方法包括:
步骤240-1:响应于对若干文本中的至少一个文字的选中操作,确定用于生成目标电子海报的目标文本;
在一些实施例中,针对若干文本中的至少一个文字的选中操作包括单击、双击、左右滑动、上下滑动、长按、悬停、面部识别、语音识别中的至少一种。值得说明的是,该针对若干文本中的至少一个文字的选中操作包括但不限于上述提及几种,本领域内的技术人员应该能够知晓,任意能够实现上述功能的操作均属于本申请实施例的保护范围。
在一些实施例中,响应于对若干文本中的至少一个文字的选中操作,向服务器发送用于生成目标电子海报的生成请求。可选的,用于生成目标电子海报的生成请求中包括基于选中的至少一个文字确定目标文本的请求。可选的,用于生成目标电子海报的生成请求中还包括基于目标文本生成目标电子海报的请求。可选的,目标文本是被选中的至少一个文字中的关键信息。
在一些实施例中,响应于对若干文本中的至少一个文字的选中操作,将选中的至少一个文字的显示态从第一显示态切换为第二显示态。其中,第一显示态是至少一个文字未被选中的状态,第二显示态是至少一个文字被选中的状态。
步骤260-1:显示基于目标文本生成的目标电子海报。
在一些实施例中,目标电子海报包括与目标文本匹配的剧集画面和剧集台词。
在一些实施例中,与目标文本匹配的剧集画面所对应的画面内容与目标文本所表达的文本内容相互匹配。可选的,与目标文本匹配的剧集画面所对应的画面内容的含义,与目标文本所表达的文本内容的含义是一致的。比如,假设目标文本为“xxx真美丽”,剧集画面1所对应的画面内容为“xxx”妆容艳丽,则认为剧集画面1与目标文本匹配。
在一些实施例中,与目标文本匹配的剧集台词所对应的文本内容与目标文本所表达的文本内容相互匹配。可选的,与目标文本匹配的剧集台词所对应的文本内容的含义,与目标文本所表达的文本内容的含义是一致的。可选的,目标文本匹配的剧集台词所对应的第一文本内容,与目标文本所表达的第二文本内容的匹配度大于或等于匹配度阈值。示例性的,假设目标文本为“xxx真美丽”,剧集台词为“xxx真好看”,“美丽”和“好看”所表达的含义一致,则认为剧集台词与目标文本匹配。
在一些实施例中,目标电子海报是服务器基于终端设备发送的用于生成目标电子海报的生成请求生成的。服务器基于用于生成目标电子海报的生成请求,确定与目标文本匹配的剧集画面和剧集台词,然后生成目标电子海报。之后,服务器向终端设备发送生成好的目标电子海报,终端设备接收以及显示该目标电子海报。
在一些实施例中,在终端设备上显示的目标电子海报是与目标文本的匹配度最高的一张目标电子海报。也即,服务器根据目标文本可能生成了多张目标电子海报,但多张目标电子海报与目标文本的匹配度是不同的。比如,假设目标电子海报1与目标文本的匹配度为90%,目标电子海报2与目标文本的匹配度为80%,则服务器仅会向终端设备发送匹配度最高的目标电子海报1。
在一些实施例中,在终端设备上显示的目标电子海报是与目标文本匹配的全部目标电子海报。可选的,在终端设备上显示的目标电子海报是与目标文本的匹配度大于匹配度阈值的目标电子海报。比如,假设目标电子海报1与目标文本的匹配度为90%,目标电子海报2与目标文本的匹配度为80%,目标电子海报3与目标文本的匹配度为60%,匹配度阈值为75%,则服务器仅会向终端设备发送目标电子海报1和目标电子海报2。
在一些实施例中,目标电子海报所对应的素材是静态的图片。在一些实施例中,目标电子海报所对应的素材是动态的视频帧。
在一些实施例中,目标电子海报还包括推广方提供的推广内容。比如,目标媒体内容还包括广告主提供的待推广的广告信息。
在一些实施例中,目标电子海报包括并列的上半区域和下半区域,上半区域显示有剧集画面,下半区域显示有剧集台词。
在一些实施例中,目标电子海报包括并列的上半区域和下半区域,上半区域显示有剧集台词,下半区域显示有剧集画面。
在一些实施例中,目标电子海报包括并列的左半区域和右半区域,左半区域显示有剧集台词,右半区域显示有剧集画面。
在一些实施例中,目标电子海报包括并列的左半区域和右半区域,左半区域显示有剧集画面,右半区域显示有剧集台词。
在一些实施例中,目标电子海报包括内容区域。内容区域显示有剧集画面,以及叠加在剧集画面上的剧集台词。可选的,剧集台词还可以是与剧集画面分开显示的。本申请实施例中对剧集画面和剧集台词的显示方式不做限定。
在一些实施例中,目标电子海报还包括信息区域。信息区域用于显示与剧集相关的信息和/或目标文本。
在一些实施例中,上述目标电子海报由客户端或服务器自动生成。在上述目标电子海报是由服务器生成的情况下,或上述目标电子海报是由与终端设备不同的其他终端生成的情况下,上述步骤240-1之后,步骤260-1之前,还包括如下步骤:
步骤251-1:发送用于生成目标电子海报的目标文本;
步骤252-1:接收基于目标文本生成的目标电子海报。
在一些实施例中,生成目标电子海报的功能需要由生成控件触发。上述步骤240-1还可以替换为如下子步骤:
步骤241-1:响应于对若干文本中的至少一个文字的选中操作,显示被选中的文本以及选中功能栏;
在一些实施例中,响应于对若干文本中的至少一个文字的选中操作,将被选中的文本从第一显示态切换为第二显示态。其中,第一显示态是未被选中的状态,第二显示态是被选中的状态。
在一些实施例中,响应于对若干文本中的至少一个文字的选中操作,显示选中功能栏。
在一些实施例中,选中功能栏包括生成控件。其中,生成控件用于触发生成目标电子海报。可选的,响应于对生成控件的触发操作,向服务器发送用于生成目标电子海报的生成请求。可选的,用于生成目标电子海报的生成请求中包括基于选中的至少一个文字确定目标文本的请求。可选的,用于生成目标电子海报的生成请求中还包括基于目标文本生成目标电子海报的请求。
在一些实施例中,选中功能栏还包括除生成控件以外的其他功能控件。可选的,选中功能栏还包括复制控件、分享控件、收藏控件中的至少一种。其中,复制控件用于复制选中的至少一个文字。分享控件用于分享选中的至少一个文字。收藏控件用于收藏选中的至少一个文字。
步骤242-1:响应于对生成控件的触发操作,将被选中的文本确定为用于生成目标电子海报的目标文本。
在一些实施例中,针对生成控件的触发操作包括单击、双击、左右滑动、上下滑动、长按、悬停、面部识别、语音识别中的至少一种。值得说明的是,该针对生成控件的触发操作包括但不限于上述提及几种,本领域内的技术人员应该能够知晓,任意能够实现上述功能的操作均属于本申请实施例的保护范围。
在一些实施例中,用于生成目标电子海报的目标文本是直接根据被选中的文本确定的。比如,将被选中的文本直接确定为用于生成目标电子海报的目标文本。
在一些实施例中,用于生成目标电子海报的目标文本是间接根据被选中的文本确定的。比如,使用语句分割算法对被选中的文本进行分析,将被选中的文本按句子进行分割,剔除被选中的文本中的不完整语句,将剩余完整语句确定为用于生成目标电子海报的目标文本。
在一些实施例中,若干文本包括呈突出显示状态的第一文本,突出显示状态用于指示第一文本支持电子海报生成功能。上述步骤241-1还可以替换为如下子步骤:
步骤2411-1:响应于对第一文本的选中操作,将第一文本显示为被选中的文本以及显示选中功能栏。
在一些实施例中,针对第一文本的选中操作包括单击、双击、左右滑动、上下滑动、长按、悬停、面部识别、语音识别中的至少一种。值得说明的是,该针对第一文本的选中操作包括但不限于上述提及几种,本领域内的技术人员应该能够知晓,任意能够实现上述功能的操作均属于本申请实施例的保护范围。
在一些实施例中,响应于对第一文本的选中操作,将第一文本从第一显示态切换为第二显示态。其中,第一显示态是未被选中的状态,第二显示态是被选中的状态。
在一些实施例中,响应于对第一文本的选中操作,显示选中功能栏。其中,选中功能栏包括生成控件。
在一些实施例中,文字媒体内容的创作者需要在发布文字媒体内容之前开启电子海报生成功能。在开启了电子海报生成功能的情况下,其他用户才能对文字媒体内容中的若干文本中的至少一个文字进行选择。在文字媒体内容的创作者未开启电子海报生成功能的情况下,即表示文字媒体内容的创作者不希望自己的创作内容用于生成相关目标电子海报,此时则无法通过对文字媒体内容中的若干文本中的至少一个文字进行选择,实现生成目标电子海报。
在一些实施例中,在目标电子海报生成之后,用户可以选择将目标电子海报分享给目标对象。上述方法还包括如下步骤:
步骤320-1:显示分享控件;
在一些实施例中,分享控件用于触发对目标电子海报进行分享。
在一些实施例中,分享控件和目标电子海报是同时显示的。
在一些实施例中,在目标电子海报已显示的情况下,显示分享控件。也即,目标电子海报和分享控件是按照先后顺序进行显示的。
步骤340-1:响应于对分享控件的触发操作,将目标电子海报分享至指定网络位置。
在一些实施例中,针对分享控件的触发操作包括单击、双击、左右滑动、上下滑动、长按、悬停、面部识别、语音识别中的至少一种。值得说明的是,该针对分享控件的触发操作包括但不限于上述提及几种,本领域内的技术人员应该能够知晓,任意能够实现上述功能的操作均属于本申请实施例的保护范围。
在一些实施例中,终端设备在生成目标电子海报的过程中,还会通过一些提示信息提示用户正在生成目标电子海报或目标电子海报生成成功或目标电子海报生成失败。
在一些实施例中,响应于对若干文本中的至少一个文字的选中操作,显示第一提示信息。在一些实施例中,响应于针对生成控件的触发操作,显示第一提示信息。第一提示信息用于指示正在生成目标电子海报。
在一些实施例中,在目标电子海报生成失败的情况下,显示第二提示信息,第二提示信息用于指示目标电子海报生成失败。在一些实施例中,第二提示信息用于指示重新选择若干文本中的至少一个文字。
在一些实施例中,在目标电子海报生成成功的情况下,显示第三提示信息,第三提示信息用于指示目标电子海报生成成功。
接下来,具体介绍目标电子海报的生成过程。可选的,目标电子海报由服务器或客户端生成。本申请实施例中,以该方法由图1所示的服务器140执行为例进行说明。该方法包括:
步骤420-1:接收客户端发送的用于生成目标电子海报的目标文本;
在一些实施例中,接收客户端的用户选中的至少一个文字。也即,接收客户端发送的被选中的文本。
在一些实施例中,目标文本是从媒体账号发布的文字媒体内容中选择得到的。也即,目标文本就是被选中的文本。
在一些实施例中,目标文本是由服务器基于被选中的文本确定的。比如,使用语句分割算法对被选中的文本进行分析,将被选中的文本按句子进行分割,剔除被选中的文本中的不完整语句,将剩余完整语句确定为用于生成目标电子海报的目标文本。
在一些实施例中,目标文本是媒体账号发布的文字媒体内容中的呈突出显示状态的第一文本,突出显示状态用于指示第一文本支持电子海报生成功能。
在一些实施例中,第一文本是基于文字媒体内容的关联剧集素材与文字媒体内容的匹配度确定的。
可选的,与文字媒体内容关联的关联剧集素材是根据文字媒体内容所对应的文本语义和关联剧集素材中的剧集台词字幕的文本语义之间的相似度确定的。计算文字媒体内容所对应的文本语义和关联剧集素材中的剧集台词字幕的文本语义之间的语义相似度,将语义相似度大于或等于语义相似度阈值的剧集素材作为与文字媒体内容相关联的关联剧集素材。进一步地,还可以按照语义相似度的大小,对关联剧集素材进行排序。
可选的,以文字媒体内容中的句子为单位,拆分文字媒体内容。将拆分后的句子与关联剧集素材中的剧集台词字幕进行匹配。计算拆分后的句子与关联剧集素材中的剧集台词字幕的相似度。在拆分后的句子与关联剧集素材中的剧集台词字幕的文本相似度大于或等于文本相似度阈值的情况下,将拆分后的句子确定为第一文本。进一步地,还可以按照文本相似度的大小,对第一文本进行排序。
在一些实施例中,第一文本的相关信息是在媒体账号发布文字媒体内容时同步发送给相关用户的。第一文本在客户端的显示方式,与文字媒体内容中除第一文本以外的文本的显示方式是不同的。示例性的,第一文本呈突出显示状态,文字媒体内容中除第一文本以外的文本呈普通显示状态。
在一些实施例中,第一文本的相关信息中包括与第一文本相关的显示信息、与第一文本相关的链接、与第一文本相关的关联剧集素材、与第一文本相关的关联目标电子海报中的至少一种。其中,与第一文本相关的显示信息用于指示第一文本在客户端中的显示方式。与第一文本相关的链接用于指示第一文本支持电子海报生成功能。比如,响应于对第一文本的触发操作,可以跳转至海报生成界面。与第一文本相关的关联剧集素材用于指示第一文本对应的关联剧集素材,该关联剧集素材用于生成与第一文本相关的目标电子海报。与第一文本相关的关联目标电子海报是指预生成的目标电子海报。此时,在用户触发第一文本的情况下,可以直接获取该关联目标电子海报。
需要说明的是,本申请实施例中所述的确定第一文本的服务器或客户端,与生成海报的服务器或客户端,可以是相同的或是不同的。本申请实施例中对此不做限定。
在一些实施例中,目标媒体内容包括动画、海报、音频、视频中的至少之一。
步骤440-1:基于目标文本生成目标电子海报;
在一些实施例中,基于目标文本在剧集数据库内匹配对应的剧集画面和剧集台词。其中,剧集数据库是服务器预先生成的。服务器预先对不同剧集进行拆分,得到包括剧集画面和剧集台词的剧集数据库。需要理解的是,本申请实施例中的剧集画面和剧集台词是相关对应的。通常情况下,多帧剧集画面可能对应同一段剧集台词。
在一些实施例中,基于与目标文本匹配的剧集画面和剧集台词,生成包括剧集画面和剧集台词的目标电子海报。
步骤460-1:向客户端发送目标电子海报。
在一些实施例中,目标媒体内容包括与目标文本匹配的剧集画面和剧集台词。
在一些实施例中,目标电子海报还包括推广方提供的推广内容。比如,目标媒体内容还包括广告主提供的待推广的广告信息。
在一些实施例中,剧集画面和剧集台词之间存在预设对应关系。上述步骤440-1还可以替换为如下子步骤:
步骤441-1:基于目标文本在预设对应关系中,查询与目标文本匹配的剧集画面和剧集台词,剧集画面和剧集台词用于生成目标电子海报。
在一些实施例中,预设对应关系是服务器预先确定的。示例性的,以第一剧集为例,第一剧集所包括的剧集画面和剧集台词的预设对应关系由如下方法确定。在上述步骤441-1之前,上述方法还包括:
步骤520-1:获取第一剧集视频和剧集台词字幕;
在一些实施例中,获取第一剧集对应的第一剧集视频和剧集台词字幕。可选的,第一剧集视频和剧集台词字幕是从视频数据库中获取的。
步骤540-1:按照剧集台词字幕的字幕起止时间,将第一剧集视频划分为至少一个视频分片;
在一些实施例中,按照同一个剧集台词字幕所对应的字幕起始时间和字幕终止时间,将第一剧集视频划分为至少一个视频分片。需要理解的是,视频分片的第一数量大于或等于剧集台词字幕的第二数量。
本申请实施例中,以视频分片与剧集台词字幕一一对应为例进行举例说明。也即,每一个视频分片均对应有一个剧集台词字幕。
步骤560-1:在视频分片中提取至少一个剧集画面;
在一些实施例中,以视频帧为单位,对视频分片进行划分。将视频分片划分为至少一个视频帧。从每个视频帧中提取剧集画面。需要理解的是,视频帧与剧集画面是一一对应的。
在一些实施例中,从视频分片中提取至少一个目标视频帧,每个目标视频帧的质量大于或等于质量阈值。
步骤580:基于至少一个剧集画面和剧集台词字幕生成预设对应关系。
在一些实施例中,根据提取到的至少一个剧集画面,生成剧集画面与剧集台词的预设对应关系。
可选的,根据目标文本与剧集台词之间匹配度,以及预设对应关系中的剧集画面与剧集台词的对应关系,确定与目标文本匹配的剧集画面和剧集台词。其中,剧集台词用于连接剧集台词和目标文本。
在一些实施例中,剧集画面和剧集台词之间存在预设对应关系。上述步骤440-1还可以替换为如下子步骤:
步骤442-1:计算目标文本对应的第一特征向量,以及计算第一特征向量与至少一个第二特征向量的向量相似度;将向量相似度最高的第二特征向量对应的剧集画面和剧集台词,确定为与目标文本匹配的剧集画面和剧集台词,剧集画面和剧集台词用于生成目标电子海报。
在一些实施例中,将向量相似度大于或等于相似度阈值的第二特征向量对应的剧集画面和剧集台词,均确定为与目标文本匹配的剧集画面和剧集台词。
在一些实施例中,第二特征向量是服务器预先确定的。示例性的,以第二剧集为例,第二特征向量由如下方法确定。在上述步骤442-1之前,上述方法还包括:
步骤610-1:获取第二剧集视频;
在一些实施例中,第二剧集视频是从视频数据库中获取的。
步骤620-1:按照划分步长将第二剧集视频划分为至少一个视频分片;
在一些实施例中,划分步长是随机的。在一些实施例中,划分步长是预设置的。比如,划分步长为3秒。
在一些实施例中,按照划分步长对第二剧集视频进行连续划分。也即,本申请实施例中的至少一个视频分片是连续的。可以理解的是,将至少一个视频分片按照顺序组合即能得到完整的第二剧集视频。
步骤630-1:提取至少一个视频分片中每个视频分片的计算机视觉信息;
在一些实施例中,基于计算机视觉算法对每个视频分片进行提取,获得每个视频分片的计算机视觉信息。可选的,计算机视觉信息包括每个视频分片中的画面主体信息和/或文本信息。其中,画面主体信息包括人体信息、物体信息、风景信息等。
步骤640-1:基于每个视频分片的计算机视觉信息,生成每个视频分片的内容描述信息;
在一些实施例中,根据提取到的每个视频分片的计算机视觉信息,生成每个视频分片的内容描述信息。
在一些实施例中,基于多模态视觉识别模型,获取至少一个视频分片中每个视频分片的内容描述信息。此时,则无需执行上述步骤630-1,获取每个视频分片的计算机视觉信息。也即,可以直接基于多模态视觉识别模型,实现上述步骤630-1和步骤640-1。
步骤650-1:基于每个视频分片的内容描述信息,生成每个视频分片的第二特征向量。
在一些实施例中,基于每个视频分片的内容描述信息,以及向量数据库,生成每个视频分片的第二特征向量。其中,向量数据库中包括内容描述信息和特征向量之间的对应关系。
在一些实施例中,基于多模态嵌入算法,生成每个视频分片的第二特征向量。可选的,基于多模态嵌入算法,输入每个视频分片所对应的声音信息、图像信息、文本信息中的至少之二,生成每个视频分片所对应的第二特征向量。此时,则无需执行上述步骤630-1和步骤640-1。也即,可以直接基于多模态嵌入算法,实现上述步骤630-1至步骤650-1。
在一些实施例中,上述方法还包括:
步骤720-1:在视频分片中提取至少一个剧集画面;
在一些实施例中,以视频帧为单位,对视频分片进行划分。将视频分片划分为至少一个视频帧。从每个视频帧中提取剧集画面。需要理解的是,视频帧与剧集画面是一一对应的。
在一些实施例中,从视频分片中提取至少一个目标视频帧,每个目标视频帧的质量大于或等于质量阈值。
步骤740-1:存储视频分片的至少一个剧集画面和第二特征向量之间的对应关系。
在一些实施例中,在已知每个视频分片对应的第二特征向量,以及每个视频分片对应的至少一个剧集画面的情况下,至少一个剧集画面与第二特征向量之间存在对应关系。
在一些实施例中,在提取视频分片所对应的至少一个剧集画面时,根据每个视频分片中的视频帧的质量确定。可选的,上述步骤560-1或步骤720-1,可替换为如下子步骤:
步骤820-1:按照至少一个打分因素对视频分片中的每个视频帧打分,得到每个视频帧对应的质量打分;
在一些实施例中,每个视频分片包括多个视频帧,多个视频帧中的每个视频帧均对应有质量分。
在一些实施例中,至少一个打分因素包括亮度、清晰度、画面主体的识别置信度中的至少一个。其中,亮度是指视频帧中的光亮度。清晰度是指视频帧中的画面清晰程度。画面主体的识别置信度是指视频帧中的画面主体被识别到的概率。
步骤840-1:基于质量打分,从视频分片中的每个视频帧中提取至少一个剧集画面。
在一些实施例中,基于每个视频帧的质量打分,从视频分片中的目标视频帧中提取至少一个剧集画面。其中,目标视频帧所对应的质量打分大于或等于质量阈值分。
在一些实施例中,按照视频分片中的每个视频帧的质量打分从高到低或从低到高进行排序,选择质量打分大于或等于质量阈值分的至少一个视频帧,从质量打分大于或等于质量阈值分的至少一个视频帧中提取至少一个剧集画面。
在一些实施例中,目标电子海报是基于与目标文本匹配的剧集画面和剧集台词对应生成的。上述步骤440-1还可替换为如下子步骤:
步骤443-1:在目标文本匹配有至少两个候选剧集画面的情况下,计算至少两个候选剧集画面的画面质量评分;
在一些实施例中,按照至少一个打分因素对每个候选剧集画面打分,得到每个候选剧集画面的画面质量评分。可选的,至少一个打分因素包括亮度、清晰度、画面主体的识别置信度中的至少一个。
步骤444-1:基于画面质量评分在至少两个候选剧集画面中,选择出目标剧集画面;
在一些实施例中,基于每个候选剧集画面的画面质量评分,从目标文本匹配的至少两个候选剧集画面中,选择出至少一个目标剧集画面。其中,目标剧集画面所对应的画面质量评分大于或等于画面质量评分阈值。
步骤445-1:基于目标剧集画面和与目标文本匹配的剧集台词,生成目标电子海报。
在一些实施例中,目标剧集画面包括至少一个,此时与目标文本匹配的剧集台词也可能包括至少一个。但是,需要理解的是,每个剧集画面对应有自己的剧集台词,也即剧集画面和剧集台词是一一对应的。比如,假设目标文本与第一剧集的第一剧集画面和第二剧集的第二剧集画面匹配。其中,第一剧集和第二剧集是不同的两个剧集。第一剧集画面对应剧集台词1,第二剧集画面对应剧集台词2。
上述剧集画面和剧集台词一一对应的情形仅适用于剧集画面存在对应的剧集台词的情况。在未查询到与目标文本匹配的剧集台词的情况下,则将目标文本或目标文本的一部分,生成为与目标文本匹配的剧集台词。
在一些实施例中,在目标文本匹配有一个候选剧集画面的情况下,将候选剧集画面确定为目标剧集画面。也即,在目标文本有且只有一个匹配的候选剧集画面的情况下,该候选剧集画面即为用于生成目标电子海报的目标剧集画面。
在一些实施例中,目标电子海报是基于裁剪后的剧集画面生成的。在上述步骤460-1之前,上述方法还包括:
步骤920-1:识别剧集画面中的画面主体的包围框;
在一些实施例中,画面主体对应有优先级。按照剧集画面中的画面主体的优先级,对剧集画面中的画面主体进行识别,并且以包围框的方式框选对应的画面主体。
在一些实施例中,画面主体包括人脸、人体、动物、植物、家具、杂物、风景中的至少一种。可选的,人脸的优先级高于人体的优先级,人体的优先级高于动物的优先级,动物的优先级高于植物的优先级,植物的优先级高于家具的优先级,家具的优先级高于杂物的优先级,杂物的优先级高于风景的优先级。
在一些实施例中,在同一优先级的画面主体包括多个的情况下,同时识别剧集画面中的同一优先级的画面主体的包围框。比如,假设同一剧集画面中包括多个人脸,则同时识别多个人脸,并以包围框的方式框选多个人脸。
步骤940-1:基于画面主体的包围框,对剧集画面裁剪,得到裁剪后的剧集画面。
在一些实施例中,基于已识别的画面主体的包围框,对剧集画面进行裁剪,得到裁剪后的剧集画面。通常情况下,裁剪后的剧集画面小于或等于裁剪前的剧集画面。
在一些实施例中,在基于剧集画面中的画面主体识别到至少两个包围框的情况下,上述步骤940-1还可替换为如下子步骤:
步骤941-1:在基于画面主体识别出至少两个包围框的情况下,融合至少两个包围框得到融合包围框;
在一些实施例中,在基于剧集画面中的画面主体识别到至少两个包围框的情况下,对识别到的至少两个包围框进行融合,得到融合包围框。可选的,融合包围框的面积是基于融合的至少两个包围框确定的。
在一些实施例中,融合包围框覆盖至少两个包围框中的全部或部分包围框。
在一些实施例中,在识别出至少两个不同画面主体的包围框且包围框的总数量大于第一阈值的情况下,基于至少两个不同画面主体分别对应的优先级,选择出至少一个待融合的包围框组。其中,包围框组中的包围框数量不超过第二阈值且包围框组中的包围框对应的优先级高于第三阈值。
通过限制包围框组中的包围框的数量,使最终基于包围框裁剪得到的剧集画面中的画面主体的数量也被限制在一定阈值内,有利于保证裁剪后的剧集画面中所包含的画面主体的数量较合理,避免整体画面杂乱。
在一些实施例中,对于至少一个待融合的包围框组中的每个包围框组,确定包围框组中的最大左上坐标、最大右上坐标、最小左下坐标、最小右下坐标。基于最大左上坐标、最大右上坐标、最小左下坐标、最小右下坐标,生成每个包围框组的融合包围框。
在一些实施例中,每个包围框均对应有左上坐标、右上坐标、左下坐标、右下坐标。示例性的,每个包围框可以通过(xstart,ystart)和(xend,yend)组合确定。其中,(xstart,ystart)表示左下坐标,(xstart,yend)表示左上坐标,(xend,ystart)表示右下坐标,(xend,yend)表示右上坐标。
在一些实施例中,在包围框组中包括多个包围框的情况下,融合包围框的可以通过min(x1start,x2start,….,xnstart),min(y1start,y2start,….,ynstart),max(x1end,x2end,….,xnend),max(y1end,y2end,….,ynend)确定。
步骤942-1:基于融合包围框对剧集画面裁剪,得到裁剪后的剧集画面。
在一些实施例中,基于确定的融合包围框,对剧集画面进行裁剪,得到裁剪后的剧集画面。通常情况下,裁剪后的剧集画面小于或等于裁剪前的剧集画面。
通过对剧集画面进行裁剪,使服务器在生成目标电子海报时能够基于裁剪后的剧集画面进行,从而有利于保证生成的目标电子海报所对应的剧集画面是基于裁剪得到的优选的剧集画面。
在一些实施例中,在得到裁剪后的剧集画面之后,上述方法还包括:基于裁剪后的剧集画面的宽高比,确定目标电子海报的布局为竖版布局或横版布局。由此,通过对剧集画面进行裁剪能够实现将剧集画面中的画面主体合理地进行展示,使生成的目标电子海报更美观,有利于增加目标电子海报的吸引力。
在一些实施例中,基于裁剪后的剧集画面的宽高比确定目标电子海报的布局之后,剧集画面可能会出现无法完整填充整个内容区域的情况下。此时,可以通过提取剧集画面的主题颜色,将主题颜色填充至内容区域内除剧集画面以外的区域,有利于使得生成的目标电子海报更自然。
在一些实施例中,根据剧集台词所对应的文字内容,动态调整剧集台词的显示尺寸。
在一些实施例中,根据剧集台词所对应的文字内容的字数和用于显示剧集台词的文本区域的区域大小,动态调整剧集台词中的每个文字的显示尺寸。可选的,剧集台词中的每个文字所占据的文字区域大小,等于,用于显示剧集台词的文本区域的区域大小与剧集台词所对应的文字内容的字数的商。比如,假设用于显示剧集台词的文本区域的区域大小为10平方厘米,剧集台词所对应的文字内容的字数为10,则剧集台词中的每个文字所占据的文字区域的大小为1平方厘米。剧集台词中的每个文字的显示尺寸与每个文字所占据的文字区域的大小成正比。
在一些实施例中,剧集台词中的每个文字所占据的文字区域大小小于或等于文字区域最大值。示例性的,假设文字区域最大值为3平方厘米。用于显示剧集台词的文本区域的区域大小为10平方厘米,剧集台词所对应的文字内容的字数为2,由于10/2=5平方厘米,5大于3,因此剧集台词中的每个文字所占据的文字区域的大小为3平方厘米。
在一些实施例中,剧集台词中的每个文字所占据的文字区域大小大于或等于文字区域最小值。示例性的,假设文字区域最小值为1平方厘米。用于显示剧集台词的文本区域的区域大小为10平方厘米,剧集台词所对应的文字内容的字数为20,由于10/20=0.5平方厘米,0.5小于1,因此剧集台词中的每个文字所占据的文字区域的大小为1平方厘米。但在剧集台词中的文字数量大于文字区域的数量的情况下,按照剧集台词中的文字顺序动态显示剧集台词。
以使剧集台词适应性显示在目标电子海报的文本区域为目标,既要避免剧集台词在文本区域的显示尺寸过大,使目标电子海报的整体画面不美观;同时又要避免剧集台词在文本区域的显示尺寸过小,导致剧集台词的清晰度不佳。
图25示出了本申请一个示例性实施例提供的媒体内容生成装置的框图。该装置包括:
显示模块2510,用于显示媒体账号发布的文字媒体内容。
在一些实施例中,媒体账号是指支持编辑以及发布文字媒体内容的客户端中所登录的用户账号。可选的,媒体账号是指客户端中支持编辑以及发布文字媒体内容的应用程序所对应的用户账号。比如,社交程序中所登录的用户账号。可选的,该社交程序需要支持编辑以及发布文字媒体内容。
在一些实施例中,文字媒体内容包括若干文本。可选的,文字媒体内容还包括图片、视频等其他形式的内容。示例性的,文字媒体内容包括公众号文章、时事评论文章等。
在一些实施例中,文字媒体内容是基于媒体账号对应的创作者预先创作得到的。
在一些实施例中,文字媒体内容还包括图片、视频等其他形式的内容。
确定模块2520,用于响应于对若干文本中的至少一个文字的选中操作,确定用于生成目标多媒体内容的目标文本。
在一些实施例中,针对若干文本中的至少一个文字的选中操作包括单击、双击、左右滑动、上下滑动、长按、悬停、面部识别、语音识别中的至少一种。值得说明的是,该针对若干文本中的至少一个文字的选中操作包括但不限于上述提及几种,本领域内的技术人员应该能够知晓,任意能够实现上述功能的操作均属于本申请实施例的保护范围。
在一些实施例中,响应于对若干文本中的至少一个文字的选中操作,向服务器发送用于生成目标多媒体内容的生成请求。可选的,用于生成目标多媒体内容的生成请求中包括基于选中的至少一个文字确定目标文本的请求。可选的,用于生成目标多媒体内容的生成请求中还包括基于目标文本生成目标多媒体内容的请求。可选的,目标文本是被选中的至少一个文字中的关键信息。
在一些实施例中,响应于对若干文本中的至少一个文字的选中操作,将选中的至少一个文字的显示态从第一显示态切换为第二显示态。其中,第一显示态是至少一个文字未被选中的状态,第二显示态是至少一个文字被选中的状态。
显示模块2510,还用于显示基于目标文本生成的目标多媒体内容。
在一些实施例中,目标多媒体内容包括与目标文本匹配的剧集画面和剧集台词。
在一些实施例中,与目标文本匹配的剧集画面所对应的画面内容与目标文本所表达的文本内容相互匹配。可选的,与目标文本匹配的剧集画面所对应的画面内容的含义,与目标文本所表达的文本内容的含义是一致的。
在一些实施例中,与目标文本匹配的剧集台词所对应的文本内容与目标文本所表达的文本内容相互匹配。可选的,与目标文本匹配的剧集台词所对应的文本内容的含义,与目标文本所表达的文本内容的含义是一致的。可选的,目标文本匹配的剧集台词所对应的第一文本内容,与目标文本所表达的第二文本内容的匹配度大于或等于匹配度阈值。
在一些实施例中,目标多媒体内容是服务器基于终端设备发送的用于生成目标多媒体内容的生成请求生成的。服务器基于用于生成目标多媒体内容的生成请求,确定与目标文本匹配的剧集画面和剧集台词,然后生成目标多媒体内容。之后,服务器向终端设备发送生成好的目标多媒体内容,终端设备接收以及显示该目标多媒体内容。
在一些实施例中,在终端设备上显示的目标多媒体内容是与目标文本的匹配度最高的一张目标多媒体内容。也即,服务器根据目标文本可能生成了多张目标多媒体内容,但多张目标多媒体内容与目标文本的匹配度是不同的。比如,假设目标多媒体内容1与目标文本的匹配度为90%,目标多媒体内容2与目标文本的匹配度为80%,则服务器仅会向终端设备发送匹配度最高的目标多媒体内容1。
在一些实施例中,在终端设备上显示的目标多媒体内容是与目标文本匹配的全部目标多媒体内容。可选的,在终端设备上显示的目标多媒体内容是与目标文本的匹配度大于匹配度阈值的目标多媒体内容。比如,假设目标多媒体内容1与目标文本的匹配度为90%,目标多媒体内容2与目标文本的匹配度为80%,目标多媒体内容3与目标文本的匹配度为60%,匹配度阈值为75%,则服务器仅会向终端设备发送目标多媒体内容1和目标多媒体内容2。
在一些实施例中,目标多媒体内容所对应的素材是静态的图片。在一些实施例中,目标多媒体内容所对应的素材是动态的视频帧。
在一些实施例中,生成目标多媒体内容的功能需要由生成控件触发。
在一些实施例中,目标多媒体内容还包括推广方提供的推广内容。比如,目标媒体内容还包括广告主提供的待推广的广告信息。
显示模块2510,还用于响应于对若干文本中的至少一个文字的选中操作,显示被选中的文本以及选中功能栏。
在一些实施例中,响应于对若干文本中的至少一个文字的选中操作,将被选中的文本从第一显示态切换为第二显示态。其中,第一显示态是未被选中的状态,第二显示态是被选中的状态。
在一些实施例中,响应于对若干文本中的至少一个文字的选中操作,显示选中功能栏。
在一些实施例中,选中功能栏包括生成控件。其中,生成控件用于触发生成目标多媒体内容。可选的,响应于对生成控件的触发操作,向服务器发送用于生成目标多媒体内容的生成请求。可选的,用于生成目标多媒体内容的生成请求中包括基于选中的至少一个文字确定目标文本的请求。可选的,用于生成目标多媒体内容的生成请求中还包括基于目标文本生成目标多媒体内容的请求。
在一些实施例中,选中功能栏还包括除生成控件以外的其他功能控件。可选的,选中功能栏还包括复制控件、分享控件、收藏控件中的至少一种。其中,复制控件用于复制选中的至少一个文字。分享控件用于分享选中的至少一个文字。收藏控件用于收藏选中的至少一个文字。
确定模块2520,用于响应于对生成控件的触发操作,将被选中的文本确定为用于生成目标多媒体内容的目标文本。
在一些实施例中,针对生成控件的触发操作包括单击、双击、左右滑动、上下滑动、长按、悬停、面部识别、语音识别中的至少一种。值得说明的是,该针对生成控件的触发操作包括但不限于上述提及几种,本领域内的技术人员应该能够知晓,任意能够实现上述功能的操作均属于本申请实施例的保护范围。
在一些实施例中,用于生成目标多媒体内容的目标文本是直接根据被选中的文本确定的。比如,将被选中的文本直接确定为用于生成目标多媒体内容的目标文本。
在一些实施例中,用于生成目标多媒体内容的目标文本是间接根据被选中的文本确定的。比如,使用语句分割算法对被选中的文本进行分析,将被选中的文本按句子进行分割,剔除被选中的文本中的不完整语句,将剩余完整语句确定为用于生成目标多媒体内容的目标文本。
在一些实施例中,若干文本包括呈突出显示状态的第一文本,突出显示状态用于指示第一文本支持多媒体内容生成功能。
显示模块2510,还用于响应于对第一文本的选中操作,将第一文本显示为被选中的文本以及显示选中功能栏。
在一些实施例中,针对第一文本的选中操作包括单击、双击、左右滑动、上下滑动、长按、悬停、面部识别、语音识别中的至少一种。值得说明的是,该针对第一文本的选中操作包括但不限于上述提及几种,本领域内的技术人员应该能够知晓,任意能够实现上述功能的操作均属于本申请实施例的保护范围。
在一些实施例中,响应于对第一文本的选中操作,将第一文本从第一显示态切换为第二显示态。其中,第一显示态是未被选中的状态,第二显示态是被选中的状态。
在一些实施例中,响应于对第一文本的选中操作,显示选中功能栏。其中,选中功能栏包括生成控件。
在一些实施例中,第一文本是文字媒体内容的创作者在发布文字媒体内容之前确定的。
在一些实施例中,文字媒体内容的创作者需要在发布文字媒体内容之前开启多媒体内容生成功能。在开启了多媒体内容生成功能的情况下,其他用户才能对文字媒体内容中的若干文本中的至少一个文字进行选择。在文字媒体内容的创作者未开启多媒体内容生成功能的情况下,即表示文字媒体内容的创作者不希望自己的创作内容用于生成相关目标多媒体内容,此时则无法通过对文字媒体内容中的若干文本中的至少一个文字进行选择,实现生成目标多媒体内容。
在一些实施例中,第一文本是后端服务器根据关联素材与文字媒体内容的匹配度,从文字媒体内容中确定的。
在一些实施例中,响应于针对至少一个目标文本中的任一目标文本的编辑操作,保留任一目标文本或删除任一目标文本。
在一些实施例中,在目标多媒体内容生成之后,用户可以选择将目标多媒体内容分享给目标对象。
显示模块2510,还用于显示分享控件。
在一些实施例中,分享控件用于触发对目标多媒体内容进行分享。
在一些实施例中,分享控件和目标多媒体内容是同时显示的。
在一些实施例中,在目标多媒体内容已显示的情况下,显示分享控件。也即,目标多媒体内容和分享控件是按照先后顺序进行显示的。
发送模块2530,用于响应于对分享控件的触发操作,将目标多媒体内容分享至指定网络位置。
在一些实施例中,针对分享控件的触发操作包括单击、双击、左右滑动、上下滑动、长按、悬停、面部识别、语音识别中的至少一种。值得说明的是,该针对分享控件的触发操作包括但不限于上述提及几种,本领域内的技术人员应该能够知晓,任意能够实现上述功能的操作均属于本申请实施例的保护范围。
在一些实施例中,目标多媒体内容包括并列的上半区域和下半区域,上半区域显示有剧集画面,下半区域显示有剧集台词。
在一些实施例中,目标多媒体内容包括并列的上半区域和下半区域,上半区域显示有剧集台词,下半区域显示有剧集画面。
在一些实施例中,目标多媒体内容包括并列的左半区域和右半区域,左半区域显示有剧集台词,右半区域显示有剧集画面。
在一些实施例中,目标多媒体内容包括并列的左半区域和右半区域,左半区域显示有剧集画面,右半区域显示有剧集台词。
在一些实施例中,目标多媒体内容包括内容区域。内容区域显示有剧集画面,以及叠加在剧集画面上的剧集台词。可选的,剧集台词还可以是与剧集画面分开显示的。本申请实施例中对剧集画面和剧集台词的显示方式不做限定。
在一些实施例中,目标多媒体内容还包括信息区域。信息区域用于显示与剧集相关的信息和/或目标文本。
在一些实施例中,终端设备在生成目标多媒体内容的过程中,还会通过一些提示信息提示用户正在生成目标多媒体内容或目标多媒体内容生成成功或目标多媒体内容生成失败。
在一些实施例中,响应于对若干文本中的至少一个文字的选中操作,显示第一提示信息。在一些实施例中,响应于针对生成控件的触发操作,显示第一提示信息。第一提示信息用于指示正在生成目标多媒体内容。
在一些实施例中,在目标多媒体内容生成失败的情况下,显示第二提示信息,第二提示信息用于指示目标多媒体内容生成失败。在一些实施例中,第二提示信息用于指示重新选择若干文本中的至少一个文字。
在一些实施例中,在目标多媒体内容生成成功的情况下,显示第三提示信息,第三提示信息用于指示目标多媒体内容生成成功。
在一些实施例中,发送模块2530,还用于向服务器发送用于生成目标多媒体内容的生成请求。
在一些实施例中,上述装置还包括:
接收模块2540,用于接收服务器发送的目标多媒体内容。
图26示出了本申请一个示例性实施例提供的媒体内容生成装置的框图。该装置包括:
接收模块2610,用于接收客户端发送的用于生成目标多媒体内容的目标文本。
在一些实施例中,接收客户端的用户选中的至少一个文字。也即,接收客户端发送的被选中的文本。
在一些实施例中,目标文本是从媒体账号发布的文字媒体内容中选择得到的。也即,目标文本就是被选中的文本。
在一些实施例中,目标文本是由服务器基于被选中的文本确定的。比如,使用语句分割算法对被选中的文本进行分析,将被选中的文本按句子进行分割,剔除被选中的文本中的不完整语句,将剩余完整语句确定为用于生成目标多媒体内容的目标文本。
在一些实施例中,目标文本是媒体账号发布的文字媒体内容中的呈突出显示状态的第一文本,突出显示状态用于指示第一文本支持多媒体内容生成功能。
在一些实施例中,第一文本是基于文字媒体内容的关联剧集素材与文字媒体内容的匹配度确定的。
可选的,与文字媒体内容关联的关联剧集素材是根据文字媒体内容所对应的文本语义和关联剧集素材中的剧集台词字幕的文本语义之间的相似度确定的。计算文字媒体内容所对应的文本语义和关联剧集素材中的剧集台词字幕的文本语义之间的语义相似度,将语义相似度大于或等于语义相似度阈值的剧集素材作为与文字媒体内容相关联的关联剧集素材。进一步地,还可以按照语义相似度的大小,对关联剧集素材进行排序。
可选的,以文字媒体内容中的句子为单位,拆分文字媒体内容。将拆分后的句子与关联剧集素材中的剧集台词字幕进行匹配。计算拆分后的句子与关联剧集素材中的剧集台词字幕的相似度。在拆分后的句子与关联剧集素材中的剧集台词字幕的文本相似度大于或等于文本相似度阈值的情况下,将拆分后的句子确定为第一文本。进一步地,还可以按照文本相似度的大小,对第一文本进行排序。
在一些实施例中,目标媒体内容包括动画、海报、音频、视频中的至少之一。
生成模块2620,用于基于目标文本生成目标多媒体内容。
在一些实施例中,基于目标文本在剧集数据库内匹配对应的剧集画面和剧集台词。其中,剧集数据库是服务器预先生成的。服务器预先对不同剧集进行拆分,得到包括剧集画面和剧集台词的剧集数据库。需要理解的是,本申请实施例中的剧集画面和剧集台词是相关对应的。通常情况下,多帧剧集画面可能对应同一段剧集台词。
在一些实施例中,基于与目标文本匹配的剧集画面和剧集台词,生成包括剧集画面和剧集台词的目标多媒体内容。
发送模块2630,用于向客户端发送目标多媒体内容。
在一些实施例中,目标媒体内容包括与目标文本匹配的剧集画面和剧集台词。
在一些实施例中,目标多媒体内容还包括推广方提供的推广内容。比如,目标媒体内容还包括广告主提供的待推广的广告信息。
在一些实施例中,剧集画面和剧集台词之间存在预设对应关系。
生成模块2620,还用于基于目标文本在预设对应关系中,查询与目标文本匹配的剧集画面和剧集台词,剧集画面和剧集台词用于生成目标多媒体内容。
在一些实施例中,预设对应关系是服务器预先确定的。
生成模块2620,还用于获取第一剧集视频和剧集台词字幕。
在一些实施例中,获取第一剧集对应的第一剧集视频和剧集台词字幕。可选的,第一剧集视频和剧集台词字幕是从视频数据库中获取的。
生成模块2620,还用于按照剧集台词字幕的字幕起止时间,将第一剧集视频划分为至少一个视频分片。
在一些实施例中,按照同一个剧集台词字幕所对应的字幕起始时间和字幕终止时间,将第一剧集视频划分为至少一个视频分片。需要理解的是,视频分片的第一数量大于或等于剧集台词字幕的第二数量。
生成模块2620,还用于在视频分片中提取至少一个剧集画面。
在一些实施例中,以视频帧为单位,对视频分片进行划分。将视频分片划分为至少一个视频帧。从每个视频帧中提取剧集画面。需要理解的是,视频帧与剧集画面是一一对应的。
在一些实施例中,从视频分片中提取至少一个目标视频帧,每个目标视频帧的质量大于或等于质量阈值。
生成模块2620,还用于基于至少一个剧集画面和剧集台词字幕生成预设对应关系。
在一些实施例中,根据提取到的至少一个剧集画面,生成剧集画面与剧集台词的预设对应关系。
在一些实施例中,剧集画面与剧集台词是一对一的对应关系。
在一些实施例中,剧集画面与剧集台词是多对一的对应关系。
通常情况下,剧集画面与剧集台词不会出现一对多的对应关系。但在一些特殊情况下,也可能存在剧集画面与剧集台词是一对多的对应关系,本申请实施例中对此不做限定。
在一些实施例中,剧集画面和剧集台词之间存在预设对应关系。
生成模块2620,还用于计算目标文本对应的第一特征向量,以及计算第一特征向量与至少一个第二特征向量的向量相似度;将向量相似度最高的第二特征向量对应的剧集画面和剧集台词,确定为与目标文本匹配的剧集画面和剧集台词,剧集画面和剧集台词用于生成目标多媒体内容。
在一些实施例中,将向量相似度大于或等于相似度阈值的第二特征向量对应的剧集画面和剧集台词,均确定为与目标文本匹配的剧集画面和剧集台词。
在一些实施例中,第二特征向量是服务器预先确定的。
生成模块2620,还用于获取第二剧集视频。
在一些实施例中,第二剧集视频是从视频数据库中获取的。
生成模块2620,还用于按照划分步长将第二剧集视频划分为至少一个视频分片。
在一些实施例中,划分步长是随机的。在一些实施例中,划分步长是预设置的。
在一些实施例中,按照划分步长对第二剧集视频进行连续划分。也即,本申请实施例中的至少一个视频分片是连续的。可以理解的是,将至少一个视频分片按照顺序组合即能得到完整的第二剧集视频。
生成模块2620,还用于提取至少一个视频分片中每个视频分片的计算机视觉信息。
在一些实施例中,基于计算机视觉算法对每个视频分片进行提取,获得每个视频分片的计算机视觉信息。可选的,计算机视觉信息包括每个视频分片中的画面主体信息和/或文本信息。其中,画面主体信息包括人体信息、物体信息、风景信息等。
生成模块2620,还用于基于每个视频分片的计算机视觉信息,生成每个视频分片的内容描述信息。
在一些实施例中,根据提取到的每个视频分片的计算机视觉信息,生成每个视频分片的内容描述信息。
在一些实施例中,基于多模态视觉识别模型,获取至少一个视频分片中每个视频分片的内容描述信息。
生成模块2620,还用于基于每个视频分片的内容描述信息,生成每个视频分片的第二特征向量。
在一些实施例中,基于每个视频分片的内容描述信息,以及向量数据库,生成每个视频分片的第二特征向量。其中,向量数据库中包括内容描述信息和特征向量之间的对应关系。
在一些实施例中,基于多模态嵌入算法,生成每个视频分片的第二特征向量。可选的,基于多模态嵌入算法,输入每个视频分片所对应的声音信息、图像信息、文本信息中的至少之二,生成每个视频分片所对应的第二特征向量。
生成模块2620,还用于在视频分片中提取至少一个剧集画面。
在一些实施例中,以视频帧为单位,对视频分片进行划分。将视频分片划分为至少一个视频帧。从每个视频帧中提取剧集画面。需要理解的是,视频帧与剧集画面是一一对应的。
在一些实施例中,从视频分片中提取至少一个目标视频帧,每个目标视频帧的质量大于或等于质量阈值。示例性的,假设视频分片1中包括视频帧1、视频帧2和视频帧3。其中,视频帧1和视频帧2的质量大于质量阈值,视频帧3的质量小于质量阈值,则仅从视频分片1中提取视频帧1和视频帧2,而忽略视频帧3。
生成模块2620,还用于存储视频分片的至少一个剧集画面和第二特征向量之间的对应关系。
在一些实施例中,在已知每个视频分片对应的第二特征向量,以及每个视频分片对应的至少一个剧集画面的情况下,至少一个剧集画面与第二特征向量之间存在对应关系。
在一些实施例中,剧集画面与第二特征向量是一对一的对应关系。
在一些实施例中,剧集画面与第二特征向量是多对一的对应关系。
在一些实施例中,剧集画面与第二特征向量是一对多的对应关系。
在一些实施例中,在提取视频分片所对应的至少一个剧集画面时,根据每个视频分片中的视频帧的质量确定。
生成模块2620,还用于按照至少一个打分因素对视频分片中的每个视频帧打分,得到每个视频帧对应的质量打分。
在一些实施例中,每个视频分片包括多个视频帧,多个视频帧中的每个视频帧均对应有质量分。
在一些实施例中,至少一个打分因素包括亮度、清晰度、画面主体的识别置信度中的至少一个。其中,亮度是指视频帧中的光亮度。清晰度是指视频帧中的画面清晰程度。画面主体的识别置信度是指视频帧中的画面主体被识别到的概率。
生成模块2620,还用于基于质量打分,从视频分片中的每个视频帧中提取至少一个剧集画面。
在一些实施例中,基于每个视频帧的质量打分,从视频分片中的目标视频帧中提取至少一个剧集画面。其中,目标视频帧所对应的质量打分大于或等于质量阈值分。
在一些实施例中,按照视频分片中的每个视频帧的质量打分从高到低或从低到高进行排序,选择质量打分大于或等于质量阈值分的至少一个视频帧,从质量打分大于或等于质量阈值分的至少一个视频帧中提取至少一个剧集画面。
在一些实施例中,目标多媒体内容是基于与目标文本匹配的剧集画面和剧集台词对应生成的。
生成模块2620,还用于在目标文本匹配有至少两个候选剧集画面的情况下,计算至少两个候选剧集画面的画面质量评分。
在一些实施例中,按照至少一个打分因素对每个候选剧集画面打分,得到每个候选剧集画面的画面质量评分。可选的,至少一个打分因素包括亮度、清晰度、画面主体的识别置信度中的至少一个。
生成模块2620,还用于基于画面质量评分在至少两个候选剧集画面中,选择出目标剧集画面。
在一些实施例中,基于每个候选剧集画面的画面质量评分,从目标文本匹配的至少两个候选剧集画面中,选择出至少一个目标剧集画面。其中,目标剧集画面所对应的画面质量评分大于或等于画面质量评分阈值。
生成模块2620,还用于基于目标剧集画面和与目标文本匹配的剧集台词,生成目标多媒体内容。
在一些实施例中,目标剧集画面包括至少一个,此时与目标文本匹配的剧集台词也可能包括至少一个。但是,需要理解的是,每个剧集画面对应有自己的剧集台词,也即剧集画面和剧集台词是一一对应的。比如,假设目标文本与第一剧集的第一剧集画面和第二剧集的第二剧集画面匹配。其中,第一剧集和第二剧集是不同的两个剧集。第一剧集画面对应剧集台词1,第二剧集画面对应剧集台词2。
上述剧集画面和剧集台词一一对应的情形仅适用于剧集画面存在对应的剧集台词的情况。在未查询到与目标文本匹配的剧集台词的情况下,则将目标文本或目标文本的一部分,生成为与目标文本匹配的剧集台词。
在一些实施例中,在目标文本匹配有一个候选剧集画面的情况下,将候选剧集画面确定为目标剧集画面。也即,在目标文本有且只有一个匹配的候选剧集画面的情况下,该候选剧集画面即为用于生成目标多媒体内容的目标剧集画面。
在一些实施例中,目标多媒体内容是基于裁剪后的剧集画面生成的。
生成模块2620,还用于识别剧集画面中的画面主体的包围框。
在一些实施例中,画面主体对应有优先级。按照剧集画面中的画面主体的优先级,对剧集画面中的画面主体进行识别,并且以包围框的方式框选对应的画面主体。
在一些实施例中,画面主体包括人脸、人体、动物、植物、家具、杂物、风景中的至少一种。可选的,人脸的优先级高于人体的优先级,人体的优先级高于动物的优先级,动物的优先级高于植物的优先级,植物的优先级高于家具的优先级,家具的优先级高于杂物的优先级,杂物的优先级高于风景的优先级。
在一些实施例中,在同一优先级的画面主体包括多个的情况下,同时识别剧集画面中的同一优先级的画面主体的包围框。比如,假设同一剧集画面中包括多个人脸,则同时识别多个人脸,并以包围框的方式框选多个人脸。
生成模块2620,还用于基于画面主体的包围框,对剧集画面裁剪,得到裁剪后的剧集画面。
在一些实施例中,基于已识别的画面主体的包围框,对剧集画面进行裁剪,得到裁剪后的剧集画面。通常情况下,裁剪后的剧集画面小于或等于裁剪前的剧集画面。
生成模块2620,还用于在基于画面主体识别出至少两个包围框的情况下,融合至少两个包围框得到融合包围框。
在一些实施例中,在基于剧集画面中的画面主体识别到至少两个包围框的情况下,对识别到的至少两个包围框进行融合,得到融合包围框。可选的,融合包围框的面积是基于融合的至少两个包围框确定的。
在一些实施例中,融合包围框覆盖至少两个包围框中的全部或部分包围框。
在一些实施例中,在识别出至少两个不同画面主体的包围框且包围框的总数量大于第一阈值的情况下,基于至少两个不同画面主体分别对应的优先级,选择出至少一个待融合的包围框组。其中,包围框组中的包围框数量不超过第二阈值且包围框组中的包围框对应的优先级高于第三阈值。
通过限制包围框组中的包围框的数量,使最终基于包围框裁剪得到的剧集画面中的画面主体的数量也被限制在一定阈值内,有利于保证裁剪后的剧集画面中所包含的画面主体的数量较合理,避免整体画面杂乱。
在一些实施例中,对于至少一个待融合的包围框组中的每个包围框组,确定包围框组中的最大左上坐标、最大右上坐标、最小左下坐标、最小右下坐标。基于最大左上坐标、最大右上坐标、最小左下坐标、最小右下坐标,生成每个包围框组的融合包围框。
在一些实施例中,每个包围框均对应有左上坐标、右上坐标、左下坐标、右下坐标。示例性的,每个包围框可以通过(xstart,ystart)和(xend,yend)组合确定。其中,(xstart,ystart)表示左下坐标,(xstart,yend)表示左上坐标,(xend,ystart)表示右下坐标,(xend,yend)表示右上坐标。
在一些实施例中,在包围框组中包括多个包围框的情况下,融合包围框的可以通过min(x1start,x2start,….,xnstart),min(y1start,y2start,….,ynstart),max(x1end,x2end,….,xnend),max(y1end,y2end,….,ynend)确定。
生成模块2620,还用于基于融合包围框对剧集画面裁剪,得到裁剪后的剧集画面。
在一些实施例中,基于确定的融合包围框,对剧集画面进行裁剪,得到裁剪后的剧集画面。通常情况下,裁剪后的剧集画面小于或等于裁剪前的剧集画面。
图27示出了本申请一个示例性实施例提供的计算机设备的结构示意图。示意性的,计算机设备1100包括中央处理单元(Central Processing Unit,CPU)1101、包括随机存取存储器(Random Access Memory,RAM)1102和只读存储器(Read-Only Memory,ROM)1103的系统存储器1104,以及连接系统存储器1104和中央处理单元1101的系统总线1105。计算机设备1100还包括帮助计算机内的各个器件之间传输信息的基本输入/输出系统1106,和用于存储操作系统1113、应用程序1114和其他程序模块1115的大容量存储设备1107。
基本输入/输出系统1106包括有用于显示信息的显示器1108和用于用户输入信息的诸如鼠标、键盘之类的输入设备1109。其中显示器1108和输入设备1109都通过连接到系统总线1105的输入/输出控制器1110连接到中央处理单元1101。基本输入/输出系统1106还可以包括输入/输出控制器1110以用于接收和处理来自键盘、鼠标、或电子触控笔等多个其他设备的输入。类似地,输入/输出控制器1110还提供输出到显示屏、打印机或其他类型的输出设备。
大容量存储设备1107通过连接到系统总线1105的大容量存储控制器(未示出)连接到中央处理单元1101。大容量存储设备1107及其相关联的计算机可读介质为计算机设备1100提供非易失性存储。也就是说,大容量存储设备1107可以包括诸如硬盘或者只读光盘(Compact Disc Read-Only Memory,CD-ROM)驱动器之类的计算机可读介质(未示出)。
计算机可读介质可以包括计算机存储介质和通信介质。计算机存储介质包括以用于存储诸如计算机可读指令、数据结构、程序模块或其他数据等信息的任何方法或技术实现的易失性和非易失性、可移动和不可移动介质。计算机存储介质包括RAM、ROM、可擦除可编程只读存储器(Erasable Programmable Read-Only Memory,EPROM)、电可擦除可编程只读存储器(Electrically Erasable Programmable Read-Only Memory,EEPROM)、闪存或其他固态存储其技术,CD-ROM、数字通用光盘(Digital Versatile Disc,DVD)或其他光学存储、磁带盒、磁带、磁盘存储或其他磁性存储设备。当然,本领域技术人员可知计算机存储介质不局限于上述几种。上述的系统存储器1104和大容量存储设备1107可以统称为存储器。
根据本申请的各种实施例,计算机设备1100还可以通过诸如因特网等网络连接到网络上的远程计算机运行。也即计算机设备1100可以通过连接在系统总线1105上的网络接口单元1111连接到网络1112,或者说,也可以使用网络接口单元1111来连接到其他类型的网络或远程计算机系统(未示出)。
本申请一示例性实施例还提供了一种计算机可读存储介质,该计算机可读存储介质中存储有至少一段程序,至少一段程序由处理器加载并执行以实现上述各个方法实施例提供的媒体内容生成方法。
本申请一示例性实施例还提供了一种计算机程序产品,计算机程序产品包括至少一段程序,至少一段程序存储在可读存储介质中;通信设备的处理器从可读存储介质中读取信令,处理器执行信令,使得通信设备执行以实现上述各个方法实施例提供的媒体内容生成方法。
应当理解的是,在本文中提及的“多个”是指两个或两个以上。本领域技术人员在考虑说明书及实践这里公开的发明后,将容易想到本申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化,这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本申请未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的,本申请的真正范围和精神由下面的权利要求指出。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成,也可以通过程序来指令相关的硬件完成,所述的程序可以存储于一种计算机可读存储介质中,上述提到的存储介质可以是只读存储器,磁盘或光盘等。
以上仅为本申请的可选实施例,并不用以限制本申请,凡在本申请的精神和原则之内,所作的任何修改、等同切换、改进等,均应包含在本申请的保护范围之内。

Claims (20)

  1. 一种媒体内容生成方法,其特征在于,所述方法由终端设备执行,所述方法包括:
    显示文字媒体内容,所述文字媒体内容包括若干文本;
    响应于对所述若干文本中的至少一个文字的选中操作,确定用于生成目标多媒体内容的目标文本,所述目标文本包括被选中的所述至少一个文字的关键信息;
    显示基于所述目标文本生成的所述目标多媒体内容。
  2. 根据权利要求1所述的方法,其特征在于,所述响应于对所述若干文本中的至少一个文字的选中操作,确定用于生成目标多媒体内容的目标文本,包括:
    响应于对所述若干文本中的至少一个文字的选中操作,显示被选中的文本以及选中功能栏,所述选中功能栏包括生成控件;
    响应于对所述生成控件的触发操作,将所述被选中的文本确定为所述目标文本。
  3. 根据权利要求2所述的方法,其特征在于,所述若干文本包括呈突出显示状态的第一文本,所述突出显示状态用于指示所述第一文本支持多媒体内容生成功能;
    所述响应于对所述若干文本中的至少一个文字的选中操作,显示被选中的文本以及选中功能栏,包括:
    响应于对所述第一文本的选中操作,将所述第一文本显示为所述被选中的文本以及显示所述选中功能栏。
  4. 根据权利要求1至3任一所述的方法,其特征在于,
    所述目标媒体内容包括与所述目标文本匹配的剧集画面和剧集台词。
  5. 根据权利要求4所述的方法,其特征在于,
    所述目标多媒体内容还包括推广方提供的推广内容。
  6. 一种媒体内容生成方法,其特征在于,所述方法由服务器执行,所述方法包括:
    接收客户端发送的用于生成目标多媒体内容的目标文本,所述目标文本是从媒体账号发布的文字媒体内容中选择得到的;
    基于所述目标文本生成所述目标多媒体内容;
    向所述客户端发送所述目标多媒体内容。
  7. 根据权利要求6所述的方法,其特征在于,所述目标媒体内容包括与所述目标文本匹配的剧集画面和剧集台词;
    所述基于所述目标文本生成所述目标多媒体内容,包括:
    基于所述目标文本在预设对应关系中,查询与所述目标文本匹配的所述剧集画面和所述剧集台词,所述剧集画面和所述剧集台词用于生成所述目标多媒体内容;或,
    计算所述目标文本对应的第一特征向量,以及计算所述第一特征向量与至少一个第二特征向量的向量相似度;
    将所述向量相似度最高的第二特征向量对应的剧集画面和剧集台词,确定为与所述目标文本匹配的所述剧集画面和所述剧集台词,所述剧集画面和所述剧集台词用于生成所述目标多媒体内容。
  8. 根据权利要求7所述的方法,其特征在于,所述基于所述目标文本在预设对应关系中,查询与所述目标文本匹配的所述剧集画面和所述剧集台词之前,还包括:
    获取第一剧集视频和剧集台词字幕;
    按照所述剧集台词字幕的字幕起止时间,将所述第一剧集视频划分为至少一个视频分片,所述视频分片与所述剧集台词字幕一一对应;
    在所述视频分片中提取至少一个剧集画面;
    基于所述至少一个剧集画面和所述剧集台词字幕生成所述预设对应关系。
  9. 根据权利要求7所述的方法,其特征在于,所述计算所述第一特征向量与至少一个第二特征向量的向量相似度之前,还包括:
    获取第二剧集视频;
    按照划分步长将所述第二剧集视频划分为至少一个视频分片;
    提取所述至少一个视频分片中每个视频分片的计算机视觉信息;
    基于所述每个视频分片的计算机视觉信息,生成所述每个视频分片的内容描述信息;
    基于所述每个视频分片的内容描述信息,生成所述每个视频分片的第二特征向量。
  10. 根据权利要求9所述的方法,其特征在于,所述方法还包括:
    在所述视频分片中提取至少一个剧集画面;
    存储所述视频分片的所述至少一个剧集画面和所述第二特征向量之间的对应关系。
  11. 根据权利要求8或10所述的方法,其特征在于,所述在所述视频分片中提取至少一个剧集画面,包括:
    按照至少一个打分因素对所述视频分片中的每个视频帧打分,得到所述每个视频帧对应的质量打分;
    基于所述质量打分,从所述视频分片中的每个视频帧中提取至少一个剧集画面;
    其中,所述至少一个打分因素包括亮度、清晰度、画面主体的识别置信度中的至少一个。
  12. 根据权利要求6至11任一所述的方法,其特征在于,所述基于所述目标文本生成所述目标多媒体内容,包括:
    在所述目标文本匹配有至少两个候选剧集画面的情况下,计算所述至少两个候选剧集画面的画面质量评分;
    基于所述画面质量评分在所述至少两个候选剧集画面中,选择出目标剧集画面;
    基于所述目标剧集画面和与所述目标文本匹配的剧集台词,生成所述目标多媒体内容。
  13. 根据权利要求7至12任一所述的方法,其特征在于,在所述基于所述目标文本生成所述目标多媒体内容之前,还包括:
    识别所述剧集画面中的画面主体的包围框;
    基于所述画面主体的包围框,对所述剧集画面裁剪,得到裁剪后的剧集画面。
  14. 根据权利要求13所述的方法,其特征在于,所述基于所述画面主体的包围框,对所述剧集画面裁剪,得到裁剪后的剧集画面,包括:
    在基于所述画面主体识别出至少两个包围框的情况下,融合所述至少两个包围框得到融合包围框,所述融合包围框覆盖所述至少两个包围框中的全部或部分包围框;
    基于所述融合包围框对所述剧集画面裁剪,得到裁剪后的剧集画面。
  15. 根据权利要求14所述的方法,其特征在于,所述画面主体对应有优先级;
    所述在基于所述画面主体识别出至少两个包围框的情况下,融合所述至少两个包围框得到融合包围框,包括:
    在识别出至少两个不同画面主体的包围框且所述包围框的总数量大于第一阈值的情况下,基于所述至少两个不同画面主体分别对应的优先级,选择出至少一个待融合的包围框组;所述包围框组中的包围框数量不超过第二阈值且所述包围框组中的包围框对应的优先级高于第三阈值;
    对于所述至少一个待融合的包围框组中的每个包围框组,确定所述包围框组中的最大左上坐标、最大右上坐标、最小左下坐标、最小右下坐标;
    基于所述最大左上坐标、最大右上坐标、最小左下坐标、最小右下坐标,生成所述每个包围框组的融合包围框。
  16. 一种媒体内容生成装置,其特征在于,所述装置包括:
    显示模块,用于显示媒体账号发布的文字媒体内容,所述文字媒体内容包括若干文本;
    确定模块,用于响应于对所述若干文本中的至少一个文字的选中操作,确定用于生成目标多媒体内容的目标文本;
    所述显示模块,还用于显示基于所述目标文本生成的所述目标多媒体内容。
  17. 一种媒体内容生成装置,其特征在于,所述装置包括:
    接收模块,用于接收客户端发送的用于生成目标多媒体内容的目标文本,所述目标文本是从媒体账号发布的文字媒体内容中选择得到的;
    生成模块,用于基于所述目标文本生成所述目标多媒体内容;
    发送模块,用于向所述客户端发送所述目标多媒体内容。
  18. 一种计算机设备,其特征在于,所述计算机设备包括:处理器和存储器,所述存储器中存储有至少一段程序,所述至少一段程序由所述处理器加载并执行,以实现如权利要求1至15任一项所述的媒体内容生成方法。
  19. 一种计算机可读存储介质,其特征在于,所述计算机可读存储介质中存储有至少一段程序,所述至少一段程序由处理器加载并执行以实现如权利要求1至15任一项所述的媒体内容生成方法。
  20. 一种计算机程序产品,其特征在于,所述计算机程序产品包括至少一段程序,所述至少一段程序存储在计算机可读存储介质中;计算机设备的处理器从所述计算机可读存储介质中读取所述至少一段程序,所述处理器执行所述至少一段程序,使得所述计算机设备执行如权利要求1至15任一项所述的媒体内容生成方法。
PCT/CN2025/098557 2024-07-12 2025-05-30 媒体内容生成方法、装置、设备、存储介质及程序产品 Pending WO2026012008A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202410942622.XA CN121327163A (zh) 2024-07-12 2024-07-12 媒体内容生成方法、装置、设备及存储介质
CN202410942622.X 2024-07-12

Publications (1)

Publication Number Publication Date
WO2026012008A1 true WO2026012008A1 (zh) 2026-01-15

Family

ID=98331455

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2025/098557 Pending WO2026012008A1 (zh) 2024-07-12 2025-05-30 媒体内容生成方法、装置、设备、存储介质及程序产品

Country Status (2)

Country Link
CN (1) CN121327163A (zh)
WO (1) WO2026012008A1 (zh)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109756751A (zh) * 2017-11-07 2019-05-14 腾讯科技(深圳)有限公司 多媒体数据处理方法及装置、电子设备、存储介质
CN110704647A (zh) * 2018-06-22 2020-01-17 北京搜狗科技发展有限公司 一种内容处理方法及装置
CN112579826A (zh) * 2020-12-07 2021-03-30 北京字节跳动网络技术有限公司 视频显示及处理方法、装置、系统、设备、介质

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109756751A (zh) * 2017-11-07 2019-05-14 腾讯科技(深圳)有限公司 多媒体数据处理方法及装置、电子设备、存储介质
CN110704647A (zh) * 2018-06-22 2020-01-17 北京搜狗科技发展有限公司 一种内容处理方法及装置
CN112579826A (zh) * 2020-12-07 2021-03-30 北京字节跳动网络技术有限公司 视频显示及处理方法、装置、系统、设备、介质

Also Published As

Publication number Publication date
CN121327163A (zh) 2026-01-13

Similar Documents

Publication Publication Date Title
US11544814B2 (en) Method and system for displaying a video poster based on artificial intelligence
US11451858B2 (en) Method and system of processing information flow and method of displaying comment information
US20230057963A1 (en) Video playing method, apparatus and device, storage medium, and program product
US10573039B2 (en) Techniques for incorporating a text-containing image into a digital image
WO2022134698A1 (zh) 视频处理方法及装置
CN112040289B (zh) 视频播放控制方法、装置、视频播放设备和可读存储介质
CN113766296B (zh) 直播画面的展示方法和装置
WO2019018434A1 (en) AUTOMATIC VIGNETTE CENTERED ON AN ACTOR / PERSON
CN111612873A (zh) Gif图片生成方法、装置及电子设备
WO2022193070A1 (zh) 直播视频交互方法、装置、设备及存储介质
WO2022247293A1 (zh) 视频直播方法和视频直播装置
CN113312516B (zh) 一种视频处理方法及相关装置
CN114139491A (zh) 一种数据处理方法、装置及存储介质
WO2023273853A1 (zh) 一种图像处理方法、装置、设备及存储介质
CN114268848B (zh) 一种视频生成方法、装置、电子设备及存储介质
WO2022088834A1 (zh) 动态影集生成方法、服务器、显示终端及可读存储介质
CN113888564B (zh) 旅游产品的广告图像展示方法、系统、设备及存储介质
CN114222147A (zh) 直播布局调整方法、装置、存储介质及计算机设备
CN110636322B (zh) 多媒体数据的处理方法、装置、智能终端及存储介质
WO2023236815A1 (zh) 三维模型传输方法及其装置、存储介质、程序产品
CN110267079B (zh) 待播放视频中人脸的替换方法和装置
CN113596574A (zh) 视频处理方法、视频处理装置、电子设备和可读存储介质
CN121327163A (zh) 媒体内容生成方法、装置、设备及存储介质
CN116137648B (zh) 视频处理方法、装置、电子设备和计算机可读存储介质
CN118524252A (zh) 视频处理方法、装置、电子设备及可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 25835911

Country of ref document: EP

Kind code of ref document: A1