CN111479168A - Method, device, server and medium for marking multimedia content hot spot - Google Patents

Method, device, server and medium for marking multimedia content hot spot Download PDF

Info

Publication number
CN111479168A
CN111479168A CN202010289336.XA CN202010289336A CN111479168A CN 111479168 A CN111479168 A CN 111479168A CN 202010289336 A CN202010289336 A CN 202010289336A CN 111479168 A CN111479168 A CN 111479168A
Authority
CN
China
Prior art keywords
multimedia
segment
statistics
interoperation
segments
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010289336.XA
Other languages
Chinese (zh)
Other versions
CN111479168B (en
Inventor
康战辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202010289336.XA priority Critical patent/CN111479168B/en
Publication of CN111479168A publication Critical patent/CN111479168A/en
Application granted granted Critical
Publication of CN111479168B publication Critical patent/CN111479168B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44204—Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44213—Monitoring of end-user related data
    • H04N21/44222—Analytics of user selections, e.g. selection of programs or purchase activity
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/4722—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting additional data associated with the content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/475—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
    • H04N21/4756—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data for rating content, e.g. scoring a recommended movie
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Social Psychology (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

A method, apparatus, server, and medium for marking multimedia content hotspots are disclosed. The method comprises the following steps: dividing the multimedia content into at least two multimedia segments; counting the interactive operation associated with each multimedia segment to obtain at least two interactive operation statistical results respectively corresponding to at least two multimedia segments; performing a test of significance of difference on the at least two interoperation statistical results to determine interoperation statistical results having significant difference; and determining at least one hotspot segment among the at least two multimedia segments based on the interoperation statistics having the significant differences.

Description

Method, device, server and medium for marking multimedia content hot spot
Technical Field
The present application relates to the field of multimedia content processing, and more particularly, to a method, an apparatus, a server, and a medium for marking a multimedia content hotspot.
Background
In the process of playing the video content, the hotspot segments of the video can be marked in the video content. In this way, when watching a video, a user can directly fast forward to a hot spot clip in the current video or directly watch the collection of the hot spot clips in the video. The manual implementation of the timestamp marking of the hot segments in the video requires a large amount of labor cost, and the identification of the hot segments may not be accurate due to the subjective effect of the manual marking.
Disclosure of Invention
According to an aspect of the present application, a method for marking a multimedia content hotspot is provided, comprising: dividing the multimedia content into at least two multimedia segments; counting the interactive operation associated with each multimedia segment to obtain at least two interactive operation statistical results respectively corresponding to at least two multimedia segments; performing a test of significance of difference on the at least two interoperation statistical results to determine interoperation statistical results having significant difference; and determining at least one hotspot segment among the at least two multimedia segments based on the interoperation statistics having the significant differences.
In some embodiments, the interactive operation includes an instant comment or play progress adjustment associated with the multimedia clip.
In some embodiments, the statistics of the interactive operations of each multimedia segment include the number of users performing the interactive operations and the number of users not performing the interactive operations in the time period of the multimedia segment.
In some embodiments, performing a test of significance of difference on the at least two interoperation statistics to determine interoperation statistics having significant difference comprises: for a first multimedia segment in two adjacent multimedia segments, determining the number of users who perform the interactive operation in users watching the first multimedia segment as a first user number and the number of users who do not perform the interactive operation as a second user number; for a second multimedia segment in two adjacent multimedia segments, determining the number of users who perform the interactive operation in users watching the second multimedia segment as a third user number and the number of users who do not perform the interactive operation as a fourth user number; determining whether the interactive operation statistics of the two adjacent multimedia fragments have a significant difference by using the first user number, the second user number, the third user number and the fourth user number; in the case that it is determined that there is a significant difference between the interoperation statistics of two adjacent multimedia segments, determining the interoperation statistics of one multimedia segment of the two adjacent multimedia segments as the interoperation statistics having the significant difference.
In some embodiments, determining at least one hotspot segment among the at least two multimedia segments based on the interoperation statistics having significant differences comprises: determining candidate interval marks of the hotspot segments based on the time stamp of one of the two adjacent multimedia segments; selecting a time interval between two candidate interval markers as a hotspot segment based on the order in which the candidate interval markers appear in the multimedia content.
In some embodiments, selecting a time interval between two candidate interval markers as a hotspot segment based on the order in which the candidate interval markers appear in the multimedia content comprises: the time interval between the 2m-1 occurrence of the candidate interval marker and the 2m occurrence of the candidate interval marker is determined as the hotspot segment, where m is an integer greater than zero.
In some embodiments, the statistics of interactions for each multimedia segment include a number of said interactions over a time period for that multimedia segment.
In some embodiments, performing a test of significance of difference on the at least two interoperation statistics to determine interoperation statistics having significant difference comprises: determining an average value and a standard deviation of the number of interactions of the at least two multimedia segments based on the number of interactions of each multimedia segment; and for each of the at least two multimedia segments, determining whether the number of interactions of the multimedia segment has a significant difference relative to the average value based on the average value and the standard deviation, and determining the statistical result of the interactions of the multimedia segment as the statistical result of the interactions having the significant difference if the number of interactions of the multimedia segment is determined to have the significant difference relative to the average value.
In some embodiments, determining that the number of interactions for the multimedia segment has a significant difference relative to the mean based on the mean and the standard deviation comprises: determining a standard score of the interoperation statistical result of the multimedia segment relative to the average value based on the standard deviation; and determining whether the interactive operation statistical result of the multimedia segment has difference significance based on the standard score.
In some embodiments, determining at least one hotspot segment among the at least two multimedia segments based on the interoperation statistics having significant differences comprises: for each of the at least two multimedia clips, determining the multimedia clip as a hotspot clip if the criterion score is greater than a predefined criterion score threshold.
According to another aspect of the present application, there is also provided an apparatus for marking a multimedia content hotspot, comprising: a pre-processing unit configured to segment multimedia content into at least two multimedia segments; a statistics unit configured to perform statistics on the interoperation associated with each multimedia segment to obtain at least two interoperation statistics corresponding to the at least two multimedia segments, respectively; a checking unit configured to perform a check of significance of difference on the at least two interoperation statistical results to determine interoperation statistical results having significant difference; and a hotspot marking unit configured to determine at least one hotspot segment among the at least two multimedia segments based on the interoperation statistics with significant differences.
In some embodiments, the interactive operation includes an instant comment or play progress adjustment associated with the multimedia clip.
In some embodiments, the statistics of the interactive operations of each multimedia segment include the number of users performing the interactive operations and the number of users not performing the interactive operations in the time period of the multimedia segment.
In some embodiments, the verification unit is configured to: for a first multimedia segment in two adjacent multimedia segments, determining the number of users who perform the interactive operation in users watching the first multimedia segment as a first user number and the number of users who do not perform the interactive operation as a second user number; for a second multimedia segment in two adjacent multimedia segments, determining the number of users who perform the interactive operation in users watching the second multimedia segment as a third user number and the number of users who do not perform the interactive operation as a fourth user number; determining whether the interactive operation statistics of the two adjacent multimedia fragments have a significant difference by using the first user number, the second user number, the third user number and the fourth user number; in the case that it is determined that there is a significant difference between the interoperation statistics of two adjacent multimedia segments, determining the interoperation statistics of one multimedia segment of the two adjacent multimedia segments as the interoperation statistics having the significant difference.
In some embodiments, the hotspot tagging unit is configured to: determining candidate interval marks of the hotspot segments based on the time stamp of one of the two adjacent multimedia segments; selecting a time interval between two candidate interval markers as a hotspot segment based on the order in which the candidate interval markers appear in the multimedia content.
In some embodiments, selecting a time interval between two candidate interval markers as a hotspot segment based on the order in which the candidate interval markers appear in the multimedia content comprises: the time interval between the 2m-1 occurrence of the candidate interval marker and the 2m occurrence of the candidate interval marker is determined as the hotspot segment, where m is an integer greater than zero.
In some embodiments, the statistics of interactions for each multimedia segment include a number of said interactions over a time period for that multimedia segment.
In some embodiments, the statistics unit is configured to: determining an average value and a standard deviation of the number of interactions of the at least two multimedia segments based on the number of interactions of each multimedia segment; and for each of the at least two multimedia segments, determining whether the number of interactions of the multimedia segment has a significant difference relative to the average value based on the average value and the standard deviation, and determining the statistical result of the interactions of the multimedia segment as the statistical result of the interactions having the significant difference if the number of interactions of the multimedia segment is determined to have the significant difference relative to the average value.
In some embodiments, determining that the number of interactions for the multimedia segment has a significant difference relative to the mean based on the mean and the standard deviation comprises: determining a standard score of the interoperation statistical result of the multimedia segment relative to the average value based on the standard deviation; for each of the at least two multimedia clips, determining the multimedia clip as a hotspot clip if the criterion score is greater than a predefined criterion score threshold.
In some embodiments, the hotspot tagging unit is configured to: determining the multimedia segment as a hotspot segment based on the criterion score being greater than a predefined criterion score threshold.
According to still another aspect of the present application, there is also provided a multimedia contents server including: a processor; and a memory having computer-readable program instructions stored therein, wherein the method as described above is performed when the computer-readable program instructions are executed by the processor.
According to yet another aspect of the present application, there is also provided a computer-readable storage medium having computer-readable instructions stored thereon which, when executed by a computer, the computer performs the method as previously described.
By using the method, the device, the server and the medium for marking the multimedia content hot spots, the multimedia segments with different significances in the multimedia content can be determined based on the statistical difference significance test results of the interactive operations in different time periods in the multimedia content, and the hot spot segments in the multimedia content can be determined according to the time stamps of the multimedia segments with different significances. The statistics of the interactive operation in different time periods in the multimedia content can reflect the user evaluation of the groupwise of the user when the user watches the content in different time periods in the multimedia content, so that the trending plot determined by the principle of the application is more accurate.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts. The following drawings are not intended to be drawn to scale in actual dimensions, with emphasis instead being placed upon illustrating the subject matter of the present application.
FIG. 1A illustrates an example of a user interaction interface according to the present application;
FIG. 1B illustrates an example of hot spot tagging in video content;
a schematic diagram of a list of hotspot segments is shown in FIG. 1C;
FIG. 2 illustrates an exemplary scene diagram of a video processing system according to the present application;
FIG. 3 shows a schematic flow diagram of a method for marking video hotspots in accordance with embodiments of the present application;
FIG. 4 illustrates an exemplary process for performing a difference significance test on two adjacent interoperation statistics based on the principles of Chi-Square-test according to an embodiment of the present application;
FIG. 5 illustrates an exemplary process of performing a difference significance test on two adjacent interoperation statistics based on the principles of the Z-score according to an embodiment of the present application;
FIG. 6 shows a schematic block diagram of an apparatus for marking video hotspots in accordance with embodiments of the present application; and
fig. 7 shows an architecture of an electronic device according to an embodiment of the application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the accompanying drawings, and obviously, the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without any creative effort also belong to the protection scope of the present application.
As used in this application and the appended claims, the terms "a," "an," "the," and/or "the" are not intended to be inclusive in the singular, but rather are intended to be inclusive in the plural unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that steps and elements are included which are explicitly identified, that the steps and elements do not form an exclusive list, and that a method or apparatus may include other steps or elements.
Although various references are made herein to certain modules in a system according to embodiments of the present application, any number of different modules may be used and run on a user terminal and/or server. The modules are merely illustrative and different aspects of the systems and methods may use different modules.
Flow charts are used herein to illustrate operations performed by systems according to embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in the exact order in which they are performed. Rather, the various steps may be processed in reverse order or simultaneously, as desired. Meanwhile, other operations may be added to the processes, or a certain step or several steps of operations may be removed from the processes.
Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making. The method and the device apply the artificial intelligence technology to the field of multimedia content processing, and determine whether the multimedia content has the hot spot fragments which are obviously different from other fragments or not by counting the interactive operation in the multimedia content, so that more accurate information of the hot spot fragments can be provided for a user. The hotspot segments referred to in the present application refer to multimedia content segments that are likely to be of higher interest to the user. As used herein, a hotspot segment that is significantly different from other segments refers to a hotspot segment in which the statistical features exceed the statistical features of the other segments by a certain threshold.
The technical scheme provided by the application particularly relates to a multimedia content processing technology. In order to improve the efficiency of playing the multimedia content in the process of playing the multimedia content to the user, a hotspot segment of the multimedia content can be marked in the played multimedia content, so that the user can directly watch a hotspot in the multimedia content.
In order to solve the problem of strong subjectivity in the process of manually labeling hot content, the application provides a method for determining hot segments based on the statistical result of user interaction behaviors of a user in the process of watching multimedia content, so that the hot segments in the multimedia content can be determined more effectively. Therefore, when a user who does not watch the multimedia content watches the multimedia content for the first time, the playing progress can be directly and quickly advanced to the hot spot segment or the collection of the hot spot segment can be directly watched according to the prompt.
FIG. 1A illustrates an example of a user interaction interface according to the present application. As shown in fig. 1A, during the viewing process, the user can make instant comments on the multimedia content, and such instant comments can be displayed in the form of a bullet screen in the user interface for playing the multimedia content. For example, in the example shown in fig. 1A, a user may input comment content through the input box 110 in the process of viewing multimedia content, and the comment content may be displayed in an interface for playing the multimedia content in a real-time manner. The instant comment sent by the user is illustrated in fig. 1A by a bullet screen. However, it will be appreciated by those skilled in the art that the user's instant comments may also be presented in any other form.
It can be understood that if a user currently viewing multimedia content is interested in the multimedia content being played, the user has a stronger chance of instant comment, that is, the number of users who send instant comments (or the number of instant comments sent by users) is greater. Thus, the hotspots in the multimedia content can be determined by counting the number of users sending instant comments (or the number of instant comments sent by the users).
FIG. 1B illustrates an example of hot spot tagging in multimedia content. In the example shown in FIG. 1B, points 120-1 and 120-2 are indicia of hot spot segments in the multimedia content. As can be seen, the current point in time (120-0) shown in FIG. 1B is located in the hotspot segment. The user may send more instant comments in the hotspot segment (e.g., more number of barrage shown in fig. 1B). Point 120-1 may be a starting point of a hotspot segment and point 120-2 may be an ending point of a hotspot segment in the multimedia content. In the process of viewing the multimedia content, the user can directly fast forward the progress bar to the time period of the hot spot clip (i.e., the time period from the point 120-1 to the point 120-2) to view the hot spot clip in the multimedia content.
Also shown in FIG. 1B is an icon 130. By selecting icon 130, the user can view a collection of hotspot segments in the multimedia content. In some embodiments, by selecting the icon 130, a mark indicating the hotspot segment may be displayed on a play progress bar displayed on the user interaction interface. The user can cause the progress bar to jump directly to the location of the hotspot segment by clicking on such a mark. In other embodiments, the user may be further presented with a list of one or more hotspot segments present in the multimedia content by selecting icon 130. A schematic diagram of a list of hotspot segments is shown in fig. 1C. In some examples, a user may view or listen by selecting one of the hotspot segments in the list of hotspot segments (e.g., hotspot segment 140-1 or 140-2 as shown in fig. 1C). In other examples, the user may choose to view a collection of hotspot segments, i.e., continuously view or listen to the content of all hotspot segments.
Although fig. 1A and 1B illustrate the user interaction behavior when multimedia content is viewed by taking a bullet screen as an example, the user interaction behavior related to the present application is not limited thereto. In fact, any interactive behavior performed by a user at a user interaction interface while viewing multimedia content may be used to implement hot spot tagging of multimedia content according to the principles provided herein. For example, the user interaction behavior may further include a user behavior of adjusting the playing progress (e.g., forward, backward, fast forward, fast backward, etc.), a query behavior performed by the user while viewing the multimedia content (e.g., searching for characters, plots, etc. related to the multimedia content), a user behavior of clicking a link on the current user interface while viewing the multimedia content, a user screen capture behavior for the current multimedia content playing interface, and so on.
Fig. 2 shows an exemplary scene diagram of a multimedia content processing system according to the present application. As shown in fig. 2, the multimedia contents processing system 200 may include a user terminal 210, a network 220, a multimedia contents server 230, and a database 240.
The user terminal 210 may be, for example, a computer 210-1, a cell phone 210-2 shown in fig. 2. It is to be appreciated that the user terminal can be virtually any electronic device capable of playing multimedia content to a user, including but not limited to desktop computers, laptop computers, tablet computers, smart phones, smart home devices, wearable devices, and the like.
The network 220 may be a single network, or a combination of at least two different networks. For example, the network 220 may include, but is not limited to, one or a combination of local area networks, wide area networks, public networks, private networks, and the like.
Server 230 may be a single server or a group of servers, with each server in the group being connected via a wired or wireless network. A group of servers may be centralized, such as a data center, or distributed. The server 230 may be local or remote.
The server 230 provided according to the present application may be configured to receive the multimedia content to be processed and mark a hot spot in the multimedia content to be processed by using the method provided in the present application. For example, the server 230 may receive the multimedia content to be processed from the user terminal. The user terminal can acquire the multimedia content to be processed through an image acquisition device (such as a camera, a video camera, a sound recorder and the like) arranged on the user terminal. For another example, the user terminal may also acquire multimedia content to be processed from an independently provided image capture device (e.g., a camera, a video camera, a scanner, etc.). For another example, the server 230 may also receive the multimedia content to be processed from a database via a network. The multimedia content to be processed as referred to herein may be video content comprising at least two consecutive image frames. Audio content comprising a time-length audio track, a combination of video and audio content and any user-perceptible form of content.
In some implementations, the server 230 may perform the multimedia content processing method provided herein by using an application built in the server. In other implementations, the server 230 may perform the multimedia content processing method provided herein by calling an application program stored outside the server.
In some examples, the server 230 may transmit the multimedia content to be processed to the user terminal, and perform the multimedia content processing method provided herein using a processing unit of the user terminal.
Database 240 may generally refer to a device having storage capabilities. The database 240 is mainly used to store various data utilized, generated, and outputted from the user terminal 210 and the server 230 in operation. The database 240 may be local or remote. The database 240 may include various memories such as a Random Access Memory (RAM), a Read Only Memory (ROM), and the like. The above mentioned storage devices are only examples and the storage devices that the system can use are not limited to these.
Database 240 may be interconnected or in communication with server 230, or a portion thereof, via network 220, or directly interconnected or in communication with server 230, or a combination thereof.
In some embodiments, database 240 may be a stand-alone device. In other embodiments, the database 240 may also be integrated in at least one of the user terminal 210 and the server 230. For example, the database 240 may be provided on the user terminal 210 or may be provided on the server 240. For another example, the database 240 may be distributed, and a part thereof may be provided on the user terminal 210 and another part thereof may be provided on the server 230.
The flow of the method for marking video or audio hotspots provided by the present application will be described in detail below. The principles of the present application will be described below in terms of video, however, one skilled in the art will appreciate that audio may be similarly processed to implement hot spot tagging for audio.
Fig. 3 shows a schematic flow diagram of a method for marking video hotspots according to an embodiment of the application. The method illustrated in fig. 3 may be performed using, for example, the user terminal 210 and/or the server 230 illustrated in fig. 2.
In step S302, the multimedia content to be processed may be divided into at least two multimedia segments. In some embodiments, the number of interactions in the multimedia content to be processed is greater than a predefined number threshold. For example, the number of instant comments in the video to be processed may be greater than 1000. The result of a video hotspot marked according to the principles of the present application will be more accurate when a sufficient number of interactive operations are included in the video to be processed.
The symbol T may be utilized hereinafteriRepresents the ith video segment in the video to be processed, wherein i is an integer greater than 0. In some embodiments, the video to be processed may be segmented at preset time intervals. In thatIn some examples, the preset time interval may be 5 s. That is, the time length of each video clip is 5 s.
It will be appreciated that the video to be processed may be segmented in any other manner by those skilled in the art, for example, each video segment may include a preset number of frames. The particular manner in which the video to be processed is segmented is not limited herein.
In step S304, the interaction operation associated with each multimedia segment may be counted to obtain an interaction operation statistical result for each multimedia segment.
In some embodiments, the interoperation associated with each video segment may be counted to obtain at least two interoperation statistics corresponding to the at least two video segments, respectively.
The interaction may include an instant comment or play progress adjustment associated with each video clip. In some examples, the interaction may include an instant comment (e.g., a bullet screen) or a play progress adjustment (e.g., forward, rewind, etc.) associated with a timestamp of any video frame in each video clip. In other examples, the interaction may also include a query behavior (e.g., search for video-related characters, plots, etc.) performed by the user while viewing the video content, a behavior of the user clicking on a link on the current user interface while viewing the video content, a screen capture behavior of the user for the current video playback interface, and so on, associated with the timestamp in each video segment. The principles of the present application will be described hereinafter with the interaction being an instant comment as an example.
In some implementations, the statistics of interactions for each video segment can be the number of users that have interacted with and the number of users that have not interacted with within the time period of the video segment. For example, the interoperation statistics for each video segment may be the number of users posting an instant comment within the time period of the video segment and the number of users not posting an instant comment within the time period of the video segment. The number of users who perform interactive operations and the number of users who do not perform interactive operations may refer to an absolute number, i.e., an actual number of users, or may refer to a relative number, i.e., a percentage of users who perform interactive operations among all users and a percentage of users who do not perform interactive operations among all users.
In other implementations, the interoperation statistics for each video segment include a number of interoperations within a time period of the video segment. For example, the interoperation statistics for each video segment may include the number of instant comments posted over the time period for that video segment.
In step S306, a test of the significance of the difference may be performed on the interaction statistics obtained in step S304. The interoperation statistics having significant differences may be determined by performing a test of significance of differences on at least two interoperation statistics. The interoperation statistics having significant differences may be used to indicate whether there is a video segment having significant differences from other video segments among the at least two video segments. For example, the interoperation statistics having a significant difference may indicate that there is a significant difference between the interoperation statistics of a certain video segment and the interoperation statistics of at least another video segment of the at least two video segments. In this case, the video clip may belong to a hotspot clip.
In some embodiments, in the case that the interactive operation statistics of each video segment are the number of users who have performed interactive operations and the number of users who have not performed interactive operations in the time period of the video segment. The test for difference significance may be performed on the interoperation statistics of two adjacent video segments based on the principle of chi-square test. In the case where it is determined that there is a significant difference between the interoperation statistics of the adjacent two multimedia segments, the interoperation statistics of one multimedia segment of the adjacent two multimedia segments may be determined as the interoperation statistics having the significant difference.
The process of performing the difference significance test on the interactive operation statistical results of two adjacent video segments based on the chi-square test principle will be described below with reference to fig. 4, and will not be described herein again.
By using the verification of the significance of the difference based on the principle of chi-square verification, it can be determined that the number of users who perform interactive operations in a certain video clip significantly changes from the number of users who perform interactive operations in the previous video clip of the video clip on the time axis of the video. For example, it may be determined that the number of users posting an instant comment in the video segment is significantly increased or decreased relative to the number of users interacting in the previous video segment based on the principle of chi-square test.
In other embodiments, where the interoperation statistics of each video segment include a number of interoperations within a time period of the video segment, a test of significance of difference may be performed on at least two interoperation statistics based on the principle of Z-score (Z-score). The process of performing the significance difference test on at least two interactive operation statistics based on the principle of Z-score will be described below with reference to fig. 5, and will not be described herein again.
With the verification of the significance of the difference performed on the basis of the principle of the Z-score, it can be determined whether the number of interactions occurring within a certain video segment time period has a significant difference with respect to the average of the number of interactions occurring within each video segment time period. In a case where it is determined that the number of the interactions of the multimedia segment has a significant difference with respect to the average value, it may be determined that the interaction statistics of the multimedia segment are the interactions statistics having the significant difference.
In step S308, at least one hotspot segment may be determined among the at least two multimedia segments based on the interoperation statistics with significant differences obtained in step S306.
As described above, in the case where the verification of the significance of the difference is performed using the principle of chi-square verification, it can be determined that the number of users who perform an interactive operation in a certain video clip significantly changes, for example, significantly increases or significantly decreases, from the number of users who perform an interactive operation in the previous video clip of the video clip on the time axis of the video. When the number of users interacting in the video clip increases significantly, it means that the video clip indicates the beginning of the hot spot clip. When the number of users interacting in the video clip is significantly reduced, it means that the video clip indicates the end of the hot clip.
In embodiments where the principle of the chi-squared test is used to perform a test of significance of differences, in the event that a significant difference is determined between the statistics of two adjacent video segments, a candidate span designation for a hotspot segment may be determined based on the timestamp of one of the two adjacent video segments. Wherein the candidate interval flag may be used to indicate the beginning or end of a hotspot segment.
For example, the timestamp of any one of the previous or next video clip (e.g., the first or last frame) of the two adjacent video clips may be determined as the candidate span mark of the hotspot clip. For another example, a position where the timestamp of any one of the previous video segment or the next video segment (e.g., the first frame or the last frame) in the two adjacent video segments is subtracted K seconds forward may be determined as the candidate interval marker. This is because it is considered that there is a lag in time from the point in time when the user starts viewing the hit section to the point in time when the user makes an instant comment. K may be any preset value, for example, K — 2.
In some embodiments, the time interval for at least one hotspot segment may be determined based on the order in which the candidate interval markers appear in the video. For example, a 2m-1 occurrence candidate interval marker may be determined to indicate the beginning of a hotspot segment, and a 2m occurrence candidate interval marker may be determined to indicate the end of a hotspot segment, where m is an integer greater than zero. Thus, the time period between the 2m-1 occurrence of the candidate interval marker and the 2m occurrence of the candidate interval marker may be determined as the time interval of the hotspot segment. Such a setting is based on the assumption that there is usually no hotspot segment at the beginning of the video. That is, there are generally fewer interactions involved in the segment at the beginning of the video. The first significant change, which was verified based on the principle of chi-square verification, represents a significant increase in the number of users interacting. Thus, the first candidate span flag implies the beginning of the first hotspot segment. The corresponding second significant change represents a significant reduction in the number of users performing the interaction after the hotspot segment ends. Thus, the second candidate span flag signifies the end of the first hotspot segment. Similarly, the third candidate span designation implies the beginning of the second hotspot segment, and so on.
In an embodiment where the verification of the significance of the difference is performed using the principle of Z-score, a video segment may be determined as the time interval of a hotspot segment in the case where it is determined that the number of interactions occurring within the time period of the video segment has a significant difference with respect to the average of the number of interactions occurring within the time period of each video segment.
By using the method for labeling the hot spot in the multimedia content, the multimedia segment with the difference significance in the video can be determined based on the statistical difference significance test result of the interactive operation in different time periods in the multimedia content, and the hot spot segment in the multimedia content can be determined according to the time stamp of the multimedia segment with the difference significance. The trending plots determined by the method are more accurate because the statistics of the interactive operations in different time periods in the multimedia content can reflect the user evaluation of the groupwise nature of the user when watching the content in different time periods in the multimedia content.
FIG. 4 illustrates an exemplary process for performing a significance of difference test on two adjacent interoperation statistics based on the principles of the Chi-Square-test according to an embodiment of the present application.
In step S402, for a first video segment of two adjacent video segments, the number of users performing an interactive operation among users watching the first video segment may be determined as a first number of users, and the number of users not performing an interactive operation may be determined as a second number of users.
In step S404, for a second video segment of the two adjacent video segments, the number of users performing the interactive operation among the users watching the second video segment may be determined as a third number of users, and the number of users not performing the interactive operation may be determined as a fourth number of users.
In step S406, it may be determined whether there is a significant difference between the interoperation statistics of the two adjacent video segments by using the first user number, the second user number, the third user number, and the fourth user number.
The following is a table of squares test to explain how to determine whether there is a significant difference between the interoperation statistics of two adjacent video segments using the first user number, the second user number, the third user number, and the fourth user number.
Assuming that there are two categorical variables X and Y with their value ranges { X1, X2} and { Y1, Y2}, the independent sample four-grid table for chi-square test shown in Table 1 can be constructed from X1, X2, Y1, Y2:
TABLE 1
y1 y2 Total of
x1 a b a+b
x2 c d c+d
Total of a+c b+d n＝a+b+c+d
Independence tests can be used to examine whether two variables are related.
According to the principles of the present application, adjacent two video segments T may be determined using chi-square testingiAnd Ti+1Whether there is a significant difference between the interoperation statistics of (1). The above classification variable X may be a video segment T in the video to be processed, and X1 may be the ith video segment TiAnd x2 can be the (i + 1) th video segment Ti+1. The above classification variable Y may be a statistical result of the interactive operations of the video segment, Y1 may refer to the number of users who have performed the interactive operations among the users who have viewed the video segment, and Y2 may refer to the number of users who have not performed the interactive operations among the users who have viewed the video segment. The hypothesis "video segment T" can be verified using the independent sample four-grid table created in the manner described aboveiNumber of users performing interactive operations and video segment Ti+1Whether there is no significant difference in the number of users performing the interactive operation "is true.
Based on the definition of the classification variable X, Y, a in the four tables of the independent samples refers to the viewing video segment TiB refers to viewing of the video segment TiC refers to viewing of the video segment Ti+1D refers to the number of users who are viewing the video segment Ti+1N is the sum of a, b, c, d above.
Based on the independent sample four-grid table, the test statistic χ can be determined based on formula (1)2：
Figure BDA0002449782290000141
Test statistic χ obtained based on formula (1)2And searching the corresponding probability value in the chi-square distribution critical value table. When the found probability value is smaller than the predetermined probability threshold, it can be determined that the interoperation statistics of the two adjacent video segments have a significant difference.
Table 2 shows a chi-square distribution critical value table, in which P (χ)2≧ k) represents a test statistic χ2Probability greater than parameter k:
TABLE 2
P(χ2≥k) 0.5 0.4 0.25 0.15 0.1
k 0.455 0.708 1.323 2.072 2.706
P(χ2≥k) 0.05 0.025 0.010 0.005 0.001
k 3.841 5.024 6.635 7.879 10.828
For example, the probability threshold may be predetermined to be 0.05. Test statistic χ determined based on equation (1)2Is greater than 3.841, it can be considered to correspond to the test statistic χ2Is less than 0.05, i.e. assume "video segment TiNumber of users performing interactive operations and video segment Ti+1The probability of "there is no significant difference in the number of users performing the interactive operation" is less than 0.05. Therefore, the inter-operation statistics of two adjacent video segments can be considered to have a significant difference. In the case where it is determined that there is a significant difference between the interoperation statistics of the adjacent two multimedia segments, the interoperation statistics of one multimedia segment of the adjacent two multimedia segments may be determined as the interoperation statistics having the significant difference.
FIG. 5 illustrates an exemplary process of performing a difference significance test on two adjacent interoperation statistics based on the principles of Z-scores according to an embodiment of the present application.
In step S502, the average μ and the standard deviation σ of the numbers of interoperations of at least two video segments included in the video may be determined based on the number of interoperations of each video segment.
In step S504, it may be determined, for each of the at least two video segments, whether the number of interactions of the video segment has a significant difference with respect to the average value based on the average value and the standard deviation.
In some embodiments, for each video segment, if the number of interactions for that video segment is greater than the mean by more than a predefined m standard deviations, the number of interactions for that video segment may be considered to have a significant difference relative to the mean.
In some implementations, for the ith video segment, a standard score for the number of interactions for the video segment relative to the average can be determined based on the standard deviation. For example, the above-mentioned criterion score Z may be determined based on the formula (2)i。
Figure BDA0002449782290000151
Wherein t isiIs the ith video segment TiIs the average of the number of interoperations of the at least two video segments, and σ is the standard deviation of the number of interoperations of the at least two video segments.
The standard score Z obtained based on the formula (2) can be comparediAnd a predefined standard score threshold ZthrTo determine tiWhether there is a significant difference with respect to the mean value μ. In some examples, ZthrMay be set to 2.5, 3.0 or 3.5. When Zi|＞ZthrWhen it is, t can be considerediThere was a significant difference from the mean μ. For example, because when the ith video clip TiIs significantly greater than the average value, the enthusiasm of the user for viewing the video segment can be considered to be high, and therefore, when Z is greateri＞ZthrWhen it is, t can be considerediThere was a significant difference from the mean μ. In the case where it is determined that the number of the interoperations of the multimedia segment has a significant difference with respect to the average value, the interoperation statistical result of the multimedia segment may be determined to be an interoperation statistical result having a significant difference.
Fig. 6 shows a schematic block diagram of an apparatus for marking video hotspots according to an embodiment of the present application.
As shown in fig. 6, apparatus 600 may include a preprocessing unit 610, a statistics unit 620, a verification unit 630, and a hot spot tagging unit 640.
The pre-processing unit 610 may be configured to split the multimedia content to be processed into at least two multimedia fragments. In some embodiments, the number of interactions in the multimedia content to be processed is greater than a predefined number threshold. For example, the number of instant comments in the video to be processed may be greater than 1000. The result of a video hotspot marked according to the principles of the present application will be more accurate when a sufficient number of interactive operations are included in the video to be processed.
In some embodiments, the video to be processed may be segmented at preset time intervals. In some examples, the preset time interval may be 5 s. That is, the time length of each video clip is 5 s.
It will be appreciated that the video to be processed may be segmented in any other manner by those skilled in the art, for example, each video segment may include a preset number of frames. The particular manner in which the video to be processed is segmented is not limited herein.
The statistics unit 620 may be configured to perform statistics on the interoperation associated with each video segment to obtain a statistics result of the interoperation for each video segment.
In some embodiments, the statistics unit 620 may be configured to count the interoperation associated with each multimedia segment to obtain at least two interoperation statistics corresponding to the at least two multimedia segments, respectively.
The interaction may include an instant comment or play progress adjustment associated with each video clip. In some examples, the interaction may include an instant comment (e.g., a bullet screen) or a play progress adjustment (e.g., forward, rewind, etc.) associated with a timestamp in each video clip. In other examples, the interaction may also include a query behavior (e.g., search for video-related characters, plots, etc.) performed by the user while viewing the video content, a behavior of the user clicking on a link on the current user interface while viewing the video content, a screen capture behavior of the user for the current video playback interface, and so on, associated with the timestamp in each video segment. The principles of the present application will be described hereinafter with the interaction being an instant comment as an example.
In some implementations, the statistics of interactions for each video segment can be the number of users that have interacted with and the number of users that have not interacted with within the time period of the video segment. For example, the interoperation statistics for each video segment may be the number of users posting an instant comment within the time period of the video segment and the number of users not posting an instant comment within the time period of the video segment. The number of users who perform interactive operations and the number of users who do not perform interactive operations may refer to an absolute number, i.e., an actual number of users, or may refer to a relative number, i.e., a percentage of users who perform interactive operations among all users and a percentage of users who do not perform interactive operations among all users.
In other implementations, the interoperation statistics for each video segment include a number of interoperations within a time period of the video segment. For example, the interoperation statistics for each video segment may include the number of instant comments posted over the time period for that video segment.
The checking unit 630 may be configured to perform a check of significance of difference on the interaction statistics obtained by the statistics unit 620. The interoperation statistics having significant differences may be determined by performing a test of significance of differences on at least two interoperation statistics. The interoperation statistics having significant differences may be used to indicate whether there is a video segment having significant differences from other video segments among the at least two video segments. For example, the interoperation statistics having a significant difference may indicate that there is a significant difference between the interoperation statistics of a certain video segment and the interoperation statistics of at least another video segment of the at least two video segments. In this case, the video clip may belong to a hotspot clip.
In some embodiments, in the case that the interactive operation statistics of each video segment are the number of users who have performed interactive operations and the number of users who have not performed interactive operations in the time period of the video segment. The test for difference significance may be performed on the interoperation statistics of two adjacent video segments based on the principle of chi-square test. In the case where it is determined that there is a significant difference between the interoperation statistics of the adjacent two multimedia segments, the interoperation statistics of one multimedia segment of the adjacent two multimedia segments may be determined as the interoperation statistics having the significant difference.
The checking unit 630 may be configured to perform a process of performing difference significance on the interactive operation statistics of two adjacent video segments based on the chi-squared checking principle described in conjunction with fig. 4, which is not described herein again.
By using the verification of the significance of the difference based on the principle of chi-square verification, it can be determined that the number of users who perform interactive operations in a certain video clip significantly changes from the number of users who perform interactive operations in the previous video clip of the video clip on the time axis of the video. For example, it may be determined that the number of users posting an instant comment in the video segment is significantly increased or decreased relative to the number of users interacting in the previous video segment based on the principle of chi-square test.
In other embodiments, where the interoperation statistics of each video segment include a number of interoperations within a time period of the video segment, a test of significance of difference may be performed on at least two interoperation statistics based on the principle of Z-score (Z-score).
The checking unit 630 may be configured to perform a process of performing significance difference on at least two interactive operation statistics based on the Z-score principle described in conjunction with fig. 5, which is not described in detail herein.
With the verification of the significance of the difference performed on the basis of the principle of the Z-score, it can be determined whether the number of interactions occurring within a certain video segment time period has a significant difference with respect to the average of the number of interactions occurring within each video segment time period. In a case where it is determined that the number of the interactions of the multimedia segment has a significant difference with respect to the average value, it may be determined that the interaction statistics of the multimedia segment are the interactions statistics having the significant difference.
The hot spot tagging unit 640 may be configured to determine at least one hot spot segment among the at least two multimedia segments based on the interoperation statistics with significant differences obtained by the checking unit 630.
As described above, in the case where the verification of the significance of the difference is performed using the principle of chi-square verification, it can be determined that the number of users who perform an interactive operation in a certain video clip significantly changes, for example, significantly increases or significantly decreases, from the number of users who perform an interactive operation in the previous video clip of the video clip on the time axis of the video. When the number of users interacting in the video clip increases significantly, it means that the video clip indicates the beginning of the hot spot clip. When the number of users interacting in the video clip is significantly reduced, it means that the video clip indicates the end of the hot clip.
In embodiments where the principle of the chi-squared test is used to perform a test of significance of differences, in the event that a significant difference is determined between the statistics of two adjacent video segments, a candidate span designation for a hotspot segment may be determined based on the timestamp of one of the two adjacent video segments. Wherein the candidate interval flag may be used to indicate the beginning or end of a hotspot segment.
For example, the timestamp of any one of the previous or next video clip (e.g., the first or last frame) of the two adjacent video clips may be determined as the candidate span mark of the hotspot clip. For another example, a position where the timestamp of any one of the previous video segment or the next video segment (e.g., the first frame or the last frame) in the two adjacent video segments is subtracted K seconds forward may be determined as the candidate interval flag. This is because it is considered that there is a lag in time from the point in time when the user starts viewing the hit section to the point in time when the user makes an instant comment. K may be any preset value, for example, K — 2.
In some embodiments, the time interval for at least one hotspot segment may be determined based on the order in which the candidate interval markers appear in the video. For example, a 2m-1 occurrence candidate interval marker may be determined to indicate the beginning of a hotspot segment, and a 2m occurrence candidate interval marker may be determined to indicate the end of a hotspot segment, where m is an integer greater than zero. Thus, the time period between the 2m-1 occurrence of the candidate interval marker and the 2m occurrence of the candidate interval marker may be determined as the time interval of the hotspot segment.
In an embodiment where the verification of the significance of the difference is performed using the principle of Z-score, a video segment may be determined as the time interval of a hotspot segment in the case where it is determined that the number of interactions occurring within the time period of the video segment has a significant difference with respect to the average of the number of interactions occurring within the time period of each video segment.
By using the device for labeling the hot spots in the multimedia content, the multimedia segments with different significances in the multimedia content can be determined based on the statistical difference significance test result of the interactive operation in different time periods in the multimedia content, and the hot spot segments in the multimedia content can be determined according to the time stamps of the multimedia segments with different significances. The trending plots determined by the method are more accurate because the statistics of the interactive operations in different time periods in the multimedia content can reflect the user evaluation of the groupwise nature of the user when watching the content in different time periods in the multimedia content.
Furthermore, the method or apparatus according to the embodiment of the present application may also be implemented by means of the architecture of the electronic device shown in fig. 7. Fig. 7 shows the architecture of the electronic device. As shown in fig. 7, the electronic device 700 may include a bus 710, one or at least two CPUs 720, a Read Only Memory (ROM)730, a Random Access Memory (RAM)740, a communication port 750 connected to a network, an input/output component 760, a hard disk 770, and the like. A storage device in the electronic device 700, such as the ROM 730 or the hard disk 770, may store various data or files used in the processing and/or communication of the methods provided herein and program instructions executed by the CPU. Electronic device 700 may also include a user interface 780. Of course, the architecture shown in fig. 7 is merely exemplary, and one or at least two components of the electronic device shown in fig. 7 may be omitted when implementing different devices, according to actual needs.
According to another aspect of the present application, there is also provided a non-transitory computer readable storage medium having stored thereon processor readable instructions which, when executed by a processor, may perform the method as previously described.
Portions of the technology may be considered "articles" or "articles of manufacture" in the form of executable code and/or associated data, which may be embodied or carried out by a computer readable medium. Tangible, non-transitory storage media may include memory or storage for use by any computer, processor, or similar device or associated module. For example, various semiconductor memories, tape drives, disk drives, or any similar device capable of providing a storage function for software.
All or a portion of the software may sometimes communicate over a network, such as the internet or other communication network. Such communication may load software from one computer device or processor to another. For example: from a server or host computer of the video object detection device to a hardware platform of a computer environment, or other computer environment implementing a system, or similar functionality related to providing information needed for object detection. Thus, another medium capable of transferring software elements may also be used as a physical connection between local devices, such as optical, electrical, electromagnetic waves, etc., propagating through cables, optical cables, air, etc. The physical medium used for the carrier wave, such as an electrical, wireless connection or optical cable, and the like, may also be considered to be the medium carrying the software. As used herein, unless limited to a tangible "storage" medium, other terms referring to a computer or machine "readable medium" refer to media that participate in the execution of any instructions by a processor.
This application uses specific words to describe embodiments of the application. Reference to "a first/second embodiment," "an embodiment," and/or "some embodiments" means a feature, structure, or characteristic described in connection with at least one embodiment of the application. Therefore, it is emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, some features, structures, or characteristics of one or more embodiments of the present application may be combined as appropriate.
Moreover, those skilled in the art will appreciate that aspects of the present application may be illustrated and described in terms of several patentable species or situations, including any new and useful combination of processes, machines, manufacture, or materials, or any new and useful improvement thereon. Accordingly, various aspects of the present application may be embodied entirely in hardware, entirely in software (including firmware, resident software, micro-code, etc.) or in a combination of hardware and software. The above hardware or software may be referred to as "data block," module, "" engine, "" unit, "" component, "or" system. Furthermore, aspects of the present application may be represented as a computer product, including computer readable program code, embodied in one or more computer readable media.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The foregoing is illustrative of the present invention and is not to be construed as limiting thereof. Although a few exemplary embodiments of this invention have been described, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this invention. Accordingly, all such modifications are intended to be included within the scope of this invention as defined in the claims. It is to be understood that the foregoing is illustrative of the present invention and is not to be construed as limited to the specific embodiments disclosed, and that modifications to the disclosed embodiments, as well as other embodiments, are intended to be included within the scope of the appended claims. The invention is defined by the claims and their equivalents.

Claims (15)

1. A method for tagging multimedia content hotspots, comprising:
dividing the multimedia content into at least two multimedia segments;
counting the interactive operation associated with each multimedia segment to obtain at least two interactive operation statistical results respectively corresponding to at least two multimedia segments;
performing a test of significance of difference on the at least two interoperation statistical results to determine interoperation statistical results having significant difference; and
determining at least one hotspot segment among the at least two multimedia segments based on the interoperation statistics having the significant differences.
2. The method of claim 1, wherein the interactive operation comprises an instant comment or a play progress adjustment associated with the multimedia clip.
3. The method of claim 1 or 2, wherein the interactive operation statistics of each multimedia segment include the number of users performing the interactive operation and the number of users not performing the interactive operation in the time period of the multimedia segment.
4. The method of claim 3, wherein performing a test of significance of difference on the at least two interoperation statistics to determine interoperation statistics having significant differences comprises:
for a first multimedia segment in two adjacent multimedia segments, determining the number of users who perform the interactive operation in users watching the first multimedia segment as a first user number and the number of users who do not perform the interactive operation as a second user number;
for a second multimedia segment in two adjacent multimedia segments, determining the number of users who perform the interactive operation in users watching the second multimedia segment as a third user number and the number of users who do not perform the interactive operation as a fourth user number;
determining whether the interactive operation statistics of the two adjacent multimedia fragments have a significant difference by using the first user number, the second user number, the third user number and the fourth user number;
in the case that it is determined that there is a significant difference between the interoperation statistics of two adjacent multimedia segments, determining the interoperation statistics of one multimedia segment of the two adjacent multimedia segments as the interoperation statistics having the significant difference.
5. The method of claim 4, wherein determining at least one hotspot segment among the at least two multimedia segments based on the interoperation statistics having significant differences comprises:
determining candidate interval marks of the hotspot segments based on the time stamp of one of the two adjacent multimedia segments;
selecting a time interval between two candidate interval markers as a hotspot segment based on the order in which the candidate interval markers appear in the multimedia content.
6. The method of claim 5, wherein selecting a time interval between two candidate interval markers as a hotspot segment based on the order in which the candidate interval markers appear in the multimedia content comprises:
the time interval between the 2m-1 occurrence of the candidate interval marker and the 2m occurrence of the candidate interval marker is determined as the hotspot segment, where m is an integer greater than zero.
7. The method of claim 1 or 2, wherein the statistics of interactions for each multimedia segment include the number of said interactions over the period of time for that multimedia segment.
8. The method of claim 7, wherein performing a test of significance of difference on the at least two interoperation statistics to determine interoperation statistics having significant differences comprises:
determining an average value and a standard deviation of the number of interactions of the at least two multimedia segments based on the number of interactions of each multimedia segment;
determining, for each of the at least two multimedia segments, based on the average and the standard deviation, whether the number of interactions for that multimedia segment has a significant difference with respect to the average,
and under the condition that the number of the interactive operations of the multimedia segment is determined to have a significant difference relative to the average value, determining the interactive operation statistical result of the multimedia segment as the interactive operation statistical result with the significant difference.
9. The method of claim 8, wherein determining whether the number of interactions of the multimedia segment has a significant difference relative to the mean based on the mean and the standard deviation comprises:
determining a standard score of the interoperation statistical result of the multimedia segment relative to the average value based on the standard deviation;
and determining whether the interactive operation statistical result of the multimedia segment has difference significance based on the standard score.
10. The method of claim 9, wherein determining at least one hotspot segment among the at least two multimedia segments based on the interoperation statistics having significant differences comprises:
for each of the at least two multimedia clips, determining the multimedia clip as a hotspot clip if the criterion score is greater than a predefined criterion score threshold.
11. An apparatus for marking multimedia content hotspots, comprising:
a pre-processing unit configured to segment multimedia content into at least two multimedia segments;
a statistics unit configured to perform statistics on the interoperation associated with each multimedia segment to obtain at least two interoperation statistics corresponding to the at least two multimedia segments, respectively;
a checking unit configured to perform a check of significance of difference on the at least two interoperation statistical results to determine interoperation statistical results having significant difference; and
a hot spot marking unit configured to determine at least one hot spot segment among the at least two multimedia segments based on the interoperation statistics having the significant differences.
12. The apparatus of claim 11, wherein the interactive operation comprises an instant comment or a play progress adjustment associated with the multimedia clip.
13. The apparatus of claim 11 or 12, wherein the interactive operation statistics of each multimedia segment include the number of users performing the interactive operation and the number of users not performing the interactive operation in the time period of the multimedia segment.
14. A multimedia content server, comprising:
a processor; and
a memory having computer-readable program instructions stored therein,
wherein the computer readable program instructions, when executed by the processor, perform the method of any of claims 1-10.
15. A computer readable storage medium having computer readable instructions stored thereon which, when executed by a processor, the processor performs the method of any one of claims 1-10.
CN202010289336.XA 2020-04-14 2020-04-14 Method, device, server and medium for marking multimedia content hot spot Active CN111479168B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010289336.XA CN111479168B (en) 2020-04-14 2020-04-14 Method, device, server and medium for marking multimedia content hot spot

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010289336.XA CN111479168B (en) 2020-04-14 2020-04-14 Method, device, server and medium for marking multimedia content hot spot

Publications (2)

Publication Number Publication Date
CN111479168A true CN111479168A (en) 2020-07-31
CN111479168B CN111479168B (en) 2021-12-28

Family

ID=71752428

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010289336.XA Active CN111479168B (en) 2020-04-14 2020-04-14 Method, device, server and medium for marking multimedia content hot spot

Country Status (1)

Country Link
CN (1) CN111479168B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112653906A (en) * 2020-12-16 2021-04-13 Vidaa美国公司 Video hotspot playing method on display device and display device
CN114630141A (en) * 2022-03-18 2022-06-14 北京达佳互联信息技术有限公司 Video processing method and related equipment

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102486796A (en) * 2010-12-03 2012-06-06 腾讯科技（深圳）有限公司 Method and device for acquiring video information
CN102737036A (en) * 2011-04-07 2012-10-17 腾讯科技（深圳）有限公司 Method and device for acquiring hot spot value words
US20140137151A1 (en) * 2009-10-15 2014-05-15 At&T Intellectual Property I, Lp Apparatus and method for transmitting media content
CN104469508A (en) * 2013-09-13 2015-03-25 中国电信股份有限公司 Method, server and system for performing video positioning based on bullet screen information content
CN104581400A (en) * 2015-02-10 2015-04-29 飞狐信息技术（天津）有限公司 Video content processing method and video content processing device
CN107071587A (en) * 2017-04-25 2017-08-18 腾讯科技（深圳）有限公司 The acquisition methods and device of video segment
CN107666623A (en) * 2017-03-07 2018-02-06 腾讯科技（深圳）有限公司 The methods of exhibiting and device of broadcast information
CN108062409A (en) * 2017-12-29 2018-05-22 北京奇艺世纪科技有限公司 Generation method, device and the electronic equipment of live video summary
CN108874832A (en) * 2017-05-15 2018-11-23 腾讯科技（深圳）有限公司 Target, which is commented on, determines method and device
CN108924576A (en) * 2018-07-10 2018-11-30 武汉斗鱼网络科技有限公司 A kind of video labeling method, device, equipment and medium
CN109104642A (en) * 2018-09-26 2018-12-28 北京搜狗科技发展有限公司 A kind of video generation method and device
CN110248258A (en) * 2019-07-18 2019-09-17 腾讯科技（深圳）有限公司 Recommended method, device, storage medium and the computer equipment of video clip

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140137151A1 (en) * 2009-10-15 2014-05-15 At&T Intellectual Property I, Lp Apparatus and method for transmitting media content
CN102486796A (en) * 2010-12-03 2012-06-06 腾讯科技（深圳）有限公司 Method and device for acquiring video information
CN102737036A (en) * 2011-04-07 2012-10-17 腾讯科技（深圳）有限公司 Method and device for acquiring hot spot value words
CN104469508A (en) * 2013-09-13 2015-03-25 中国电信股份有限公司 Method, server and system for performing video positioning based on bullet screen information content
CN104581400A (en) * 2015-02-10 2015-04-29 飞狐信息技术（天津）有限公司 Video content processing method and video content processing device
CN107666623A (en) * 2017-03-07 2018-02-06 腾讯科技（深圳）有限公司 The methods of exhibiting and device of broadcast information
CN107071587A (en) * 2017-04-25 2017-08-18 腾讯科技（深圳）有限公司 The acquisition methods and device of video segment
CN108874832A (en) * 2017-05-15 2018-11-23 腾讯科技（深圳）有限公司 Target, which is commented on, determines method and device
CN108062409A (en) * 2017-12-29 2018-05-22 北京奇艺世纪科技有限公司 Generation method, device and the electronic equipment of live video summary
CN108924576A (en) * 2018-07-10 2018-11-30 武汉斗鱼网络科技有限公司 A kind of video labeling method, device, equipment and medium
CN109104642A (en) * 2018-09-26 2018-12-28 北京搜狗科技发展有限公司 A kind of video generation method and device
CN110248258A (en) * 2019-07-18 2019-09-17 腾讯科技（深圳）有限公司 Recommended method, device, storage medium and the computer equipment of video clip

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112653906A (en) * 2020-12-16 2021-04-13 Vidaa美国公司 Video hotspot playing method on display device and display device
CN112653906B (en) * 2020-12-16 2023-07-18 Vidaa美国公司 Video hot spot playing method on display equipment and display equipment
CN114630141A (en) * 2022-03-18 2022-06-14 北京达佳互联信息技术有限公司 Video processing method and related equipment

Also Published As

Publication number Publication date
CN111479168B (en) 2021-12-28

Similar Documents

Publication Publication Date Title
JP6891170B2 (en) Future viewing forecasts for video segments to optimize stem resource utilization
CN110210883B (en) Group control account identification method, device, server and storage medium
US20110258211A1 (en) System and method for synchronous matching of media samples with broadcast media streams
CN111479168B (en) Method, device, server and medium for marking multimedia content hot spot
US10097888B2 (en) Determining audience engagement
CN101365102B (en) Audience rating statistical method and system based on video content recognition
CN111090813B (en) Content processing method and device and computer readable storage medium
CN105718861A (en) Method and device for identifying video streaming data category
CN109144858B (en) Fluency detection method and device, computing equipment and storage medium
CN108924381B (en) Image processing method, image processing apparatus, and computer readable medium
CN111314732A (en) Method for determining video label, server and storage medium
EP3989158A1 (en) Method, apparatus and device for video similarity detection
CN111683274A (en) Bullet screen advertisement display method, device and equipment and computer readable storage medium
CN110659604A (en) Video detection method, device, server and storage medium
CN112202849A (en) Content distribution method, content distribution device, electronic equipment and computer-readable storage medium
US20090092313A1 (en) Information processing apparatus and method, program, and recording medium
CN111444362B (en) Malicious picture interception method, device, equipment and storage medium
CN113296666A (en) Anchor exposure data reporting method and device, terminal equipment and storage medium
CN113297417B (en) Video pushing method, device, electronic equipment and storage medium
CN113515670A (en) Method, device and storage medium for identifying state of movie and television resource
CN111259194B (en) Method and apparatus for determining duplicate video
CN113869926A (en) Advertisement identification method and device, electronic equipment and storage medium
CN113408470A (en) Data processing method, data processing apparatus, electronic device, storage medium, and program product
CN110752958A (en) User behavior analysis method, device, equipment and storage medium
CN114005062A (en) Abnormal frame processing method, abnormal frame processing device, server and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40025834

Country of ref document: HK

GR01 Patent grant
GR01 Patent grant