CN111818331B - Method, device, equipment and medium for detecting bullet screen display fluency - Google Patents

Method, device, equipment and medium for detecting bullet screen display fluency Download PDF

Info

Publication number
CN111818331B
CN111818331B CN202010820863.9A CN202010820863A CN111818331B CN 111818331 B CN111818331 B CN 111818331B CN 202010820863 A CN202010820863 A CN 202010820863A CN 111818331 B CN111818331 B CN 111818331B
Authority
CN
China
Prior art keywords
bullet screen
video
screen display
bullet
video image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010820863.9A
Other languages
Chinese (zh)
Other versions
CN111818331A (en
Inventor
周云鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Huya Technology Co Ltd
Original Assignee
Guangzhou Huya Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Huya Technology Co Ltd filed Critical Guangzhou Huya Technology Co Ltd
Priority to CN202010820863.9A priority Critical patent/CN111818331B/en
Publication of CN111818331A publication Critical patent/CN111818331A/en
Application granted granted Critical
Publication of CN111818331B publication Critical patent/CN111818331B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00—Diagnosis, testing or measuring for television systems or their details
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44204—Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/488—Data services, e.g. news ticker
    • H04N21/4884—Data services, e.g. news ticker for displaying subtitles

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the invention discloses a method, a device, equipment and a medium for detecting bullet screen display fluency. The method comprises the following steps: acquiring a recorded video to be detected, and converting the detected video into a plurality of video image frames; wherein at least one bullet screen is displayed in the video to be detected; identifying each video image frame, and determining coordinate position data of at least two bullet screens in a plurality of continuous video image frames respectively; judging whether the at least two bullet screens are smoothly displayed in the video to be detected according to the coordinate position data of the at least two bullet screens in the continuous video image frames respectively; and determining the bullet screen display fluency of the video to be detected according to the judgment result of each bullet screen. According to the technical scheme, the video barrage display fluency can be effectively detected.

Description

Method, device, equipment and medium for detecting bullet screen display fluency
Technical Field
The embodiment of the invention relates to the technical field of bullet screen display, in particular to a bullet screen display fluency detection method, device, equipment and medium.
Background
With the development and the rise of video websites and live broadcast technologies, the concept of "barrage" also starts to be in depth. The bullet screen is a large number of comments displayed in a subtitle form, and all users watching videos can give personal comments to the videos being watched, and the given comments can be displayed on the videos like subtitles, so that all other people can see the comments. In general, in video or live broadcast rooms with high popularity, the barrage amount is quite large, and whether the barrage display is smooth or not directly influences the viewing experience of the user. Therefore, how to detect the fluency of the barrage display in the video is a problem to be solved urgently at present.
Disclosure of Invention
The embodiment of the invention provides a method, a device, equipment and a medium for detecting the fluency of bullet screen display, so as to realize effective detection of the fluency of video bullet screen display.
In a first aspect, an embodiment of the present invention provides a method for detecting fluency of bullet screen display, including:
acquiring a recorded video to be detected, and converting the detected video into a plurality of video image frames; wherein at least one bullet screen is displayed in the video to be detected;
identifying each video image frame, and determining coordinate position data of at least two bullet screens in a plurality of continuous video image frames respectively;
judging whether the at least two bullet screens are smoothly displayed in the video to be detected according to the coordinate position data of the at least two bullet screens in the continuous video image frames respectively;
and determining the bullet screen display fluency of the video to be detected according to the judgment result of each bullet screen.
In a second aspect, an embodiment of the present invention further provides a device for detecting fluency of bullet screen display, where the device includes:
the video conversion module is used for acquiring a recorded video to be detected and converting the video to be detected into a plurality of video image frames; wherein at least one bullet screen is displayed in the video to be detected;
the bullet screen identification module is used for identifying each video image frame and determining coordinate position data of at least two bullet screens in a plurality of continuous video image frames;
the bullet screen smoothness independent judgment module is used for judging whether the at least two bullet screens are displayed smoothly in the video to be detected according to the coordinate position data of the at least two bullet screens in the continuous video image frames respectively;
and the video smoothness comprehensive judgment module is used for determining the bullet screen display smoothness of the video to be detected according to the judgment result of each bullet screen.
In a third aspect, an embodiment of the present invention further provides a computer device, where the computer device includes:
one or more processors;
a memory for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the method for detecting fluency of bullet screen display according to any embodiment.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the method for detecting fluency of bullet screen display according to any embodiment is implemented.
According to the technical scheme provided by the embodiment of the invention, the recorded video to be detected is firstly converted into a plurality of video image frames, then each video image frame is identified to obtain the coordinate position data of at least two bullet screens in the plurality of continuous video image frames respectively, whether the at least two bullet screens are smoothly displayed in the video to be detected is judged according to the coordinate position data, and then the bullet screen display smoothness of the video to be detected can be determined according to the judgment result of each bullet screen, so that the effective detection on the video bullet screen display smoothness is realized. The technical scheme is simple to realize and can be applied to monitoring of the fluency of bullet screen display.
Drawings
Fig. 1 is a flowchart of a method for detecting fluency of bullet screen display according to a first embodiment of the present invention;
fig. 2 is a flowchart of a method for detecting fluency of bullet screen display in the second embodiment of the present invention;
fig. 3 is a flowchart of a method for detecting fluency of bullet screen display in the third embodiment of the present invention;
fig. 4 is a schematic block diagram of a detecting device for pop-up screen fluency according to a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of a computer device in the fifth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart of a method for detecting fluency of bullet screen display according to an embodiment of the present invention. The method can be executed by the detection device for detecting the fluency of the bullet screen display provided by any embodiment of the invention, and the device can be composed of hardware and/or software and can be generally integrated in computer equipment.
As shown in fig. 1, the method for detecting the fluency of bullet screen display provided by the embodiment includes the following steps:
s110, acquiring the recorded video to be detected, and converting the video to be detected into a plurality of video image frames, wherein at least one bullet screen is displayed in the video to be detected.
And the bullet screen refers to a commenting caption popped up when a user watches a video on a network. Wherein, a bullet screen refers to a piece of commenting caption popped up when a user watches videos on the network. The content of the bullet screens delivered by different users may be the same, and the text content of the bullet screens delivered by the same user at different times may also be the same, and these same bullet screens are not one bullet screen and should be considered as multiple bullet screens.
The video to be detected refers to a segment of online video which needs to be subjected to barrage display fluency detection, and the segment of online video can be live video, recorded video and the like. Wherein, a plurality of barrages are displayed in the video to be detected.
In an alternative embodiment, the duration of the video to be detected may be 3-5 minutes, for example, by capturing the video on a screen. This embodiment is not particularly limited, and only the recording and the obtaining of the online video displayed with the plurality of barrages are completed.
After the video to be detected is acquired, the video to be detected is converted into video image frames. For example, the video to be detected is decoded to obtain a plurality of video image frames corresponding to the video to be detected, and the decoding mode is matched with the encoding mode of the time to be detected.
And S120, identifying each video image frame, and determining coordinate position data of at least two bullet curtains in a plurality of continuous video image frames respectively.
And identifying each video image frame, and determining the bullet screen displayed in each video image frame.
In one example, character recognition may be performed on each video image frame to determine a bullet screen in the form of text displayed in each video frame. In another example, image recognition may be performed on each video image frame to determine a bullet screen in the form of a picture displayed in each video. In view of the variety of display forms of the bullet screen, the identification method of the video image frame in this embodiment is not particularly limited, and may be one identification method or a plurality of identification methods.
In this step, the barrage displayed in each video image frame may be identified by identifying all the barrages in the video image frame, or by identifying a part of the barrage in the video image frame, for example, only the barrage in a middle region of the video image frame.
After each video image frame is identified to obtain the bullet screens displayed in each video image frame, the bullet screens are subjected to de-duplication based on the continuity of the bullet screens displayed in different video image frames, and a plurality of bullet screens displayed in the video to be detected can be obtained. The continuity of the bullet screen displayed in different video image frames does not include, but is not limited to, the continuity of the video image frame sequence numbers and the continuity of the display positions in the video image frames.
Optionally, the bullet screen area in each video image frame is identified, and a plurality of bullet screens displayed in each video image frame are determined.
And analyzing the coordinate position data of the plurality of bullet screens displayed in each video image frame, and obtaining the coordinate position data of each bullet screen in the corresponding continuous plurality of video image frames based on the display continuity of the bullet screens in different video image frames.
Taking the bullet screen A as an example, assuming that the bullet screen A can be identified in the 5 th to 14 th video image frames, the coordinate position data of the bullet screen A in the 5 th to 14 th video image frames can be obtained through the step.
Assuming that the text contents of the two bullet screens are the same, such as "" refuel "", if the text "" refuel "" is recognized in the 5 th to 14 th video image frames and the 40 th to 49 th video image frames, it is determined that two bullet screens exist, the text contents of which are both "" -refuel "", and then coordinate position data of one of the bullet screens in the 5 th to 14 th video image frames and coordinate position data of the other bullet screen in the 40 th to 49 th video image frames are obtained. For another example, if a plurality of texts ". about..
S130, judging whether the at least two bullet screens are smoothly displayed in the video to be detected according to coordinate position data of the at least two bullet screens in a plurality of continuous video image frames respectively.
Taking a bullet screen as an example, analyzing difference information between coordinate position data of the bullet screen in a plurality of continuous video image frames to determine whether the bullet screen is displayed smoothly in the video to be detected.
For example, assuming that coordinate position data of the bullet screen a in the 5 th to 14 th video image frames are (x1, y1), (x2, y2), (x3, y3), (x4, y4), (x5, y5), (x6, y6), (x7, y7), (x8, y8), (x9, y9), (x10, y10), if the difference between two adjacent coordinate positions is equal or almost equal, it can be determined that the bullet screen a displays fluency in the video to be detected.
As an optional implementation manner, judging whether at least two bullet screens display fluently in the video to be detected according to coordinate position data of the at least two bullet screens in a plurality of consecutive video image frames respectively includes:
acquiring the quantity of coordinate position data of the at least two bullet screens in a plurality of continuous video image frames and the difference value between the coordinate position data of the at least two bullet screens in two adjacent video image frames;
judging whether the quantity of coordinate position data of each bullet screen in a plurality of continuous video image frames is greater than a set quantity threshold value or not and the difference value between the coordinate position data in two adjacent video image frames meets a preset equivalence condition or not according to each bullet screen; if so, determining that the bullet screen is smoothly displayed in the video to be detected; and if not, determining that the bullet screen is displayed in the video to be detected unsmoothly.
The number of the coordinate position data of each bullet screen in the continuous multiple video image frames refers to how many continuous video image frames each bullet screen has coordinate position data, that is, how many continuous video image frames each bullet screen has appeared.
Assuming that the bullet screen A can be identified in the 5 th to 14 th video image frames, the number of coordinate position data of the bullet screen A in a plurality of consecutive video image frames is 10.
Considering that the text length of each bullet screen is different, the moving speed in the video to be detected is different, and further, the frequency of each bullet screen acquired in each video frame image obtained by decoding the video to be detected is also different. Therefore, in the present embodiment, a quantity threshold is introduced to measure the frequency of each bullet screen being collected. If a bullet screen is stuck in the video to be displayed, the frequency of capturing the bullet screen in the video frame image is small, and the number of coordinate position data in a plurality of consecutive video frame images is small.
Furthermore, in this embodiment, when determining whether a bullet screen is displayed smoothly in the video to be detected, it is necessary to comprehensively determine whether the number of coordinate position data of the bullet screen in a plurality of consecutive video image frames and a difference value between the coordinate position data of the bullet screen in two adjacent video image frames, if two determination conditions, namely "the number of coordinate position data in a plurality of consecutive video image frames is greater than a set number threshold" and "the difference value between the coordinate position data in two adjacent video image frames satisfies a preset equivalence condition", are simultaneously satisfied, it is determined that the bullet screen is displayed smoothly in the video to be detected, and if the two determination conditions cannot be simultaneously satisfied, it is determined that the bullet screen is displayed not smoothly in the video to be detected. Wherein the preset equivalence conditions may refer to being completely equal or approximately equal.
S140, determining the bullet screen display fluency of the video to be detected according to the judgment result of each bullet screen.
After a judgment result of whether each bullet screen displays fluency in the video to be detected is obtained, a conclusion of whether the bullet screen of the video to be detected displays fluency can be obtained according to the judgment result of each bullet screen.
For example, for all detected bullet screens, if the judgment result of the bullet screen larger than the preset ratio (e.g., 5%) is that the display in the video to be detected is not smooth, it is judged that the display of the bullet screen in the video to be detected is not smooth.
As an optional implementation manner, determining the bullet screen display fluency of the video to be detected according to the judgment result of each bullet screen, includes: and if at least one bullet screen is displayed in the video to be detected unsmoothly, determining that the video to be detected is displayed unsmoothly on the bullet screen.
In this embodiment, as long as there is a barrage to detect and detect that video barrage shows not smooth in waiting to detect the video, then judge that it is not smooth to detect video barrage display, and it is smooth only to detect every barrage that detects all shows in waiting to detect the video, just judges that it shows smooth to detect video barrage to this has improved and has detected the smooth judgement standard of video barrage display.
According to the technical scheme provided by the embodiment of the invention, the recorded video to be detected is firstly converted into a plurality of video image frames, then each video image frame is identified to obtain the coordinate position data of at least two bullet screens in the plurality of continuous video image frames respectively, whether the at least two bullet screens are smoothly displayed in the video to be detected is judged according to the coordinate position data, and then the bullet screen display smoothness of the video to be detected can be determined according to the judgment result of each bullet screen, so that the effective detection on the video bullet screen display smoothness is realized. The technical scheme is simple to realize and can be applied to monitoring of the fluency of bullet screen display.
Further, in an optional implementation manner, after determining that the video bullet screen to be detected is displayed fluently, the method further includes: and determining the bullet screen display optimization direction of the video to be detected according to the quantity of the coordinate position data of at least one bullet screen which is displayed in the video to be detected and is not fluent in a plurality of continuous video image frames and the difference value between the coordinate position data of two adjacent video image frames.
And after the video bullet screen to be detected is determined to be displayed unsmoothly, acquiring and analyzing various bullet screens which are displayed unsmoothly in the video to be detected. For each bullet screen, the number of coordinate position data of each bullet screen in a plurality of continuous video image frames can be analyzed, namely the number of the coordinate position data of each bullet screen appears in the continuous video image frames, and the difference value between the coordinate position data of each bullet screen in two adjacent video image frames can also be analyzed. And counting the analysis result of each bullet screen, and determining the bullet screen display optimization direction of the video to be detected according to the statistical result, for example, the number of coordinate position data of each bullet screen in a plurality of continuous video image frames can be increased, and the difference value between the coordinate position data of each bullet screen in two adjacent video image frames can be equalized.
In the technical scheme, the bullet screen display fluency is detected to be detected, and the obtained detection result can provide an optimized direction for the development of bullet screen display so as to improve the bullet screen display fluency.
Example two
Fig. 2 is a flowchart of a method for detecting pop-up screen display fluency according to a second embodiment of the present invention. The present embodiment is embodied on the basis of the above embodiment, wherein identifying each video image frame and determining coordinate position data of at least two bullet screens in a plurality of consecutive video image frames may specifically be:
performing character recognition on each video image frame, and determining each bullet screen display text and coordinate position information of each bullet screen display text included in each video image frame;
grouping all bullet screen display texts according to a preset first direction coordinate range to obtain a plurality of groups of bullet screen display text sets; the first direction is the vertical direction of the moving direction of the bullet screen in the video to be detected;
classifying each group of bullet screen display text sets according to the text content and the frame number of the video image to which the bullet screen display text sets belong to obtain bullet screen display text subsets corresponding to the bullet screens;
and correspondingly taking the coordinate position data of each bullet screen display text included in the bullet screen display text subset corresponding to each bullet screen as the coordinate position data of each bullet screen in a plurality of continuous video image frames.
As shown in fig. 2, the method for detecting the fluency of bullet screen display provided by the embodiment includes the following steps:
s210, acquiring the recorded video to be detected, and converting the video to be detected into a plurality of video image frames, wherein at least one bullet screen is displayed in the video to be detected.
S220, performing character recognition on each video image frame, and determining each bullet screen display text and coordinate position information of each bullet screen display text included in each video image frame.
The bullet screen display text refers to bullet screen text displayed in one of the video image frames.
As a bullet screen moves in the video to be detected, a bullet screen display text corresponding to the bullet screen displayed in the consecutive video image frames can be obtained. That is, a plurality of identical bullet screen display texts obtained by character recognition of a plurality of consecutive video image frames are actually one bullet screen.
The coordinate position information of the bullet screen display text refers to coordinate position information of the bullet screen display text in the video image frame, and may be, for example, coordinate position information of a first character (or a character) of the bullet screen display text in the coordinate position information of the video image frame, where the coordinate position information may be represented in the form of (x, y).
And S230, grouping the bullet screen display texts according to a preset first direction coordinate range to obtain a plurality of groups of bullet screen display text sets.
The first direction is the vertical direction of the moving direction of the bullet screen in the video to be detected. For example, if the moving direction of the bullet screen in the video to be detected is a horizontal direction (e.g., from right to left), the first direction is a vertical direction; for another example, if the moving direction of the bullet screen in the video to be detected is a vertical direction (e.g., from bottom to top), the first direction is a horizontal direction.
The first direction coordinate range refers to a coordinate range of a line of bullet screen areas in the video image frame in the first direction. For example, the moving direction of the bullet screens in the video image frame is the horizontal direction, the bullet screen display area in the video image frame can display 5 lines of bullet screens, the coordinate range of the first line of bullet screens in the vertical direction (i.e. the first direction) is y1-y2, the coordinate range of the second line of bullet screens in the vertical direction is y2-y3, the coordinate range of the third line of bullet screens in the vertical direction is y3-y4, the coordinate range of the fourth line of bullet screens in the vertical direction is y4-y5, the coordinate range of the fifth line of bullet screens in the vertical direction is y5-y6, and at this time, the coordinate ranges y1-y2, y2-y3, y3-y4, y4-y5, and y5-y6 in the vertical direction are several preset coordinate ranges in the first direction.
And grouping the bullet screen display texts according to a plurality of preset first direction coordinate ranges to obtain a plurality of groups of bullet screen display text sets, wherein the first direction coordinate ranges of the bullet screen display texts in the video image frames in each group of bullet screen display text sets are the same.
Continuing with the previous example, this step may obtain 5 groups of barrage display text sets: each bullet screen display text in the 1 st group of bullet screen display text sets belongs to a first line of bullet screens in the video image frame, and the coordinate range of the first direction is y1-y 2; each bullet screen display text in the group 2 bullet screen display text set belongs to a second row of bullet screens in the video image frame, and the coordinate range in the first direction is y2-y 3; and so on.
And S240, classifying each group of bullet screen display text sets according to the text content and the frame number of the video image to which the bullet screen display text sets belong to obtain bullet screen display text subsets corresponding to the bullet screens.
Analyzing the text content and the video image frame number of each bullet screen display text included in each group of bullet screen display text sets, respectively combining a plurality of bullet screen display texts with the same text content and continuous video image frame numbers to form each bullet screen display text subset, and determining a bullet screen corresponding to each bullet screen display text subset according to the text content. And each bullet screen display text in the bullet screen display text subset is arranged according to the sequence number of the video image frame.
It is worth noting that as the barrage moves in the video to be detected, the barrage gradually disappears in the video to be detected, so that it may be recognized that the barrage display text corresponds to an incomplete barrage in the video image frame, at this time, the barrage display text subsets corresponding to the incomplete barrages may be obtained, and the number of the barrage display texts in the barrage display text subsets is usually very small, so that the barrage display text subsets may be ignored without further processing.
As an optional implementation manner, after obtaining the bullet screen display text subsets corresponding to the respective bullet screens, the method further includes: and performing duplicate removal processing on the bullet screen display text subset according to the coordinate position data of the bullet screen display text.
And after the bullet screen display text subsets corresponding to the bullet screens are obtained, carrying out duplication elimination processing on the corresponding bullet screen display text subsets according to the coordinate position data of the bullet screen display texts in each bullet screen display text subset. That is, the bullet screen display texts with the same coordinate position data in the bullet screen display text subset are deduplicated, and only one bullet screen display text is reserved. Optionally, a bullet screen display text with the smallest video image frame sequence number is reserved.
The advantages of such an arrangement are: aiming at the condition that the recording frame rate of the video to be detected is greater than the picture frame rate of the video to be detected, the phenomenon that a plurality of video image frames obtained by decoding the video to be detected are repeated exists, and the bullet screen display text subsets are subjected to de-duplication processing, so that the problem that bullet screen display smoothness judgment is inaccurate due to the existence of repeated data can be solved.
And S250, correspondingly taking the coordinate position data of each bullet screen display text included in the bullet screen display text subset corresponding to each bullet screen as the coordinate position data of each bullet screen in a plurality of continuous video image frames.
The bullet screen display text subsets corresponding to the bullet screens comprise bullet screen display texts, namely bullet screen display texts of the bullet screens in a plurality of continuous video image frames, and further the bullet screen display text subsets comprise coordinate position data of the bullet screen display texts, namely the coordinate position data of the corresponding bullet screen in the plurality of continuous video image frames.
As an alternative embodiment, correspondingly using the coordinate position data of each bullet screen display text included in the bullet screen display text subset corresponding to each bullet screen as the coordinate position data of each bullet screen in a plurality of consecutive video image frames includes:
correspondingly taking coordinate position data in the second direction of each bullet screen display text included in the bullet screen display text subsets corresponding to the bullet screens as the coordinate position data of each bullet screen in a plurality of continuous video image frames; wherein the second direction is perpendicular to the first direction.
For example, when the moving direction of the bullet screen in the video to be detected is a horizontal direction (e.g., from right to left), the first direction is a vertical direction, and the second direction is a horizontal direction, the x coordinate of each bullet screen display text included in the bullet screen display text subset corresponding to each bullet screen in the horizontal direction may be used as the coordinate position data of each bullet screen in the continuous multiple video image frames.
For another example, when the moving direction of the bullet screen in the video to be detected is a vertical direction (e.g., from bottom to top), the first direction is a horizontal direction, and the second direction is a vertical direction, then the y coordinate of each bullet screen display text included in the bullet screen display text subset corresponding to each bullet screen in the vertical direction may be used as the coordinate position data of each bullet screen in the continuous multiple video image frames.
S260, acquiring the number of bullet screen display texts in the bullet screen display text subsets corresponding to the bullet screens and the difference value between the coordinate position data of the bullet screens in the two adjacent video image frames.
The number of the bullet screen display texts in the bullet screen display text subset corresponding to one bullet screen is the number of the coordinate position data of the bullet screen in a plurality of continuous video image frames. The larger the number of the bullet screen display texts in the bullet screen display text subset is, the higher the frequency of the bullet screen in the video to be detected is, and the smoother the bullet screen is displayed in the video to be detected.
When the coordinate position data of each bullet screen in a plurality of continuous video image frames exist in the form of (x, y) coordinates, the difference value between the coordinate position data is the distance between two coordinate points; when the coordinate position data of each bullet screen in a plurality of continuous video image frames exist in the form of x coordinates or y coordinates, the difference value between the coordinate position data is the difference value between corresponding coordinate values.
S270, judging whether the number of bullet screen display texts in the corresponding bullet screen display text subsets is larger than a set number threshold value or not according to each bullet screen, wherein the difference value between coordinate position data in two adjacent video image frames meets a preset equivalent condition; if so, determining that the bullet screen is smoothly displayed in the video to be detected; and if not, determining that the bullet screen is displayed in the video to be detected unsmoothly.
S280, if at least one bullet screen is displayed in the video to be detected to be unsmooth, determining that the display of the bullet screen of the video to be detected is unsmooth.
S290, determining the bullet screen display optimization direction of the video to be detected according to the number of coordinate position data of at least one bullet screen which is displayed in the video to be detected and is not fluent in a plurality of continuous video image frames and the difference value between the coordinate position data of two adjacent video image frames.
For those parts of this embodiment that are not explained in detail, reference is made to the aforementioned embodiments, which are not repeated herein.
According to the technical scheme, effective detection of the bullet screen display fluency of the video to be detected is achieved, the realization is simple, and the method and the device can be applied to monitoring of the bullet screen display fluency. Moreover, the bullet screen display fluency detection result of the to-be-detected video can also provide an optimized direction for the development of bullet screen display so as to further improve the fluency of bullet screen display.
EXAMPLE III
Fig. 3 is a flowchart of a method for detecting pop-up screen display fluency according to a third embodiment of the present invention. The embodiment provides a specific implementation manner on the basis of the above embodiment, wherein the video to be detected is a live video, and each barrage moves from right to left in the live video at a certain speed.
As shown in fig. 3, the method for detecting the fluency of bullet screen display provided by the embodiment includes the following steps:
s310, acquiring the recorded video to be detected, and decoding the video to be detected into a plurality of video image frames, wherein at least one bullet screen is displayed in the video to be detected.
And entering a video live broadcast room, opening a bullet screen function, and recording live broadcast video to obtain recorded video to be detected.
And S320, performing character recognition on each video image frame, and determining each bullet screen display text and the coordinate position information of each bullet screen display text included in each video image frame.
Optionally, each bullet screen displays coordinate position information of the text as (x)i,yi) The subscript i is the serial number of the video image frame.
S330, grouping the bullet screen display texts according to a preset y coordinate range to obtain a plurality of groups of bullet screen display text sets.
Each group of bullet screen display text sets respectively correspond to a y coordinate range, namely, the ordinate of each bullet screen display text in each group of bullet screen display text sets belongs to the same y coordinate range. Optionally, the coordinate position information of each bullet screen display text in each group of bullet screen display text sets is represented by an abscissa xiThe subscript i is the serial number of the video image frame.
S340, classifying each group of bullet screen display text sets according to the text content and the frame number of the video image to obtain bullet screen display text subsets corresponding to each bullet screen, and performing duplicate removal processing on the bullet screen display text subsets according to the x coordinates of the bullet screen display texts.
Optionally, the bullet screen display texts in each group of bullet screen display text sets are arranged according to the video image frame number. In each group of bullet screen display text sets, only the first bullet screen display text with the same x coordinate is reserved.
Similar to the bullet screen display text set and the bullet screen display text subset, the bullet screen display texts included in the video image frames obtained by character recognition of each video image frame and the coordinate position information of each bullet screen display text may also be stored in a set form, and the representation form of the set is not particularly limited in this embodiment.
And S350, taking the x coordinate of each bullet screen display text included in the bullet screen display text subset corresponding to each bullet screen as the coordinate position data of each bullet screen in a plurality of continuous video image frames.
S360, obtaining the number of bullet screen display texts in the bullet screen display text subsets corresponding to each bullet screen and the difference value between the coordinate position data of each bullet screen in the two adjacent video image frames.
S370, judging whether the number of bullet screen display texts in the corresponding bullet screen display text subsets is larger than a set number threshold value or not and the difference value between coordinate position data in two adjacent video image frames meets a preset equivalent condition for each bullet screen; if so, determining that the bullet screen is smoothly displayed in the video to be detected; and if not, determining that the bullet screen is displayed in the video to be detected unsmoothly.
And S380, if at least one bullet screen is displayed in the video to be detected unsmoothly, determining that the display of the bullet screen of the video to be detected is unsmoothly.
And S390, determining the bullet screen display optimization direction of the video to be detected according to the number of the coordinate position data of at least one bullet screen which is displayed in the video to be detected and is not fluent in a plurality of continuous video image frames and the difference value between the coordinate position data of two adjacent video image frames.
For those parts of this embodiment that are not explained in detail, reference is made to the aforementioned embodiments, which are not repeated herein.
Example four
Fig. 4 is a schematic block diagram of a detecting device for pop-up screen display fluency according to a fourth embodiment of the present invention. The device can be realized in a software and/or hardware mode, and can be generally integrated in computer equipment. As shown in fig. 4, the apparatus includes: the system comprises a video conversion module 410, a bullet screen identification module 420, a bullet screen smoothness independent judgment module 430 and a video smoothness comprehensive judgment module 440. Wherein the content of the first and second substances,
a video conversion module 410 configured to obtain a recorded video to be detected and convert the detected video into a plurality of video image frames; wherein at least one bullet screen is displayed in the video to be detected;
the bullet screen identification module 420 is configured to identify each video image frame and determine coordinate position data of at least two bullet screens in a plurality of continuous video image frames respectively;
the bullet screen smoothness individual judgment module 430 is configured to judge whether the at least two bullet screens are displayed smoothly in the video to be detected according to coordinate position data of the at least two bullet screens in the continuous multiple video image frames respectively;
and the video fluency comprehensive judgment module 440 is configured to determine the bullet screen display fluency of the video to be detected according to the judgment result of each bullet screen.
According to the technical scheme provided by the embodiment of the invention, the recorded video to be detected is firstly converted into a plurality of video image frames, then each video image frame is identified to obtain the coordinate position data of at least two bullet screens in the plurality of continuous video image frames respectively, whether each bullet screen is displayed smoothly in the video to be detected is judged according to the coordinate position data, and then the bullet screen display smoothness of the video to be detected can be determined according to the judgment result of each bullet screen, so that the effective detection on the display smoothness of the video bullet screen is realized.
In an alternative embodiment, the bullet screen fluency individual judging module 430 is configured to obtain the number of coordinate position data of the bullet screen in a plurality of consecutive video image frames and the difference between the coordinate position data in two adjacent video image frames; for each bullet screen, judging whether the number of coordinate position data of the bullet screen in a plurality of continuous video image frames is greater than a set number threshold value or not, and the difference value between the coordinate position data in two adjacent video image frames meets a preset equivalence condition; if so, determining that the bullet screen is smoothly displayed in the video to be detected; and if not, determining that the bullet screen is displayed in the video to be detected unsmoothly.
In an alternative embodiment, the bullet screen identification module 420 includes:
the character recognition unit is used for performing character recognition on each video image frame and determining each bullet screen display text and coordinate position information of each bullet screen display text included in each video image frame;
the first bullet screen display text grouping unit is used for grouping the bullet screen display texts according to a preset first direction coordinate range to obtain a plurality of groups of bullet screen display text sets; the first direction is the vertical direction of the moving direction of the bullet screen in the video to be detected;
the second grouping unit of the bullet screen display texts is set to classify each group of bullet screen display text sets according to the text content and the frame number of the video image to which the bullet screen display text sets belong, so that bullet screen display text subsets corresponding to the bullet screens are obtained;
and the bullet screen coordinate position determining unit is arranged to correspondingly take the coordinate position data of each bullet screen display text included in the bullet screen display text subsets corresponding to each bullet screen as the coordinate position data of each bullet screen in a plurality of continuous video image frames.
In an alternative embodiment, the bullet screen coordinate position determining unit is configured to correspondingly take coordinate position data in the second direction of each bullet screen display text included in the bullet screen display text subset corresponding to each bullet screen as coordinate position data of each bullet screen in a plurality of continuous video image frames; wherein the second direction is perpendicular to the first direction.
In an alternative embodiment, the bullet screen identification module 420 further includes:
and the duplication elimination processing unit is used for carrying out duplication elimination processing on the bullet screen display text subsets according to the coordinate position data of the bullet screen display texts after the bullet screen display text subsets corresponding to the bullet screens are obtained.
In an optional implementation manner, the video smoothness comprehensive judgment module 440 is configured to determine that the video bullet screen to be detected is displayed not smoothly if at least one bullet screen is displayed not smoothly in the video to be detected.
In an optional embodiment, the apparatus further comprises: and the optimization direction determining module is used for determining the bullet screen display optimization direction of the video to be detected according to the quantity of coordinate position data of at least one bullet screen displaying unsmooth in the video to be detected in a plurality of continuous video image frames and the difference value between the coordinate position data in two adjacent video image frames after the bullet screen of the video to be detected is determined to be displayed unsmoothly.
The detection device for bullet screen display fluency provided by the embodiment of the invention can execute the detection method for bullet screen display fluency provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
EXAMPLE five
Fig. 5 is a schematic structural diagram of a computer apparatus according to a fifth embodiment of the present invention, as shown in fig. 5, the computer apparatus includes a processor 50, a memory 51, an input device 52, and an output device 53; the number of processors 50 in the computer device may be one or more, and one processor 50 is taken as an example in fig. 5; the processor 50, the memory 51, the input device 52 and the output device 53 in the computer apparatus may be connected by a bus or other means, and the connection by the bus is exemplified in fig. 5.
The memory 51 is used as a computer-readable storage medium, and can be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the method for detecting fluency of pop-up display in the embodiment of the present invention (for example, the video conversion module 410, the pop-up identification module 420, the pop-up fluency individual determination module 430, and the video fluency comprehensive determination module 440 in the device for detecting fluency of pop-up display shown in fig. 4). The processor 50 executes various functional applications and data processing of the computer device by running software programs, instructions and modules stored in the memory 51, so as to implement the method for detecting fluency of bullet screen display.
The memory 51 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the computer device, and the like. Further, the memory 51 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 51 may further include memory located remotely from the processor 50, which may be connected to a computer device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 52 is operable to receive input numeric or character information and to generate key signal inputs relating to user settings and function controls of the computer apparatus. The output device 53 may include a display device such as a display screen.
EXAMPLE six
An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, where the computer program is executed by a computer processor to perform a method for detecting fluency of bullet screen display, where the method includes:
acquiring a recorded video to be detected, and converting the detected video into a plurality of video image frames; wherein at least one bullet screen is displayed in the video to be detected;
identifying each video image frame, and determining coordinate position data of at least two bullet screens in a plurality of continuous video image frames;
judging whether the at least two bullet screens are smoothly displayed in the video to be detected according to the coordinate position data of the at least two bullet screens in the continuous video image frames respectively;
and determining the bullet screen display fluency of the video to be detected according to the judgment result of each bullet screen.
Of course, the computer program of the computer-readable storage medium storing the computer program provided in the embodiment of the present invention is not limited to the above method operations, and may also perform related operations in the method for detecting fluency of bullet screen display provided in any embodiment of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods of the embodiments of the present invention.
It should be noted that, in the embodiment of the device for detecting fluency of bullet screen display, each unit and each module included in the embodiment are only divided according to functional logic, but are not limited to the above division, as long as the corresponding function can be realized; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments illustrated herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims (10)

1. A method for detecting barrage display fluency is characterized by comprising the following steps:
acquiring a recorded video to be detected, and converting the video to be detected into a plurality of video image frames; wherein at least one bullet screen is displayed in the video to be detected;
identifying each video image frame, and determining coordinate position data of at least two bullet screens in a plurality of continuous video image frames respectively;
judging whether the at least two bullet screens are smoothly displayed in the video to be detected according to the coordinate position data of the at least two bullet screens in the continuous video image frames respectively;
and determining the bullet screen display fluency of the video to be detected according to the judgment result of each bullet screen.
2. The method of claim 1, wherein determining whether the at least two bullet screens are smoothly displayed in the video to be detected according to the coordinate position data of the at least two bullet screens in a plurality of consecutive video image frames respectively comprises:
acquiring the quantity of coordinate position data of the at least two bullet screens in a plurality of continuous video image frames and the difference value between the coordinate position data of the at least two bullet screens in two adjacent video image frames;
for each bullet screen, judging whether the number of coordinate position data of the bullet screen in a plurality of continuous video image frames is greater than a set number threshold value or not, and the difference value between the coordinate position data in two adjacent video image frames meets a preset equivalence condition;
if so, determining that the bullet screen is smoothly displayed in the video to be detected;
and if not, determining that the bullet screen is displayed in the video to be detected unsmoothly.
3. The method of claim 1 or 2, wherein identifying each of the video image frames and determining coordinate position data of at least two bullet screens in a plurality of consecutive video image frames comprises:
performing character recognition on each video image frame, and determining each bullet screen display text and coordinate position information of each bullet screen display text included in each video image frame;
grouping the bullet screen display texts according to a preset first direction coordinate range to obtain a plurality of groups of bullet screen display text sets; the first direction is the vertical direction of the moving direction of the bullet screen in the video to be detected;
classifying each group of bullet screen display text sets according to the text content and the frame number of the video image to which the bullet screen display text sets belong to obtain bullet screen display text subsets corresponding to the bullet screens;
and correspondingly taking the coordinate position data of each bullet screen display text included in the bullet screen display text subset corresponding to each bullet screen as the coordinate position data of each bullet screen in a plurality of continuous video image frames.
4. The method according to claim 3, wherein correspondingly regarding coordinate position data of each bullet screen display text included in the bullet screen display text subset corresponding to each bullet screen as coordinate position data of each bullet screen in a plurality of consecutive video image frames comprises:
correspondingly taking coordinate position data in the second direction of each bullet screen display text included in the bullet screen display text subsets corresponding to the bullet screens as the coordinate position data of each bullet screen in a plurality of continuous video image frames;
wherein the second direction is perpendicular to the first direction.
5. The method of claim 3, after obtaining the subset of bullet screen display text corresponding to each bullet screen, further comprising:
and carrying out duplicate removal processing on the bullet screen display text subset according to the coordinate position data of the bullet screen display text.
6. The method according to claim 1 or 2, wherein determining the bullet screen display fluency of the video to be detected according to the judgment result of each bullet screen comprises:
and if at least one bullet screen is displayed in the video to be detected unsmoothly, determining that the video to be detected is displayed unsmoothly on the bullet screen.
7. The method of claim 6, further comprising, after determining that the video bullet screen display to be detected is not smooth:
and determining the bullet screen display optimization direction of the video to be detected according to the quantity of coordinate position data of at least one bullet screen which is displayed in the video to be detected and is not fluent in a plurality of continuous video image frames and the difference value between the coordinate position data of two adjacent video image frames.
8. The utility model provides a detection apparatus for barrage shows fluency which characterized in that includes:
the video conversion module is used for acquiring a recorded video to be detected and converting the video to be detected into a plurality of video image frames; wherein at least one bullet screen is displayed in the video to be detected;
the bullet screen identification module is used for identifying each video image frame and determining coordinate position data of at least two bullet screens in a plurality of continuous video image frames;
the bullet screen smoothness independent judgment module is used for judging whether the at least two bullet screens are displayed smoothly in the video to be detected according to the coordinate position data of the at least two bullet screens in the continuous video image frames respectively;
and the video smoothness comprehensive judgment module is used for determining the bullet screen display smoothness of the video to be detected according to the judgment result of each bullet screen.
9. A computer device, characterized in that the computer device comprises:
one or more processors;
a memory for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method for detecting fluency of bullet screen display according to any of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method for detecting fluency of bullet-screen display according to any of claims 1-7.
CN202010820863.9A 2020-08-14 2020-08-14 Method, device, equipment and medium for detecting bullet screen display fluency Active CN111818331B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010820863.9A CN111818331B (en) 2020-08-14 2020-08-14 Method, device, equipment and medium for detecting bullet screen display fluency

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010820863.9A CN111818331B (en) 2020-08-14 2020-08-14 Method, device, equipment and medium for detecting bullet screen display fluency

Publications (2)

Publication Number Publication Date
CN111818331A CN111818331A (en) 2020-10-23
CN111818331B true CN111818331B (en) 2021-11-23

Family

ID=72860209

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010820863.9A Active CN111818331B (en) 2020-08-14 2020-08-14 Method, device, equipment and medium for detecting bullet screen display fluency

Country Status (1)

Country Link
CN (1) CN111818331B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113225606B (en) * 2021-04-30 2022-09-23 上海哔哩哔哩科技有限公司 Video barrage processing method and device

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8073915B1 (en) * 2008-05-16 2011-12-06 Google Inc. Detection of chain-letters in user-generated comments for websites
US20180132007A1 (en) * 2016-11-09 2018-05-10 International Business Machines Corporation Crowdsourced text captioning corrections and augmentation
CN108449609B (en) * 2018-02-09 2020-03-06 广州虎牙信息科技有限公司 Live broadcast room event identification method and device, electronic equipment and machine readable medium
CN109413450B (en) * 2018-09-30 2021-04-27 武汉斗鱼网络科技有限公司 Bullet screen data integrity verification method and device, terminal and storage medium
CN109413506B (en) * 2018-09-30 2021-06-11 武汉斗鱼网络科技有限公司 Bullet screen data verification method and device, terminal and storage medium

Also Published As

Publication number Publication date
CN111818331A (en) 2020-10-23

Similar Documents

Publication Publication Date Title
US10075742B2 (en) System for social media tag extraction
TWI399973B (en) Methods, apparatus and machine readable medium storing instructions to distinguish a signal originating from a local device from a boradcast signal
CN108769776B (en) Title subtitle detection method and device and electronic equipment
CN107295362B (en) Live broadcast content screening method, device and equipment based on image and storage medium
CN104618803A (en) Information push method, information push device, terminal and server
CN105933783A (en) Bullet screen play method and device and terminal equipment
CN111095939B (en) Identifying previously streamed portions of media items to avoid repeated playback
CN111124888B (en) Method and device for generating recording script and electronic device
CN105657547B (en) A kind of detection method and device of similar video and pirate video
CN105718861A (en) Method and device for identifying video streaming data category
WO2021088422A1 (en) Application message notification method and device
CN112153373A (en) Fault identification method and device for bright kitchen range equipment and storage medium
CN109729429A (en) Video broadcasting method, device, equipment and medium
CN112752158A (en) Video display method and device, electronic equipment and storage medium
CN111818331B (en) Method, device, equipment and medium for detecting bullet screen display fluency
CN111401238A (en) Method and device for detecting character close-up segments in video
CN107797784B (en) Method and device for acquiring adaptive resolution of splicing processor
CN114550079A (en) Image processing method, device, equipment and storage medium
US20140012792A1 (en) Systems and methods for building a virtual social network
CN101339662B (en) Method and device for creating video frequency feature data
US10631050B2 (en) Determining and correlating visual context on a user device with user behavior using digital content on the user device
CN112818984B (en) Title generation method, device, electronic equipment and storage medium
CN110414471B (en) Video identification method and system based on double models
TWI466544B (en) Marquee characters aquisition system and method thereof
CN104735531A (en) Image quality analysis based 3D signal automatic identification method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant